Skip to content

[RESEARCH]: Hot-shard skew benchmark (Zipfian theta where per-shard eviction loss exceeds remap cost) #170

@ELares

Description

@ELares

Empirical follow-up split from #32 (closed by ADR-0002/0004/0005). #32 concluded on paper that under shared-nothing (ADR-0002) the owned per-shard store needs no concurrent reclamation machinery (ADR-0004/0005), and that hot-shard load skew is mitigated by detection plus routing/migration rather than shared state. The remaining question is empirical and cannot be settled on paper.

Question

At what Zipfian skew (theta) and key-cardinality does per-shard eviction's hit-rate loss exceed the cost of remapping/splitting a hot shard? Does cross-shard hot-key detection need a shared frequency sketch, and if so can it be shared without contention?

Done when

  • A benchmark on the harness ([DESIGN]: Reproducible benchmark and memory-model harness #8) sweeps theta and reports per-shard vs global-policy hit-rate and tail latency under skew.
  • A recommendation on hot-key detection/mitigation (per-shard counters vs shared sketch vs key-splitting) with measured overhead.
  • Results recorded; ADR-0002/0005 amended only if the data contradicts the shard-per-core stance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:concurrencyArea: concurrencyarea:performanceArea: performanceresearchComparative research grounded in prior-art systems

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions