diff --git a/docs/CHARTER.md b/docs/CHARTER.md new file mode 100644 index 0000000..179de3f --- /dev/null +++ b/docs/CHARTER.md @@ -0,0 +1,72 @@ +# IronCache charter + +This is the charter for IronCache. It fixes the thesis, ratifies the five +tenets and their strict conflict order, and points to the documents that govern +everything downstream. The vision EPIC (#1) is the canonical statement; this +charter is the governing reference. When a design issue and this charter +disagree, the charter wins until amended here. + +## Thesis + +The most efficient Redis-wire-compatible cache in the world, shipped as a single +static Rust binary that is both the server and the CLI. Efficient is the reason +to exist; Compatible is the constraint that makes it useful. We borrow the +shared-nothing thread-per-core shape that lets one process beat a single-threaded +keyspace, and we measure ourselves honestly per core, not by fat-box aggregate +QPS. + +## The five tenets + +Ranked, with conflicts resolved strictly in this order: + +**Compatible > Efficient > Simple > Scalable > AI-Driven.** + +- **Compatible.** RESP2 and RESP3 wire format and observable command semantics, + measured against a pinned Valkey rather than relicensed Redis. Valkey 8.x is + wire-identical to Redis 7.2 RESP2/RESP3 [valkey-resp-identical] and is BSD-3, + so it is a license-safe conformance oracle. A command is either supported with + Valkey-identical observable behavior or documented as unsupported. We never + bend the wire or a command's behavior to win a benchmark. +- **Efficient.** Throughput-per-core, memory-at-fixed-hit-ratio, and p99/p999 + tail latency. Never fat-box aggregate QPS: the marketing "25x" framing pits 64 + threads against 2 [dragonfly-25x-thread-asymmetry], and at one core the + shared-nothing leader is only at parity with Redis [dragonfly-single-core-parity]. + Per-core is the bar we hold ourselves to. +- **Simple.** Zero-config single-binary operation: one binary, safe cache + defaults, install to first GET in under a minute, kernel-only at runtime. No + JVM, no .NET, no sidecar. +- **Scalable.** Single-node-first: one process uses every core. Multi-node is a + real distributed design derived from the architecture spec, never an emulated + single-process stopgap. +- **AI-Driven.** Off-path advisor only. AI mines prior art and verifies claims + (this repository is the proof); in the engine, learned policies are allowed + only off the hot path and only when they never compromise the contract, + determinism, or tail latency. + +## Conflict resolution, worked + +When two designs conflict, the higher tenet wins: + +- **Compatible vs Efficient:** a faster encoding that changes an observable + reply loses. Keep the reply. +- **Efficient vs Simple:** a per-core win that needs a tuning knob ships, with a + safe default, over a slower zero-knob path. +- **Simple vs Scalable:** no second binary or external coordinator to get + multi-node; the single binary carries it. +- **Scalable vs AI-Driven:** a deterministic resharding primitive beats a + learned placement heuristic. + +## Governing documents + +Read these before opening a design issue: + +- The vision EPIC (#1): the canonical thesis and tenets. +- The committed non-goals (issue #10; the ratified register lands at + `docs/NON_GOALS.md`): what IronCache will not do, each traced to a tenet or a + deferred milestone. +- [Prior-art survey](PRIOR_ART.md) and [`prior-art/claims.yaml`](prior-art/claims.yaml): + the version-pinned single source of truth for every numeric claim (#6). +- [ADR index](adr/INDEX.md) and issue #4: where load-bearing decisions are + ratified and superseded. +- [Glossary](GLOSSARY.md) and [Invariants](INVARIANTS.md) (#3): the shared + vocabulary and the properties we refuse to violate. diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md new file mode 100644 index 0000000..4ac0828 --- /dev/null +++ b/docs/GLOSSARY.md @@ -0,0 +1,29 @@ +# Glossary + +The canonical meaning of project-wide terms. When a design issue and a review +disagree about what a term means, this file wins. Each term names the issue that +owns its design. Companion to [INVARIANTS.md](INVARIANTS.md); both roll up to +issue #3. + +- **shard**: the unit of keyspace partition inside one process. A key maps to a + shard by `k = HASH(KEY) % N`, where N is at most the core count + [dragonfly-shard-formula]. Distinct from a Redis-wire hash slot. Owner: #24. +- **slot**: a Redis Cluster hash slot (0 to 16383), a protocol-level concept for + client routing and resharding. Many slots map onto one IronCache shard; the + two are not interchangeable. Owner: #70. +- **hot path**: the per-request GET/SET execution path inside a shard's owning + core. State touched here must be shard-local: no locks, no cross-core atomics + [glommio-locks-never-necessary]. Owner: #24. +- **advisor**: an out-of-band component that observes traffic and proposes + tuning (encoding thresholds, admission, knobs). Advisors never sit on the hot + path and never make a decision the hot path must block on. Owner: #88. +- **codec**: the wire-level encoder/decoder for RESP2/RESP3 frames. Strictly + protocol; carries no storage semantics. Owner: #15. +- **encoding**: the in-memory representation of a value (inline string, + listpack-equivalent, hashtable, and so on). An implementation detail surfaced + only through `OBJECT ENCODING` compatibility. Owner: #40. +- **tier**: a storage level distinguished by latency and cost (in-memory versus + cold or compressed). Orthogonal to shard and encoding. Owner: #66. + +This file is updated whenever a term's canonical meaning changes; it does not +re-litigate the owning issue's design. diff --git a/docs/INVARIANTS.md b/docs/INVARIANTS.md new file mode 100644 index 0000000..383cdaa --- /dev/null +++ b/docs/INVARIANTS.md @@ -0,0 +1,58 @@ +# Load-bearing invariants + +The properties every design, ADR, and review must respect. They are stated once +here, ranked-tenet aware, and made CI-checkable where the language allows. +Companion to [GLOSSARY.md](GLOSSARY.md); both roll up to issue #3. Conflicts +resolve in tenet order: Compatible > Efficient > Simple > Scalable > AI-Driven. + +1. **Shared-nothing.** A key's shard is owned by exactly one core; there is no + hot-path shared mutable state, and cross-shard work is explicit message + passing [seastar-shared-nothing]. This replaces Redis's single command thread + [redis-command-execution-single-threaded] with per-core ownership. Owner: #24. +2. **Determinism.** Time, network, disk, and RNG are reached only through + abstractions, so a seeded replay of the same input yields byte-identical + eviction and expiry decisions. No direct clock or rand calls on decision + paths. Owner: #31. +3. **Memory honesty.** `maxmemory` is accounted against allocator-attributed + bytes, not naive object sizes [redis-maxmemory-accounting], so the limit is an + honest bound rather than a fragmentation-blind estimate. Owner: #41. +4. **No fork.** No persistence, snapshot, or maintenance path may call `fork()`. + Copy-on-write RSS doubling is avoided structurally; reclamation uses + background threads, not process forks. Owner: #59. +5. **Behavioral-equivalence contract.** The compatibility bar is behavioral + equivalence with the Valkey/Redis oracle, not bit-identical replies. Reply + structure, types, and observable side effects must match; the precise + error-text bar is set by the conformance issue. Behavioral equivalence is + chosen over bit-identity because it preserves client compatibility (the top + tenet) without freezing the internal representations the Efficient tenet + needs to evolve. Owner: #95. + +## CI-checkability + +Where a property is mechanical, CI will enforce it once the engine code lands: + +- Invariant 4: a lint forbids `fork()` and any libc fork binding anywhere in the + tree. +- Invariant 2: a lint forbids direct `std::time`, `Instant::now`, and `rand` + calls outside the approved abstraction modules. +- Invariant 1: the hot-path crate denies `std::sync` lock and shared-atomic + imports. + +Invariants 3 and 5 are enforced by test suites owned by their issues (#41, #95), +not by lints. These lints are specified here and become active with the first +Rust crate; until then this file is the contract reviewers cite. + +## Ownership map + +| Concern | Issue | +| --- | --- | +| Shared-nothing / shard ownership | #24 | +| Cross-shard coordinator and message passing | #29 | +| Determinism and seeded-replay testing | #31 | +| Memory-honesty accounting and allocator | #41 | +| No-fork reclamation and background drop | #59, #51 | +| Behavioral-equivalence and error-text bar | #95 | +| Encoding terminology and `OBJECT ENCODING` | #40 | +| Tenets and conflict ordering | #2 | +| ADR cross-reference | #4 | +| Coherence enforcement across issues | #5 | diff --git a/docs/README.md b/docs/README.md index e1276df..420c88f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -13,6 +13,16 @@ issues are the specification. in CI by [`../scripts/ci/check-prior-art-claims.sh`](../scripts/ci/check-prior-art-claims.sh). - [research/](research/): the per-dimension research corpus (one document per caching dimension) and the machine-readable [`corpus.json`](research/corpus.json). +- [CHARTER.md](CHARTER.md): the thesis, the five ranked tenets and their + conflict order, and the governing-document index. +- [GLOSSARY.md](GLOSSARY.md) and [INVARIANTS.md](INVARIANTS.md): the canonical + vocabulary and the load-bearing invariants every design must respect. +- [adr/](adr/): the Architecture Decision Records, their registers + ([INDEX](adr/INDEX.md), [OPEN](adr/OPEN.md), [QUESTIONS](adr/QUESTIONS.md)), + and the ADR format. +- [ROADMAP.md](ROADMAP.md): the implementation-readiness sequencing (thin + vertical slice, waves, gate set). [AUDIT.md](AUDIT.md): the pre-implementation + audit record. The authoritative, evolving design lives in the [GitHub issues](https://github.com/ELares/IronCache/issues), indexed from the diff --git a/scripts/ci/check-prior-art-claims.sh b/scripts/ci/check-prior-art-claims.sh index e3764fb..c1cb00a 100755 --- a/scripts/ci/check-prior-art-claims.sh +++ b/scripts/ci/check-prior-art-claims.sh @@ -15,6 +15,9 @@ CLAIMS="$ROOT/docs/prior-art/claims.yaml" # Prose files that cite claim ids. Add to this list as the docs grow. PROSE_FILES=( "$ROOT/docs/PRIOR_ART.md" + "$ROOT/docs/CHARTER.md" + "$ROOT/docs/GLOSSARY.md" + "$ROOT/docs/INVARIANTS.md" ) fail=0