Skip to content

feat: compile-cache executor + fastcache-cc sccache-style launcher - #33

Open
Yaraslaut wants to merge 23 commits into
masterfrom
feature/compile-cache-executor
Open

feat: compile-cache executor + fastcache-cc sccache-style launcher#33
Yaraslaut wants to merge 23 commits into
masterfrom
feature/compile-cache-executor

Conversation

@Yaraslaut

@Yaraslaut Yaraslaut commented Jul 23, 2026

Copy link
Copy Markdown
Member

Compile-cache executor + fastcache-cc launcher

Implements the server-side executor from docs/specs/executor.md §5 and a
drop-in sccache-style compiler launcher that uses it — together making a
shared compile cache correct across machines at different checkout depths,
ending the /showIncludes poisoning treadmill.

The problem

A cached compile value is not a portable blob. On a hit, the producing
machine's captured /showIncludes output is replayed verbatim; Ninja records
those header paths into .ninja_deps. When the producer's build-tree→header
depth differs from the consumer's (canonically, a CI runner nested one level
deeper), the replayed paths don't exist on the consumer, so Ninja marks
everything dirty and no-op builds recompile hundreds of objects. The existing
client-side repair is a last-writer-wins treadmill.

The approach (Option C — server canonicalizes, client localizes)

A new IProtocolHandler (CompileCacheHandler, autodetect magic 0xFC) over
the unchanged IStorage stack:

  • On STORE: canonicalizes every path in the value's captured text regions
    into machine-neutral <SRCROOT>/<BUILDTREE> tokens, using the producer's
    layout carried in the frame. The object blob is a separate field and is never
    rewritten. The stored value has no machine identity — so no producer can
    poison a shared entry.
  • On FETCH: serves the canonical value verbatim; the client localizes tokens
    to its own layout. The server stays off the per-consumer transform path.

The canonicalization rule is normative in docs/specs/compile-cache-canon-v1.md
with conformance vectors as the client↔server parity contract.

Server components

Area Component
Path rewriting CompileCache/PathCanon — absolute↔token + showIncludes/MSVC-diagnostics/depfile region grammar
Value framing CompileCache/CompileValue — object blob (untouched) + grammar-tagged text regions
Cohort tracking CompileCache/CohortManifest — cohort-id→key-set + reverse index
Storage hook IStorage::Prefetch — warm a tier with no read side effect (LayeredStorage pulls L2→L1; forwarded through all decorators)
Protocol ProtocolAutodetect 0xFC flavor + CompileCacheHandler (STORE/FETCH + leading-key cohort prefetch)

The launcher — tools/fastcache-cc

A real drop-in for sccache, wired via CMAKE_CXX_COMPILER_LAUNCHER:

  • Keys on hash(compiler-id + preprocess output + checkout-relativized args) — same content at different checkout roots ⇒ same key, so machines share entries. Isolated in CacheKey for tuning.
  • HIT: reproduces the object and replays stdout/stderr with /showIncludes localized to the local layout, so Ninja records valid deps.
  • MISS: runs the real compiler (stdout/stderr captured separately via CreateProcess), STOREs the canonicalized result, passes output through.
  • Any cache error → falls back to a plain real compile and (with FASTCACHE_VERBOSE) warns. The build never breaks because the cache is down.
  • Config via FASTCACHE_ADDR/SRCROOT/BUILDTREE/COHORT/VERBOSE env (the SCCACHE_* model).

Testing

  • Full suite green (881/881, clangcl-debug, PEDANTIC_COMPILER_WERROR=ON), including unit tests for every server component, the launcher's command parsing and cache-key parity across checkout roots, and a poisoning-regression test.
  • Real-compilation E2E (cl.exe + clang-cl):
    • compile-cache-testclient/run-crossdepth.ps1 — store from a deep path, fetch+localize from a shallow one; every include path resolves. ALL CROSS-DEPTH CHECKS PASSED.
    • fastcache-cc/run-launcher-e2e.ps1 — drives the launcher as the compiler: first compile MISSes and stores; second (object deleted) HITs and reproduces the object byte-identically; content stored from a deep checkout HITs from a shallow one with /showIncludes localized. ALL LAUNCHER E2E CHECKS PASSED.

Whole-codebase validation (real compile database)

The launcher was exercised against a large real C++ codebase by replaying its
compile_commands.json (4223 TUs) through fastcache-cc twice (populate, then
measure). Second-pass hit rate: 99.7% (393/394 cacheable) on a 400-TU
sample.
This surfaced and fixed several real launcher defects (all now
covered by the fix commit):

  • Keying preprocess must be captured from stdout only — merging 2>&1
    folded diagnostics in at a non-stable interleave point, making the key
    nondeterministic (nothing ever hit).
  • Child stdout/stderr must be drained concurrently — sequential draining
    truncated large (>64 KB) preprocessed output nondeterministically.
  • Windows arg quoting hardened (backslash runs before a closing quote) so
    MSVC/SDK include paths with spaces (C:\Program Files (x86)\…) survive.
  • TUs referencing __TIME__/__DATE__/__TIMESTAMP__ are skipped (they
    re-key every second; same as sccache).

Operational note: objects can be large (observed max ~122 MB), so run the
daemon with --storage-max-value 256M (this also raises the frame-payload cap);
the 16 MiB default would leave ~7% of this codebase's objects uncached.

Scope / follow-ups

  • The cache key is a deliberate v1 (preprocess + relativized-args hash), isolated so it can be tuned against real dev↔CI hit rates.
  • Localized path separators are normalized to / in some segments (Ninja matches separator-insensitively — cosmetic); non-showIncludes diagnostic paths aren't localized yet.
  • Phase-2 grammar should still be reconciled against the empirical /showIncludes diff in executor.md §6 before freezing.

Design record

  • docs/superpowers/specs/2026-07-23-compile-cache-executor-design.md
  • docs/superpowers/plans/2026-07-23-compile-cache-executor.md, …-fastcache-cc-launcher.md
  • docs/specs/compile-cache-canon-v1.md

🤖 Generated with Claude Code

Yaraslau Tamashevich and others added 14 commits July 23, 2026 15:04
Records the approved design consolidating the executor direction from
docs/specs/executor.md into concrete decisions: server canonicalizes on
store / client localizes on fetch (Option C), a new CompileCacheProtocol
handler over the unchanged IStorage stack, client-tagged value regions
(object blob never touched), a shared versioned canonicalization spec with
a conformance suite, and executor-owned cohort prefetch over a Phase-1
Prefetch hook + cohort manifest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…canon v1)

PathCanon maps absolute paths to machine-neutral <SRCROOT>/<BUILDTREE>
tokens and back, and rewrites path spans inside captured compiler text
regions (showIncludes / MSVC diagnostics) per a grammar tag. Governed by
docs/specs/compile-cache-canon-v1.md with conformance vectors as the
server<->client parity contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… regions)

EncodeCompileValue/DecodeCompileValue serialize a compile result as an
untouched object blob plus zero or more grammar-tagged text regions, with
bounds-checked decoding that rejects truncated, wrong-version, or
unknown-grammar frames. Big-endian lengths match the project wire helpers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ffects

IStorage gains a Prefetch(key, now) primitive: warm an entry into the
in-memory tier with no client-read side effect (no hit/miss stat change,
no LRU-as-hit promotion). Default Peeks (correct for single-tier);
LayeredStorage overrides to pull L2->L1 via the existing mirror path;
Sharded/Tracing/Notifying/WriteErrorReporting forward it. Backs the
compile-cache executor cohort prefetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rage

CohortManifest records which cache keys belong to a build cohort, stored
in-cache under a control-byte-prefixed key so no new storage interface is
needed. AddKey appends atomically via IStorage::Update (idempotent, capped
at MaxKeysPerCohort); Keys lists them in insertion order. Backs the
executor's leading-key cohort prefetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…, serve canonical on FETCH)

Adds the CompileCacheHandler IProtocolHandler behind a new 0xFC autodetect
magic byte. STORE canonicalizes every text region with the producer layout
(object blob untouched), stores the machine-neutral value, and records
cohort membership. FETCH serves the canonical value verbatim for the client
to localize. Cross-depth portability verified end-to-end: a value stored
from a deep layout localizes to a shallow consumer. CacheEngine gains
Prefetch + Storage accessors the executor drives.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
STORE now records a reverse key->cohort index alongside the forward
manifest. On a FETCH hit the handler discovers the fetched key's cohort and
warms the rest of that cohort into L1 (after the reply is sent, guarded per
connection so a cohort is primed once). Verified: fetching one member of a
cohort warms its siblings into L1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The reference-localizer client links the same PathCanon/CompileValue codec
the server uses and speaks the 0xFC protocol over TCP: `store` compiles with
/showIncludes and STOREs the framed result; `fetch` localizes the canonical
value to the consumer layout and verifies every include path resolves.
run-crossdepth.ps1 proves the value-portability guarantee with real cl.exe
and clang-cl compilations — an object produced at a deep path is usable from
a shallow checkout. Off by default (FASTCACHE_BUILD_TESTCLIENT). Prints
generic counts only; no project source or private paths are read or emitted.
Also lands the executor design spec that motivated the work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ze test path literals

Adds CompileCache/ + CompileCacheHandler to the AGENT.md architecture map and
an 'implemented (server side)' note to executor.md linking the design and
canon spec. Replaces example checkout-path literals in the unit tests with a
generic D:\project root so no specific checkout path is baked into fixtures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tocol

fastcache-cc <compiler> <args> fronts a compile: computes a cross-machine
key (preprocess output + checkout-relativized args + compiler id), FETCHes
from fastcached, and on a HIT reproduces the object + replays stdout/stderr
(showIncludes localized to this machine) so Ninja records valid deps. On a
MISS it runs the real compiler with split stdout/stderr capture, STOREs the
canonicalized result, and passes output through. Any cache error falls back
to a real compile and warns (FASTCACHE_VERBOSE). Config via FASTCACHE_* env.
Unit-tested: command parsing + key parity across checkout roots.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a HIT/MISS verbose trace to the launcher and an E2E harness that drives
fastcache-cc as the compiler with real cl and clang-cl: first compile MISSes
and stores, second (object deleted) HITs and reproduces the object
byte-identically; content stored from a deep checkout HITs from a shallow one
with showIncludes localized. All checks pass for both compilers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Yaraslaut Yaraslaut changed the title feat: compile-cache executor (custom protocol, path canonicalization, cohort prefetch) feat: compile-cache executor + fastcache-cc sccache-style launcher Jul 23, 2026
Yaraslau Tamashevich and others added 9 commits July 23, 2026 17:27
…real builds

Validated against the real LASTRADA compile database (compile_commands.json):

- Capture the preprocess for keying from stdout only (not merged 2>&1); the
  merged interleave of two buffered streams is unstable and was making the
  cache key nondeterministic.
- Drain child stdout/stderr concurrently (a second thread) so large
  preprocessed output (>64KB pipe buffer) is not truncated/deadlocked.
- Harden Windows argument quoting (double backslash runs before a closing
  quote) so MSVC/SDK include paths with spaces survive to the child compiler.
- Skip translation units that reference __TIME__/__DATE__/__TIMESTAMP__ (they
  re-key every second and can never hit), matching sccache. Falls back to a
  plain compile.
- Surface STORE outcome under FASTCACHE_VERBOSE for diagnosability.

Sample of 150 real TUs: 99.3% hit rate on the second pass (147/148 cacheable;
2 .rc resource files fall back correctly, 1 residual header-introduced
__TIME__ TU).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lue-cap guidance

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s-folder hits)

RelativizeArgs only tokenized the source root, leaving include args that point
into the in-checkout build tree (…\out\build\vcpkg_installed, generated
headers) as machine-absolute paths. Those paths differ across checkouts, so the
key differed and cross-machine/cross-folder hits collapsed to near zero on any
real project. Now relativizes against both roots (PathCanon emits <BUILDTREE>
for the longer-matching build-tree paths, <SRCROOT> otherwise).

Caught by replaying a real 4223-TU compile database from a mirrored checkout at
a different absolute path: cross-folder hit rate went 7.5% -> 100% on the
verified sample after this fix. Unit test added for the build-tree case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The disk tier was hardcoded unbounded (opts.maxBytes = 0), so a build cache
grew without limit. Add --storage-max-disk: the total budget is split evenly
across physical shards and each CoW tree evicts its LRU tail to fit (the
eviction path already existed and is tested). Default 0 keeps the previous
unbounded behaviour. Lets an operator hold a cache within a fixed disk
allotment (e.g. --max-memory 5G --storage-max-disk 10G).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Compile the launcher from just its own sources plus the dependency-free
PathCanon/CompileValue/Crc32c cores instead of linking the whole FastCache
library, so it pulls in no vcpkg deps (zstd/lz4/yaml-cpp). Add
FASTCACHE_STATIC_LAUNCHER (on by default) to statically link the MSVC CRT
(/MT). A Release build then depends only on always-present Windows system
DLLs (KERNEL32, WS2_32) — a 0.6 MB copy-anywhere executable needing no VC++
redistributable on build agents.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
storage_max_value was already a YAML key; storage_max_disk (the on-disk L2
cap) was CLI-only. Wire it into YamlReader with the same byte-size grammar so
the whole cache config can live in one file. Tests: both keys parse suffixes,
disk defaults to 0 (unbounded), malformed value rejected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…yout

LocalizeOne joined the root and tail with a hardcoded backslash while
ToNative picked its separator behind #if defined(_WIN32). On POSIX the two
disagreed and produced a mixed path (D:\project\include/foo.h), failing the
Localize round-trip and cross-depth tests on every non-Windows runner.

The separator is a property of the layout being localized to, not of the
host: the cache is shared across machines, so a Windows consumer layout must
localize to backslashes even when the daemon runs on Linux. Derive it from
the target root (SeparatorOf) and use that one value for both the join and
the tail (JoinLocalized); the #ifdef is gone, so output no longer depends on
where the code was compiled.

Also collapse LocalizeOne's two copy-pasted sentinel branches into a
sentinel -> root descriptor table, and take RewriteRegion's callable by
const& — it is invoked once per matched span, never forwarded, which
clang-tidy flagged as cppcoreguidelines-missing-std-forward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ne refs

clang-tidy flagged two defects in CompileCacheHandler.cpp that the previous
CI run never reached (it aborted at the first error in PathCanon.cpp):

- cppcoreguidelines-avoid-reference-coroutine-parameters: ReadLengthPrefixed
  took ByteReader& . A reference parameter is not stored in the coroutine
  frame, so it dangles if the caller's object dies across a suspend point.
  Every other protocol coroutine in the tree already takes ByteReader*; this
  was the lone deviation, so follow the established convention.
- readability-function-cognitive-complexity: Run scored 74 against the 60
  threshold. Extract the STORE and FETCH arms into HandleStore/HandleFetch
  returning a Next{Continue,Abort} outcome, pull the region loop into
  CanonicalizeRegions and the warm-up into PrefetchCohort. Run is now a thin
  dispatch loop over the opcode.

Behaviour is unchanged: same frames read, same replies written, same
best-effort cohort bookkeeping, same abort-on-write-failure semantics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI's tidy job walks the translation units in order and stops at the first
failure, so each fix uncovered the next layer. Sweeping every file this PR
touches locally found 16 findings; this clears all of them in the PR's own
files:

- bugprone-unchecked-optional-access (6): the check cannot see a has_value()
  guard through Catch2's REQUIRE macro. Compare via value_or so the access
  needs no guard, keeping the assertion just as strong (an absent value fails
  the comparison instead of the REQUIRE).
- readability-container-contains (4): find(...) != npos -> contains(...).
- modernize-use-designated-initializers (3): brace-init of TextRegion.
- cppcoreguidelines-avoid-reference-coroutine-parameters (1): WriteBytes took
  its payload by const&; a reference parameter is not stored in the coroutine
  frame, so take it by value.
- bugprone-easily-swappable-parameters (1): StoreFrame took four adjacent
  string_views. Group them into a named StoreFields struct so the fields
  cannot be transposed at a call site.

Left alone: a clang-analyzer-cplusplus.NewDelete report in SharedValue.hpp,
which this PR does not touch and which the Linux/libc++ CI build does not
flag (analyzer path-sensitivity differs by standard library).

Behaviour is unchanged; 824 test cases / 11964 assertions pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant