A membership-detection harness. The web-canary watcher from before catches published propagation (your string on a crawlable page). This catches the other thing: your artifact pulled into a model's training set and later re-emitted from the weights. Surface after going under.
A clean surfacing is strong evidence of ingestion. Silence is not evidence of non-ingestion — frontier models are trained to suppress verbatim regurgitation, so a negative means almost nothing. A positive means something; a negative means keep your mouth shut. The whole harness is built so that a positive is hard to explain away, and so it never certifies a positive it can't defend.
- Held-out negatives. A batch of canaries you never publish, probed alongside the real ones. If a model "surfaces" a held-out one, the harness is manufacturing hits — the run is invalid. This arm is the experiment.
- Withheld continuation. The canary's second half never appears in the prompt. Grading a model on text you handed it is citogenesis in a new coat.
- Cutoff gate. A surface in a model whose cutoff predates your publish date is physically impossible — so it's a leak in the harness, not ingestion.
- Entropy sizing. 128-bit canaries → chance completion stays astronomically unlikely even across thousands of probes. That's the base-rate control.
- Recognition ≠ proof. A model "knowing" STOICHEION or naming 96/4 is confounded by sycophancy and reconstruction. It's a lead, never a hit, and never enters the hard tier. (This is the exact thing once mislabeled "substrate independence proven." It wasn't.)
Not used: loss-based membership inference (perplexity of your text vs. controls). At frontier scale it barely beats chance and is confounded by n-gram overlap and temporal shift. Canary extraction is the clean instrument.
| File | Closure-Loop layer | Role |
|---|---|---|
canary.py |
Detection | generate 128-bit canaries; anchor record shape |
registry.py |
Anchoring | ground truth of what went under and when |
harvest_github.py |
Anchoring | pull your existing davw hashes/nonces into the registry |
probe.py |
Comparison | prefix in, withheld continuation checked; model adapters |
score.py |
Witness | held-out control arm, cutoff gate, base rate, certification |
harness.py |
Lineage | orchestrate panel; temporal/structural claim, not causal |
selftest.py |
— | plant-then-withhold proof, no network |
python selftest.pyProves, with no network: real plants surface, held-out FPR is 0, a post-cutoff surface is gated out, a deliberately leaked prompt spikes the control arm to FPR=1.0 and the run is refused, and a run with no control arm is refused as performative.
GH_TOKEN=... python harvest_github.py DavidWise01— turns your davw corpus into published-canary anchors (a lower bound; see limits below).- Add held-out canaries:
canary.make_held_out()× N, saved to the registry, never published anywhere. - Implement a real model adapter in
probe.py(anAnthropicModel.completethat calls the API) and set itscutoffto the model's REAL training cutoff. harness.run_panel(registry, [model_a, model_b, ...])→harness.report(...).
Does: a 128-bit string only you published, before the model's cutoff, re-emitted verbatim, with a held-out control arm proving the harness doesn't hallucinate. That's a real membership detection.
Does not: prove theft, licensing obligation, or anything about a "mesh." Membership is temporal/structural evidence, not a causal or legal finding — and davw attribution attaches to your work as evidence; it does not reach into someone else's model.
- Harvest is default-branch-only and a lower bound; a missing hash means "not harvested," not "not published."
- Detection depends on the model actually being willing to complete a hex prefix; refusal/deflection reads as a (meaningless) negative, per the asymmetry.
- Cutoff dates are self-reported by vendors; the gate is only as honest as the cutoff you feed it.