(Rhymes with "steely." An upright stone the ancients carved laws into and set in public, so no one could pretend the rules said otherwise.)
AI can generate a thousand catalog entries, lore pages, or quiz questions with ease. Most tools that check that output only check whether it looks valid. Does it parse, match a schema, satisfy a judge model? A lazy shortcut passes all of that. stele checks something different: whether the work matches the material it was supposed to come from: the derivations, the approvals, the things that must never ship.
It catches what schema checks miss: fields that should have been derived from source data but were hand-typed to match; the same template recycled with the nouns swapped; facts nobody approved presented as settled; sealed content that leaked into output that should never have contained it. Individual items can all look fine on their own; the batch and the process still go red.
The demo is two commands. No setup, no account, plain Python:
make demo # run the checks on a clean batch: everything passes
make catch # run them on a cheating batch: it gets caught, and the report names why
(Run from the repository root. make catch exits nonzero on purpose: a caught batch is the tool's failure signal, so CI can act on it.)
Or skip the demo and point it at your own files right now. docs/QUICKSTART.md has three 15-minute tracks: stop secrets from shipping (python3 -m kit seals init), check a generated batch (python3 -m kit fingerprint --batch yours.jsonl), and catch a derived file that was edited by hand. First RED on your own data, one sitting.
Limits, stated plainly. stele catches lazy generators taking shortcuts. It does not catch a determined adversary deliberately tuning output to beat the checks. Seals are disclosure checks, not a confidentiality proof: GREEN means the enabled gates passed, not that prose is good or that a clever leak is impossible.
Three demo packs ship under demo/, each self-contained:
- catalog: a fictional product catalog with derived pricing history and embargoed products
- lore: a small fictional world with timeline, sealed spoilers, and facts that start unapproved
- quiz: an employee-training question bank where the answer key itself is the sealed content (the smallest pack, and the template for writing your own)
Where this came from. A year of production work on a large fictional-world project (four constructed languages, hundreds of canon entries, dozens of characters, sealed spoilers that had to stay sealed) where AI could produce material faster than one person could read it. The real limit was never generation; it was how much a single reviewer could actually verify. The checks were built there incident by incident as a generator found new shortcuts, then hardened over months of daily use. The founding incident, with the real numbers: a generated batch of 500 words passed every validity check while quietly dodging the derivation rules, with near-0% historical change against a measured 96.5% natural rate. It was caught, reverted, gated at an 85% floor, and re-authored to 100%; that gate, generalized, is kit/fingerprint.py. Before release the mechanisms were red-teamed across three AI model families, each trying to break them.
And it was field-tested on real model output. A real model, briefed with the catalog's canon and rules and no knowledge of the gates, generated 30 products, which passed all the gates and were still wrong (30 malformed derived names, 3 drifted prices, found by hand-verification). That unplanted catch exposed a gap, the gap became the batch_derivation gate the same day, and the raw model output is committed at demo/catalog/batches/field-haiku.json so you can re-run the whole thing. The corrections log entry is 2026-08-04-003. That loop, catch to rule to gate, is the product working on itself.
How it fits. Validators and eval tools check outputs one at a time against a spec. stele checks the batch and the process: whether derived content was actually derived, whether the batch shows shortcut patterns, whether restricted or unapproved material moved. Use both; they catch different failures.
This is a reference implementation, not a library to depend on. The CLI is the on-ramp; for anything deeper, copy the code into your own project rather than importing it. Stdlib Python only (3.10–3.13); nothing to install, nothing to configure. The method is the deliverable. PROCESS.md is the production loop the gates live inside; docs/CALIBRATION.md is how to set floors without lying to yourself.
- "Isn't this just division dressed up?" The math is boring on purpose; boring math is auditable. The value is knowing which dimensions catch real shortcuts (learned from production failures), the calibration discipline, and the anti-bypass plumbing where naive versions die.
- "Who checks the checker?" The canaries: planted fake secrets the scanner must keep catching, or nothing it says is trusted.
- "Couldn't I build this in a weekend?" The code, yes. Knowing which checks matter cost a year of a generator finding new ways to be lazy. Also: the field-test batch in this repo passed every gate we had and was still wrong. The eighth gate exists because of it.
- "How is this different from validators and evals?" Those check outputs one at a time against a spec. Keep them. This checks the batch and the process: was the derivable derived, does the batch have the shape of real work, did sealed or unapproved material move.
- "The floors are magic numbers." They're documented defaults that calibrate from your reference corpus, and the note-only tier exists so uncalibrated checks report without veto power. A public precision/recall study is the one open item we'd most welcome as a contribution.
License. Code is MIT. Docs are CC BY 4.0; credit Shawn Terry (shawnterry.com).