docs(operations): record CR-DD-016 Track B classifier-fallback investigation - #181
Merged
coreytshaffer merged 3 commits intoAug 29, 2026
Merged
Conversation
…igation Read-only investigation record for Track B of CR-DD-016's Sequencing section, which required that track stay read-only until its findings warrant a separately scoped CR. This is that read-only output. No CR identifier is minted, no implementation authority is requested or granted, and no triage_core/ source, schema, test, or fixture changed. Settles the three questions Track B was scoped to answer: 1. "refactor" IS the universal terminal fallback of both classify and classify_deterministic (classifier.py:58, :78). 2. It resolves PERMISSIVELY: sensitivity=low -> local_heavy with human_review_required=False. An unclassifiable prompt fails toward autonomous execution rather than toward review. This sharpens CR-DD-016's "semantically wrong" characterization -- the direction of the wrongness is the finding. 3. The Aug-8 trial motivating CR-DD-016 provably did NOT traverse the fallback, confirming that CR's Corrective Amendment by probe rather than by inference. The general question is unanswerable from recorded evidence by construction: bugfix and refactor both collapse onto task_class=code_repair (client.py:466, :468) before anything is written, so the two code_repair trials in 2026-08-01-corrected are permanently ambiguous. Also records, as findings rather than conclusions: the model system prompt and the regex cascade disagree on "format" (classifier.py:33 vs :64), giving the same prompt opposite governance outcomes decided by a race; the model path lost that race 5/5 on the measured host; and the stated 1.5s guard costs ~3015ms wall clock. Method and its limits stated in the document: static inspection at main fa66c64, probes that call the real _build_resilience_route_input and choose_resilience_route rather than mirroring them, read-only ledger census, and five live local Ollama calls that wrote nothing. A refuted hypothesis (that resolve_model burns a second timeout budget) is kept in the record; it measures 0ms because it short-circuits on an explicit model. The cause of the 2x timeout inside requests/urllib3 was NOT isolated and is not asserted. Recommends two separate future CRs -- fallback semantics/evidence, and classifier provenance/divergence -- deliberately not combined, per the sequencing rule CR-DD-017 restates. Track C remains untouched. Documentation only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PR #179 (CR-DD-012B) merged while this record was being drafted, landing work that overlaps Finding 3. Reconcile rather than let the record read as current when part of it no longer is. - New "Reconciliation against main at c798e0e" section: what #179 added (CLASSIFICATION_REASON_CODES with deterministic_classifier_match / _default at governed_decision.py:65-72, set via _classification_reason_code at run_plan.py:221-235/:468), and its effect on each finding. - Finding 3 recorded as PARTIALLY SUPERSEDED for the governed-decision path only. The ordinary tc run path still classifies without provenance (client.py:215) and still collapses bugfix/refactor onto code_repair at sensitivity=low before route evidence is written (client.py:774-821). Whether the governed reason reaches persisted route evidence is left explicitly untraced, not asserted. - Findings 1, 2 and 4-7 recorded as standing: classifier.py, resilience_router.py and backends.py are byte-identical between fa66c64 and c798e0e, and the client.py maps moved without changing content. Finding 7's --plan/execution seam arguably widens, since the plan path gained provenance execution still lacks. - Citation drift stated plainly: references into client.py, run_plan.py and route_events.py were verified at fa66c64 and do NOT resolve at c798e0e; the other three files resolve at both. - Deferred follow-up candidate 1 narrowed so a future CR does not re-scope work #179 already merged. All nine new citations verified against origin/main. Documentation only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
main advanced twice more after the reconciliation was written against c798e0e. Re-pin it to the current tip and record what each merge did. - Reconciliation retitled to 96eb3d3, with a table of the three merges that landed while this record was drafted: #179 (CR-DD-012B, the only one to touch triage_core/), #180 (CR-DD-016 closeout, docs only), and #166 (CR-132, docs only -- one new CR document). - Verified directly that `git diff c798e0e 96eb3d3 -- triage_core/ tests/ schemas/` is empty, so no source fact changed across those two merges and every finding that held at c798e0e holds at 96eb3d3. - All reconciliation citations re-verified against 96eb3d3. - Citation drift re-checked using the two stale anchors as negative controls: run_plan.py:59 and route_events.py:217 still point at the wrong content at 96eb3d3, confirming the drift is a real property of the record rather than an artifact of how it was measured. No finding changed. Documentation only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
✅ Deploy Preview for poetic-quokka-0fd859 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Read-only investigation record for Track B of CR-DD-016's Sequencing section, which required that track stay read-only "until its own findings warrant a separately scoped CR." This is that read-only output. One new file, 301 lines, documentation only.
No CR identifier is minted, no implementation authority is requested or granted, and no
triage_core/source, schema, test, or fixture is changed by this PR.Track B was scoped to settle three questions. All three are now answered:
Is
"refactor"genuinely the universal terminal fallback? Yes.classify_deterministic's terminal no-match return (classifier.py:78) is reached by every path throughclassify(classifier.py:58) — no backend, construction failure, exception, timeout, empty response, or non-member response. Confirmed empirically: the empty string classifies asrefactor.What downstream routing depends on it? This sharpens CR-DD-016's characterization. That CR called the fallback "semantically wrong"; the record establishes it is wrong in the permissive direction —
refactormaps tosensitivity="low", routes tolocal_heavy, and yieldshuman_review_required=False. An unclassifiable prompt fails toward autonomous execution rather than toward review. This is a property of the composition ofclassifier.pyandclient.py's maps; neither file states it.Did any trial traverse it? The Aug-8 trial motivating CR-DD-016 provably did not — confirming that CR's Corrective Amendment by probe rather than by inference. But the general question is unanswerable from recorded evidence by construction:
bugfixandrefactorboth collapse ontotask_class=code_repairbefore anything is written, so the twocode_repairtrials in2026-08-01-correctedare permanently ambiguous.Method, and its limits
Static inspection at
fa66c64; probes that call the real_build_resilience_route_inputandchoose_resilience_routerather than re-implementing their logic; a read-only census of everyroute_audit/route_decisionpair in the repository's ledgers. Notc runinvoked, no ledger written, no synthetic execution created.Stated plainly in the document rather than buried: measuring the timeout required five live local Ollama calls. They wrote nothing and touched no ledger, route, capability, privacy, or authority surface, but two findings depend on them.
Two deliberate non-assertions:
resolve_modelburns a second budget — it measures 0 ms, short-circuiting on an explicit model) is kept in the record, and the cause insiderequests/urllib3is left unisolated rather than guessed at.Reconciliation against current
mainmainadvanced three times while this was drafted. The record carries a reconciliation section rather than reading as current when part of it no longer is:triage_core/?c798e0eda96b6696eb3d3PR #179 landed
CLASSIFICATION_REASON_CODES(governed_decision.py:65-72) distinguishingdeterministic_classifier_defaultfromdeterministic_classifier_match— exactly the provenance Finding 3 reports as absent, so Finding 3 is recorded as partially superseded for the governed-decision path. It is not fully resolved: the ordinarytc runpath still classifies without provenance (client.py:215) and still collapsesbugfix/refactor(client.py:774-821). Whether the governed reason reaches persisted route evidence is left explicitly untraced, not asserted.Findings 1, 2 and 4–7 stand:
git diff c798e0e 96eb3d3 -- triage_core/ tests/ schemas/is empty, andclassifier.py,resilience_router.py,backends.pyare byte-identical tofa66c64.Test plan
docs/operations/. No non-docs/path touched;git statusclean throughout.file:linecitations in the findings verified to resolve to the text attributed to them atfa66c64.96eb3d3.run_plan.py:59androute_events.py:217still resolve to the wrong content at96eb3d3, confirming the drift is a real property of the record rather than an artifact of the checking method.triage_core.__file__pinned to this worktree) so results are not silently taken from the main checkout.main— clean add, no conflict.No test-suite run: this PR adds a documentation record and changes no code.
What this does not grant
Merge authority for this PR. Any implementation change to the classifier or its maps. Any reopening of CR-DD-016, its Corrective Amendment, or its closeout. Any resolution of Track C, which remains untouched and separately scoped.
The record recommends two future CRs — fallback semantics/execution-path provenance, and classifier provenance/divergence — deliberately not combined, per the sequencing rule CR-DD-017 restates. Candidate 1 is explicitly narrowed so it does not re-scope what #179 already merged. Neither is proposed here.
🤖 Generated with Claude Code