Shift doable-trd-context toward reusable knowledge gathering, and add the connected loop

[yuefengz]

Codebase context: gathering reusable product knowledge

Status: two pieces exist, built independently. This is the plan to join them, and a proposed shift in what the plugin's output is for.

Mirrors doable-platform: docs/design/coding-agent-context.md (PR getdoable/doable-platform#2959). Paths are repo-qualified.

The headline

The plugin gathers product knowledge. It does not author TRD sections.

Most of what a codebase can tell us is not feature-specific. The role model, entity lifecycles and their constraints, how test data gets created, how auth works, what the preview environment looks like — none of that changes when you move from testing checkout to testing invitations. Producing it per-feature and throwing it away means paying for the same discovery on every TRD, and paying the client's engineer for it too.

Knowledge that outlives one TRD belongs in the knowledge base, which already exists, is already queried during TRD creation, and is already how a TRD resolves unknowns and checks assumptions.

  client's codebase
        │  (their coding agent reads it; source never leaves)
        ▼
  product knowledge ──────► knowledge base ──────► any TRD
                                    │           create_from_knowledge()
                                    │           enrich_with_knowledge()
                                    ▼
                            reused across features,
                            refreshed when code moves

The TRD side already works: TRDAssistant.create_from_knowledge(retriever) and enrich_with_knowledge(retriever) take any implementation of the Retriever protocol (doable-platform: src/foundation/knowledge/retrieval.py — two methods). Satisfying that protocol means nothing downstream changes: test generation, execution, critic and reporting are untouched.

Two tiers of knowledge

Tier	Examples	Scope	Refresh
Durable	product/repo map; roles and what each can do; entity lifecycles and one-way transitions; how each entity gets created; auth mechanism; preview-environment shape	Per workspace, not per feature	When the code moves
Per-question	answers to specific open questions in one TRD round	One round	Not reused verbatim; may promote to durable

The durable tier is the one currently being left on the floor, and it is most of the value. It maps cleanly onto what the knowledge base already models — InlineKnowledge with type foundational or domain_concept is exactly "a fact about this product that many TRDs will need."

The output does not have to look like a TRD

An earlier revision of the design doc proposed parsing plugin output into TRD sections — flows[] into §3, testData[] into §5. That was the wrong direction. It forces feature-shaped, single-use output and discards the reuse. The plugin's job is to produce good knowledge; deciding which parts a given TRD needs is retrieval's job, and retrieval already exists.

What to keep from v0.1.5

These are good and should survive the shift. Several matter more under a reuse model, not less.

• Five truth planes. desired / implemented / deployed / reference / inference on every piece of evidence, with a conflict blocking only when no authority establishes which oracle to use. This stops implementation detail silently becoming a test expectation, and handles code being newer than the deployed build. Do not introduce a second confidence scale anywhere in this system. Carry the plane onto every knowledge item.
• Local canonical file vs shareable artifact. The right shape; transfers unchanged.
• Evidence with observable anchors. Claims tied to routes, labels, messages rather than file paths.
• Refresh with content fingerprints and contextRevision. Under a per-feature model this was bookkeeping. Under a reuse model it is load-bearing: durable knowledge that silently goes stale is worse than no knowledge, and fingerprints are how staleness gets detected rather than assumed.
• Batched clarification. Ask the human once, in one round, only when it changes the answer.
• Deterministic validation and rendering rather than trusting the model to self-police.

What shifts

• Scope unit. Today everything hangs off a featureId. Durable knowledge is workspace-scoped — the role model is not a property of the checkout feature. Expect a workspace-scoped record alongside the per-feature ones.
• Destination. Today the artifact is a file a human uploads to seed one TRD. It should also become a knowledge contribution any later TRD can retrieve.
• Reuse on the next feature. Today a second feature starts over. It should start from what is already known and ask only about the delta.

MVP scope

Three parts. The endpoints exist; the two skills do not.

Naming. Do not call the setup step "intake" — that word is taken by doable-trd-intake / doable-intake.json, and a second meaning would collide. connect describes what it does.

Part 1 — doable-connect (one-time per workspace, refreshed when code moves)

Where durable knowledge gets established. Run once; everything it learns is reused by every later round and every later TRD.

1. Check credentials. DOABLE_API_URL and DOABLE_API_KEY; if absent, stop and point at the README.
2. Verify the round trip. GET /code-context/repo-map. A 401 means a bad or rotated key — say so plainly.
3. Build the workspace map, reusing what exists. If .doable/features/*/doable-intake.json is present, read repositories[] from it rather than rediscovering — v0.1.5 already did that work. Fill gaps by looking around, then ask the human one batched round about what is unresolved: repos not found locally, where they are cloned, which are user-facing, which branch reflects production. Offer to clone anything missing.
4. Save it. PUT /code-context/repo-map.
5. Contribute durable knowledge. Beyond the repo map, capture the cross-cutting facts every TRD needs — the role model and what each role can do, entity lifecycles and one-way transitions, how each entity gets created, the auth mechanism, the preview-environment shape. Each item carries its truth plane, observable anchors, and the commit it was read at. These go to the knowledge base, not to a TRD.
6. Confirm what was stored, and that these questions will not be asked again unless the code moves.

A refresh replaces the map wholesale rather than merging — a stale entry produces confidently wrong answers. Fingerprints decide what actually needs re-reading.

Part 2 — doable-answer-questions (per round)

1. Read the map. GET /code-context/repo-map. If null or empty, stop and route the user to doable-connect — do not silently improvise a map.
2. Pull. GET /code-context/questions?status=open. Tell the user how many are open and roughly what they cover before starting.
3. Answer. Use scope_hint plus the map to decide where to look. Dispatch a subagent per repository rather than reading several repos in one context — mixing repos is the main way these answers go wrong.
4. Push. POST …/answer, or POST …/skip with a reason when the code genuinely cannot settle it. Skipping is a real outcome; a question left open comes back every round.
5. Promote what is durable. An answer about the role model or an entity lifecycle is not a per-round fact — flag it so it lands as durable knowledge rather than being spent on one TRD.
6. Summarize: answered, skipped, which answers contradicted Doable's current understanding, and anything flagged as UI-hidden-but-backend-allowed.

Part 3 — the connection

Built in doable-platform PR #2959, pending two qa-supabase tables.

Endpoint	Caller	Purpose
GET /code-context/repo-map	client	read the map at session start
PUT /code-context/repo-map	client	write it during connect
GET /code-context/questions	client	pull what's open (capped at 25)
POST /code-context/questions/{id}/answer	client	submit, close
POST /code-context/questions/{id}/skip	client	close with a reason
POST /code-context/questions	us	publish
GET /code-context/answers/document	us	render answers for review

Auth is one token. Authorization: Bearer <org API key>. The existing organization API key path lets a caller omit the org slug entirely, so there is nothing else to configure.

// PUT /code-context/repo-map
{ "repos": [ { "name": "web-frontend",
               "role": "customer-facing web app",
               "surfaces": ["checkout", "account settings"],
               "user_facing": true,
               "local_path": "/path/to/clone",     // optional
               "default_branch": "main" } ],       // optional
  "notes": "anything that applies across repos" }  // optional

// GET /code-context/questions?status=open  →
{ "questions": [ { "id": "…", "mode": "corroborate|gap_fill|discover",
                   "question": "…",
                   "why": "what it unblocks",
                   "claim_under_test": "…",        // corroborate only
                   "scope_hint": ["web-frontend"],
                   "answer_shape": "enumeration|contract|yes_no|prose",
                   "priority": 100 } ],
  "open_count": 7,
  "repo_map_configured": true }

// POST /code-context/questions/{id}/answer
{ "answer": {
    "verdict": "confirmed|refuted|partial|unknown|not_applicable",
    "body": "plain-language finding, observable terms only",
    "anchors": ["DELETE /projects/{id} -> 204", "Toast: 'Project deleted'"],
    "confidence": "CONFIRMED|INFERRED|UNKNOWN",
    "evidence_checked": ["route_registration","repo_wide_search","git_history"],
    "repos_consulted": ["web-frontend","api"],
    "commit_shas": { "web-frontend": "a1b2c3d4e5f6" },
    "proposed_questions": ["Is there an undo window?"] } }

// POST /code-context/questions/{id}/skip
{ "reason": "Needs a product decision — the code supports both behaviors." }

Two server-side rules, both returning 422 otherwise:

• verdict: "refuted" requires all three values in evidence_checked. An agent that has not done all three answers unknown. This is gap 1 enforced at the boundary.
• commit_shas should always be sent. Without it an item cannot be cited or re-checked later — and for durable knowledge that is what staleness detection hangs off.

repo_map_configured comes back on the questions response, so Part 2's first step costs no extra call.

Have each skill report its version on start. Skill and API are coupled because the skill encodes endpoint shapes; when a client reports something odd you want to know what they are running.

Not in the MVP

• Durable-knowledge ingest endpoint. Part 1 step 5 describes what to capture; the endpoint that accepts it is gap 2 below. Until then durable items ride in as answers to standing questions.
• Notification when questions are waiting — we ping the pilot client directly.
• Auto-publishing questions from TRD unknowns. Publish by hand first, to learn which are worth asking.
• MCP. The REST endpoints are needed either way; a façade can wrap them later.
• Any scheduling, polling, or background work.

Gaps to bridge, in priority order

1. No guard against silent omission

This is the failure that motivated the whole kit — an earlier manual survey of a real client codebase reported three shipped features as "not built" after looking in a single module.

Nothing requires proving a capability's absence before leaving it out. The failure mode is subtler than in a survey: the skill builds context from what it finds, so a missed capability produces no claim to challenge. It is simply not in the artifact, and nothing surfaces for review. Under a reuse model this compounds — a gap in durable knowledge is inherited by every later TRD.

Suggested shape. Before a capability the feature's vocabulary implies is left out, require three checks — route/handler registration, a repo-wide search on several plausible names, and git history for added-then-reverted work — and record which were performed. If not all three, the outcome is a bounded entry in unknowns[], not silence. Enforce in the schema rather than the prompt: prompt instructions are suggestions, and this one has already been ignored once in the field.

2. Knowledge has nowhere to land except one TRD

Plugin output goes in through TRDAssistant.create(input_files=...) as a document attached to one TRD. Nothing puts it in the knowledge base, so the second feature starts from zero.

Suggested shape. A CodebaseRetriever implementing the existing Retriever protocol over stored codebase knowledge, plus an ingest path turning plugin output into knowledge items. The mapping is mostly natural: durable facts become InlineKnowledge (foundational for cross-cutting things like the role model, domain_concept for entity semantics, subtle_rule for non-obvious constraints), carrying truth plane, observable anchors, repos consulted and commit SHAs in metadata.

One schema tension to resolve. KnowledgePointer is built to point into a document with a line range, and codebase knowledge deliberately has no shareable locator — that is the privacy model working. So codebase knowledge fits InlineKnowledge (whose source_refs are optional) far better than KnowledgePointer. Decide this explicitly rather than forcing an empty source_locations.

3. Exclusivity constraints have nowhere to go

Some state cannot be shared: one active subscription per account, unique emails or slugs, seat and quota caps, one-way transitions (a paid invoice cannot be unpaid).

The field guide gets close — "include cleanup steps when unique data, destructive changes, or shared environments require isolation" — but cleanup is after the fact. Account matching needs a before-assignment fact, and there is no field for it, so it never reaches the scheduler.

Why it matters more than it sounds. The failure looks like a product bug: the test fails, the critic writes it up, someone investigates something that was never broken. And it is often intermittent — fine in one running order, broken in another. That is the most expensive kind of noise, because it makes people distrust every other result.

This is durable knowledge, not per-feature: "emails must be unique" is true for every TRD.

Suggested shape. An optional sharingConstraint on testDataRequirement, in observable terms, carried through to account_requirements, which match_accounts() already reads.

4. No path to check an existing draft

The skill builds context from scratch. conflicts[] with claims[] across truth planes captures spec-versus-code disagreement well — but only for disagreements it stumbles into. Nothing takes a TRD we already have and walks it claim by claim.

That is the round-2 workflow, and most of the value. A disagreement is not a correction to apply to the spec — it marks a place where one of the two is wrong and a test has to decide. Silently rewriting the draft to match the code destroys the value, because an expectation taken from the implementation can never fail.

Suggested shape. A mode that accepts a claim list and returns a verdict per claim (confirmed / refuted / partial / unknown) with evidence, rather than authoring an artifact from nothing. Same contract as the loop's corroborate mode, so the two can share it.

One design decision to settle

Should permissions be enumerated exhaustively — every role against every protected surface — or scoped to what a named feature needs?

The field guide takes the scoped line deliberately: "Create an actor only when role or account differences materially affect a requested flow… Prefer one executable test actor."

That was reasonable under a per-feature model. Under a reuse model it is worth revisiting: the role model is durable knowledge, enumerated once and reused by every TRD, so the cost is paid once rather than per feature. Exhaustive enumeration also feeds the permissions variant category, which is structural in our depth model precisely because it is enumerable rather than exercise-discovered. Make this an explicit decision rather than leaving two documents quietly disagreeing.

Related and already handled well: UI-hidden-but-API-allowed is item 1 in the risk-first inspection list, and the field guide correctly refuses to claim server-side enforcement from client-side evidence alone. The open question is only whether to hunt those actively — each is directly executable as a permission test, since a browser agent can navigate straight to the URL — or keep recording them defensively.

Where the connected skills live, and the release gate

Decided: the two new skills ship as a second plugin in this repo, and the release gate gets generalized to allow it. Not started — this is the spec.

doable-trd-context declares a context-only boundary — no MCP, no authentication, no calls to Doable. That is deliberate and documented, so the connected skills cannot live inside it.

scripts/verify-release.mjs enforces exactly what a connected plugin needs to break:

• release must contain exactly one Skill — counted across the whole repo
• no mcpServers / apps / hooks in any manifest
• scripts must be network-free — fetch, curl, axios, WebSocket all fail
• Authorization: Bearer in any text file is a hard failure, including in a documented curl example where it is only a placeholder

Those assertions back a real claim: the README says Doable never receives repository access and the plugin does not authenticate to Doable. The gate makes that mechanically checkable rather than a promise — a client's security reviewer can run npm test and see it enforced.

So generalize per-plugin rather than loosen globally. The verifier is written around a single plugin: pluginRoot and skillRoot constants, a flat required[] file list, const pluginName = "doable-trd-context", marketplace-entry assertions naming that source path, and version sync across three manifests plus package.json. Iterate over a declared set of plugins and attach each assertion to the plugin it belongs to:

• Skill count becomes per-plugin, not repo-wide.
• Network-free assertions and the Authorization: Bearer scan stay bound to doable-trd-context, so that plugin keeps its proof.
• The connected plugin gets its own rules: no secret values, Bearer only as a documented placeholder.
• Intake-specific checks (schema producer.skillVersion, renderer SKILL_VERSION) stay with intake.
• README.md and PRIVACY.md describe two products with different network postures.

What to avoid: deleting or globally relaxing the assertions to make the build pass. That retires a guarantee someone deliberately built, and nobody notices until a client asks. Done per-plugin, the only claim lost is "no plugin in this repo touches the network" — weaker than "this plugin doesn't, and CI proves it," which survives.

Rules that must survive any of this

Anchor every claim to something observable from outside — a route and status code, a button label, an error message, an email subject. Never a file path: those leak internals and a tester cannot act on them. Quote user-visible strings exactly; a test looking for the wrong button label fails for the wrong reason.

Code answers descriptive questions, never normative ones. What screens exist, what roles exist, what data is required — yes. What should happen when X — no.

Prove existence; never infer absence.

One confidence vocabulary. The truth planes already exist; do not add a parallel scale.

Every durable item carries the commit it was read at. Reusable knowledge without a staleness signal decays silently, and silent decay is worse than a gap.

What to measure

• Round-trip time from publishing a question to having an answer, against the email cycle it replaces.
• Discovery reused rather than repeated — how much of feature two's context came from what feature one already established. This is the number the knowledge-base framing exists to move.
• TRD unknowns resolved, versus how many stayed open.
• Spec-versus-code disagreements surfaced, and how many were real bugs.
• Test cases that would not have existed otherwise, especially permission tests.
• False negatives: a capability omitted or called absent that was actually shipped. If gap 1 is fixed, this should go to zero.

Blocking, before any of this functions

• Two tables in qa-supabase: code_context_questions and code_context_repo_maps. DDL is in the module docstring at doable-platform: src/qa_app/repositories/code_context/supabase.py.
• API docs: run python scripts/export_api_docs.py in doable-platform, commit inside the docs/api submodule, bump the pointer.
• An organization API key for the pilot client, plus the API base URL.