Prevent false greens when a negative check rejects for the wrong reason

[Silentpartnercoding]

Problem

A negative conformance check can report SUCCESS when the implementation returns the expected rejection for an unrelated reason before the requirement under test is exercised.

Example: a vector is intended to verify that a declaration absent from a validly signed manifest is rejected. A broken verifier rejects the same input at an earlier version guard. A check that compares only the final verdict sees reject == reject and stays green even though manifest membership was never evaluated.

This failure mode was independently reported in the live signed-declarations discussion at modelcontextprotocol/modelcontextprotocol#3140. The thread also identified the security boundary: causal detail should be observable to the harness/auditor, not required on the MCP wire.

Minimal reproduction

Protocol-neutral fixture: https://gist.github.com/Silentpartnercoding/2c5ddb251d4b60f337b86b0452409950

It runs one vector against a correct verifier and a deliberately short-circuiting mutant:

Harness	Correct verifier	Short-circuit mutant
Verdict only	PASS	PASS — false green
Reason bound	PASS	FAIL — mutant killed

The fixture binds four machine-readable observations:

{
  "verdict": "reject",
  "property_under_test": "manifest_membership_binding",
  "property_reached": true,
  "stop_reason": "DECLARATION_NOT_IN_MANIFEST"
}

The mutant instead reports property_reached: false and stop_reason: "UNSUPPORTED_VERSION". Its final verdict is still reject.

Local result on Node v24.18.1: 3/3 tests pass, including the required false-green and mutant-kill assertions.

Relationship to current conformance behavior

This appears adjacent to, but narrower than, #248:

• untestableCheck() already makes a missing prerequisite red and records details: { untestable: true, reason }.
• AGENTS.md already requires deliberately broken implementations for new scenarios.
• Some helpers deliberately construct otherwise-valid inputs so a negative probe isolates one failure dimension.

Those are strong safeguards. The remaining gap is a reusable contract ensuring that an expected negative verdict came from the check's target property rather than an earlier guard. ConformanceCheck.details can already carry the observation, so this may require a convention/helper and negative meta-test rather than a new result status or runner.

Proposed direction for discussion

For negative checks where multiple rejection paths can produce the same wire result:

1. Name the target with the existing stable check ID or an equivalent property_under_test identifier.
2. Emit SUCCESS only after the harness has exercised and observed the intended property.
3. If an earlier guard or missing prerequisite prevents that, emit the same check ID as FAILURE/WARNING with machine-readable details identifying that the property was not reached and why.
4. Add a deliberately broken implementation that returns the expected wire verdict through the wrong guard; the vitest must prove the check rejects that mutant.

Questions for maintainers:

• Should this be an explicit negative-check convention in AGENTS.md, a small helper over ConformanceCheck.details, or simply applied case-by-case?
• Is the existing untestable detail shape the right vocabulary to extend, or should causal reachability remain separate from missing prerequisites?
• Once SEP-3140 has a sponsor and a normative spec diff, would its signed-manifest membership check be a useful pilot?

This issue does not propose an MCP wire field, a new CLI entry point, or implementation of SEP-3140 before its requirements are ready.

AI assistance disclosure: OpenAI Codex assisted with the reproduction, tests, analysis, and drafting this issue. I reviewed the fixture, evidence, and draft and remain responsible for the claims and publication.

[OfficialAbhinavSingh]

Happy to take this.

A concrete instance is already in the repo. auth/resource-mismatch decides its verdict from !authorizationRequestMade alone, so a client that throws before making a single HTTP request scores SUCCESS:

SUCCESS  resource-mismatch-rejected
"Client correctly rejected authorization when PRM resource does not match server URL"
details: { authorizationRequestMade: false }

It never requested the PRM document, so it never read the resource value it was required to compare. The four RFC 9207 rejection checks in issuer-parameter.ts have the same shape via authReached && !tokenRequestMade.

auth/metadata-issuer-mismatch, in that same file, already gates its verdict on the metadata fetch — so the pattern exists here, it just isn't applied consistently.

Plan: bind each verdict to the prerequisite the harness already observes (prm-pathbased-requested, authorization-server-metadata), and route "prerequisite not observed" through notTestable() (#248) rather than SUCCESS, recording propertyReached / stopReason in details. Plus an inert-client negative test so the check is shown to catch something and not just to avoid false-positiving.

One thing this does not close, and I would rather name it than imply otherwise: a client that receives the redirect and then aborts before the token request for an unrelated reason is still indistinguishable from one that rejected on iss. Separating those needs a signal from inside the client, which a black-box harness does not have. This closes the "never reached the requirement at all" class.