tools-list: cover the 2026-07-28 SHOULD on deterministic tools/list ordering

[AmirK-S]

The requirement

docs/specification/2026-07-28/server/tools.mdx, "Capabilities" section, lines 71 to 74:

Servers SHOULD return tools in a deterministic order (i.e., the same ordering across requests when the underlying set of tools has not changed). Deterministic ordering enables clients to reliably cache the tool list and improves LLM prompt cache hit rates when tools are included in model context.

The sentence is new in 2026-07-28 (it is absent from 2025-11-25/server/tools.mdx). It landed through modelcontextprotocol/modelcontextprotocol#2516, merged 2026-04-13. It is not a SEP, so there is no src/seps/ row to attach it to.

The gap

Nothing in the suite exercises it. grep -rn deterministic src/scenarios src/seps only hits an unrelated comment in src/scenarios/server/tasks/lifecycle.ts, no src/seps/*.yaml row carries the sentence, and I found no open or closed issue or PR about tools/list ordering. The closest prior art is #332 and its PR #333, which were about the neighbouring MUST NOT on connection-invariance of the list, not about ordering. A server that reshuffles its tool list on every request passes tools-list today.

Proposed shape

One more check in the existing tools-list scenario rather than a new scenario, per "fewer scenarios, more checks":

• Check id tools-list-deterministic-order. SHOULD, so WARNING (severity follows the keyword, per AGENTS.md).
• Emitted from the 2026-07-28 wire on: the check carries source: { introducedIn: '2026-07-28' }, written as a literal since that revision is published and dated, and the scenario only takes the extra probes when ctx.specVersion is at or past it.
• The scenario sends three consecutive tools/list requests and compares the name sequences. Three rather than two so that a shuffling server cannot pass by repeating one order by chance as easily.
• Same multiset of names, different order: WARNING, with the first divergent probe and index in errorMessage and the three orders in details.orders.
• Multiset changed between probes: reported as untestable under the Standardize how scenarios report setup/execution failures #248 policy (Not testable: ..., details.untestable: true, WARNING severity), not as a violation. The spec scopes the SHOULD to "when the underlying set of tools has not changed", so from the outside a sample cannot tell a nondeterministic server from one whose tools legitimately changed between two requests.
• Fewer than two tools on every probe: INFO, nothing to compare, mirroring tools-name-format.
• A repeated tools/list request that fails is reported as untestable on the new check, so it never turns tools-list itself red.

The neighbouring MUST NOT in the same paragraph ("MUST NOT vary per-connection or as a side effect of other requests on the connection") is deliberately left out, following the reasoning in #332.

Negative fixture: examples/servers/typescript/tools-list-rotated-order.ts, a server on the sessionless 2026-07-28 wire that advertises the same four tools and rotates the list by one position on every tools/list, plus a negative.test.ts case asserting WARNING on the new check while tools-list itself still passes. Unit tests cover the helper that builds the check.

On the sampling concern from #332

#332 closed the proposal for list connection-invariance checks (PR #333) because a two-snapshot probe "can only sample" a "doesn't vary" claim, "with a real false-positive risk against servers whose lists legitimately change over time". This requirement reads differently to me: the spec text itself carves out the set-changed case, so the check treats any set change as untestable and only reports a violation when the identical set comes back in a different order. I do not see a path left where a conformant server is reported as violating the SHOULD. A set change does still surface as a WARNING under the untestable policy, which CI treats as a failure, so a server whose tools legitimately change between two back-to-back requests would show yellow rather than green. If that is the wrong trade, the alternatives are SKIPPED for that branch, or retrying the probes until the set is stable. Happy to go either way.

One question on the frozen requirement set

requirements/2026-07-28.yaml lists tools-list as scored and is frozen at scenario granularity, so a new WARNING check inside tools-list raises the bar for a revision that already shipped: an implementation at 100% could drop without moving. The requirement itself is 2026-07-28 text, so it belongs to that revision rather than being accumulated on top of it, but the file has no added-after-release marker at check granularity. Tell me which you prefer: leave it as is, or add a way to mark a check as not scored for a frozen revision. The new check id will also land in untracked in src/seps/traceability.json for lack of a yaml row; I can add one if you want the sentence tracked.

Evidence so far

• everything-server, --spec-version 2026-07-28: tools-list, tools-name-format, tools-list-deterministic-order, wire-schema-valid all SUCCESS.
• rotating fixture, same wire: tools-list-deterministic-order WARNING, the other three SUCCESS.
• typescript-sdk conformance server at main as of 2026-09-05 (5119ee7fd779), via npm start -- sdk typescript-sdk --mode server --scenario tools-list --spec-version 2026-07-28: 4/4, tools-list-deterministic-order SUCCESS on 28 tools.
• python-sdk conformance server at main as of 2026-09-05 (7bb486a10fa6), same command with python-sdk: 4/4, tools-list-deterministic-order SUCCESS.
• npm run check and npm run build pass; npm test: 536 passed, 0 failed (524 on main). The three "Unhandled Rejection" entries vitest prints are present on main before this change. Full outputs will be attached to the PR.

If this shape is acceptable I will open the PR against main.

AI disclosure

Per AI_POLICY.md: this issue, the check, the fixture and the tests were written primarily by Claude Code, pointed at this one SHOULD as AGENTS.md asks, under my direction. Replies in this thread may be AI-assisted as well.

[AmirK-S]

PR opened: #492. It carries the check, the rotating-order fixture, the negative test, and the typescript-sdk and python-sdk runs. The two questions above (frozen requirement set, traceability row) are restated in the PR notes; I will follow whatever you decide on them.