Skip to content

test(mcp): baseline official 2026-07-28 conformance #1661

Description

@groupthinking

Verified change

The official MCP conformance project merged several material checks on September 6–7, 2026:

  • PR #488: client authorization must send the RFC 8707 resource value exactly as published by Protected Resource Metadata; normalizing a bare origin by adding / can break exact-match authorization servers.
  • PR #466: the runner now sends resource in both authorization and token requests when configured.
  • PR #492: three consecutive tools/list calls on the 2026-07-28 wire must keep an unchanged tool set in deterministic order; divergence is a conformance warning that CI treats as failure.
  • PR #468: the validator now accepts extension result envelopes such as Tasks.

Repository search found no pinned official conformance baseline in EventRelay.

Why this matters

Agent Factory/OpenProtocol currently have project-specific fixture tests, including the draft Skills harness in #1655, but no implementation-neutral proof of the underlying MCP wire contract. Auth-resource normalization and non-deterministic discovery can respectively break enterprise OAuth and tool/prompt caches even when local unit tests pass.

Acceptance criteria

  • pin an immutable official conformance commit; do not consume floating main
  • inventory the MCP client/server surfaces being certified and record exclusions
  • exercise applicable 2026-07-28 server scenarios against fixture/local-only endpoints
  • exercise client authorization with a bare-origin PRM resource and prove exact preservation in both authorization and token requests
  • prove stable tools/list order across at least three unchanged probes
  • accept versioned extension result envelopes without weakening core JSON-RPC validation
  • retain explicit coverage for pre-2026 dated-wire notification behavior where supported
  • store a machine-readable receipt: conformance commit, implementation commit, scenario/check IDs, versions, warnings, failures, and skipped reasons
  • fail closed on required checks; do not hide warnings in aggregate success
  • run without production endpoints, secrets, paid providers, merge, or deployment

Relationship to existing work

This complements #1655; it does not replace Skills-specific origin, digest, collision, or approval-revocation checks.

Estimated effort

1–2 engineering days for a pinned local harness and baseline receipt.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions