Skip to content

pack:verify should render an MCP App from the installed tarball, not just assert sandbox_proxy.html exists #2003

Description

@cliffhall

No check covers the intersection that #1859 fell through, and the two that come closest each miss it by one axis:

Check Runs against Drives the App path?
smoke:web:app the repo build tree yes — connect → open app → data-app-status="ready"
pack:verify the installed tarball no — only GET / on the prod web boot

So a packaging regression that keeps the file present-but-unreachable (wrong relative position, a rename with a stale reader, a future .npmignore/allowlist edit) passes both: the repo smoke never installs, and pack:verify only asserts the path exists (scripts/pack-and-verify.mjs:179 tarball, :244 on disk) without ever loading a widget through it. smoke-web-app.mjs's own header already names this as the open seam — "pack:verify proves the file ships, this proves the App path works. Neither subsumes the other."

Verified manually against the v2.2.0 tarball (#1993): packed, installed into a throwaway consumer, ran the installed mcp-inspector --web, deep-linked into mcp_app_demo, and the widget reached data-app-status="ready" with mcp-app-demo initialized in App logs. That manual pass is the thing to automate.

Scope

Add a step to pack:verify, after the existing prod --web boot, that drives the App flow against the installed bin:

  1. Spawn test-servers/build/server-composable.js --config test-servers/configs/mcp-app-http.json and parse the announced URL from stderr (console.error) — the port is resolved by findAvailablePort() and is not necessarily the config's 3130.
  2. Navigate the same deep link smoke-web-app.mjs uses (serverUrl + transport=http + autoConnect=<token> + openApp=mcp_app_demo + appArgs + autoOpen=<token>).
  3. Assert [data-testid="apps-form"][data-app-status="ready"], the contract in clients/web/README.md.

Notes for whoever picks this up

  • Most of the logic already exists in scripts/smoke-web-app.mjs. Prefer extracting its drive-the-App-flow half into scripts/lib/ and calling it from both, over a second copy — the two would drift, and the deep-link shape is the part that silently rots.
  • The test server stays a repo fixture. It is not in the tarball and should not be; only the client under test comes from the install.
  • Playwright resolution: pack:verify runs from the repo root, so use the createRequire based at clients/web/package.json that both browser smokes use — a bare import("playwright") resolves relative to scripts/.
  • Cost: pack:verify is already network-bound (a real npm install) and local/release-only, not part of validate/ci, so a Chromium run is affordable here.
  • Teardown must stop the App server on every failure path — see Three small cleanups flagged in the v2.2.0 merge review #2000 item 3, the same orphan bug in the existing smoke.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

CIGithub ActionschoreMaintenance: deps, build tooling, CI, cleanup — no user-facing behavior changemcp-appsMCP Apps relatedv2Issues and PRs for v2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions