feat: bundle a stateless MCP server into the server at /mcp with OAuth 2.1#219
Merged
Conversation
Adds oauth_clients + oauth_auth_codes tables (RFC 7591 dynamic client registration + PKCE authorization codes) and a nullable tokens.expires_at column (NULL = never expires, preserving existing token behavior). Wires expiry enforcement into authenticateUser's API-token lookup and expired-row cleanup into the janitor. Also adds the mcpAuth middleware wrapper that attaches a WWW-Authenticate header to 401s so MCP clients can discover the OAuth metadata endpoint (mounted on /mcp in a later commit). Part of plan 003 (bundled MCP server + OAuth 2.1).
Both will back the stateless Streamable-HTTP /mcp endpoint added in a follow-up commit. Versions align with the lockfile's existing MCP SDK version (1.29.0, already present transitively via packages/mcp) and the latest @hono/mcp release (0.3.0). Part of plan 003 (bundled MCP server + OAuth 2.1).
Bundles a stateless Streamable-HTTP MCP server (@hono/mcp + @modelcontextprotocol/sdk) at POST /mcp: a fresh McpServer/transport is built per request (no Mcp-Session-Id, no server-to-client push), and every tool re-enters the REST API in-process via Hono's app.request(path, init, c.env) forwarding the caller's own Authorization/Cookie header - one source of truth for validation, limits, and response shapes. GET/DELETE /mcp return 405 (this server never needs to resume a stream or terminate a session). 15 devicesdk_* tools cover whoami, project/device listing, status, metrics, env vars, script upload/versions/deploy, hardware commands, and a local docs search. devicesdk_docs_search queries a SQLite FTS5 index built at server-build time from docs/public/**/*.md (apps/server/scripts/build-docs-index.ts, wired into the "build" script) - offline, version-pinned search with no runtime network call. The path-to-URL mapping mirrors apps/website/scripts/build-content.ts's routePathFromRel/resolveRoutePath. devicesdk_device_logs wraps GET .../logs faithfully even though that REST endpoint is currently deprecated (always 410, pointing at the watch WebSocket) - the tool surfaces that response as-is rather than special-casing it, consistent with the loopback design wrapping the REST API unchanged. Verified end to end: tools/list returns all 15 tools; tools/call exercised for every tool including error paths (503 device not connected, 404 unknown version, cross-user project isolation) against a live server instance. Part of plan 003 (bundled MCP server + OAuth 2.1).
Adds a minimal OAuth 2.1 authorization server hosted by the same process: authorization-code grant with mandatory PKCE (S256), open (rate-limited) dynamic client registration (RFC 7591), no refresh tokens - clients re-run the flow after the 30-day access token expires. Access tokens are managed API tokens (revocable in the dashboard's Tokens page), minted with description "MCP: <client_name>". - oauth/metadata.ts: RFC 9728 protected-resource + RFC 8414 authorization-server discovery documents, issuer/endpoints derived from the request origin. - oauth/register.ts: POST /oauth/register, permissive redirect_uri validation (https, any http including non-loopback - this is a LAN self-host product - or a custom scheme for native clients). - oauth/authorizePage.ts: GET/POST /oauth/authorize consent screen behind cliAuthUser, modeled on endpoints/cli-auth/approvalPage.ts (CSRF cookie, same card UI). Client/redirect_uri mismatches render an error page rather than redirecting (OAuth security BCP); every other validation failure redirects back to the now-trusted redirect_uri with ?error=. - oauth/token.ts: POST /oauth/token, authorization_code grant only, RFC 6749 error shape. Validates the code, PKCE, client_id, and redirect_uri (in that order) before deleting the code and minting the token, so a client that makes a correctable mistake isn't locked out of its still-valid code. - oauth/store.ts: shared DB helpers + the RFC 6749/7591 JSON error helper. Verified end to end against a live server: DCR (success + invalid_client_metadata), full authorize -> consent -> PKCE token exchange, code single-use and PKCE/ redirect_uri mismatch rejections, deny path, minted-token visibility in /v1/tokens (managed:true) and revocation via DELETE locking out /mcp, and the auth.ts expiry regression case both ways (NULL-expiry token keeps working; forced-expiry token 401s on both REST and /mcp). Part of plan 003 (bundled MCP server + OAuth 2.1).
…cs indexer - tests/e2e/mcp.test.ts: 401+WWW-Authenticate without credentials, GET/DELETE 405, tools/list returns exactly the 15 registered tools, tools/call for whoami/list_projects/env_set+env_list roundtrip/send_command against an unconnected device/docs_search, and cross-user project-list isolation. Builds a real docs index from docs/public in beforeAll so docs_search is exercised against actual content. - tests/e2e/oauth.test.ts: discovery metadata shape, DCR happy path + invalid_client_metadata, full authorize->consent->PKCE token exchange with access token working on /mcp, code single-use / expiry / wrong-verifier / mismatched-redirect_uri all rejecting with invalid_grant, deny path, minted token visible in /v1/tokens (managed:true) with revocation locking out /mcp, and the auth.ts expires_at WHERE-clause regression both directions (forced-expiry token 401s on REST and /mcp; a NULL-expiry token keeps working). - tests/unit/docs-index.test.ts: runs the real indexer against a 3-page fixture dir, asserting page count, path mapping (dir/page.md -> /docs/dir/page/, _index.md -> /docs/), frontmatter passthrough, BM25 ranking (a densely-relevant page outranks a tangential mention), snippet content, and that a query made entirely of FTS5 operator syntax never throws. scripts/build-docs-index.ts: guard the `main()` side effect behind `import.meta.main` so importing the module for its exported helpers (as the new unit test does) no longer rebuilds the production index or calls process.exit() - a latent footgun the previous version had. docsSearch.ts: add resetDocsSearchCache() (mirrors foundation/logger.ts's resetLogger) so tests can point the module at a fresh index instead of being stuck with whatever was cached by an earlier call. Full suite: `bun test src tests` -> 344 pass, 0 fail, across 28 files. Coverage gate (`bun run scripts/coverage-gate.ts`, >=85% required): 89.6% functions, 96.44% lines. Part of plan 003 (bundled MCP server + OAuth 2.1).
…oint devicesdk init no longer preconfigures the npx @devicesdk/mcp stdio server (that package is being removed - the server now bundles MCP itself at /mcp). generateMcpJson() is now async and reuses the host already resolved by createProject() earlier in the same init() call (getApiUrl(), which caches env var / --host / ~/.devicesdk/credentials.json resolution), falling back to the default mDNS hostname (http://devicesdk.local:8080) if that somehow comes back empty. Test changes: mock getApiUrl() in init.test.ts (the real one does mDNS discovery + process.exit(1) with no host configured - never let a unit test touch that) and add coverage for the new .mcp.json shape, the fallback host, and not overwriting an existing .mcp.json. Part of plan 003 (bundled MCP server + OAuth 2.1).
The server now bundles MCP itself at /mcp (previous commits); the npm stdio package that shelled out to the CLI has no users to migrate (owner confirmed) and is removed outright rather than deprecated in place. - git rm -r packages/mcp, pnpm-lock.yaml updated. - AGENTS.md, README.md, CONTRIBUTING.md: drop the packages/mcp table rows and "CLI/MCP run on plain Node" phrasing (MCP is now part of the Bun server); add apps/server/src/mcp/ + apps/server/src/oauth/ to AGENTS.md's Source-of-Truth table and a Server-architecture bullet describing the MCP/OAuth subsystems, matching how every other subsystem there is documented. - docs/public/cli/init.md: the two .mcp.json mentions now describe the bundled-server / HTTP form instead of the npx package. Historical CHANGELOG.md entries and docs/public/changelog.md's past-release notes are left untouched (they describe what shipped at the time). Unpublishing/deprecating the already-published npm versions needs npm credentials a human has to run - not attempted here. docs/public/mcp.md itself is rewritten in the next commit. BREAKING CHANGE: @devicesdk/mcp is no longer published from this repo. Existing installs of the npm package keep working against whatever server they're pointed at (it's a thin CLI wrapper) but will not receive updates. Part of plan 003 (bundled MCP server + OAuth 2.1).
Follow-up to the previous commit (packages/mcp removal) - this is the actual doc/reference cleanup that commit's message described (a failed `git add` with a stale pathspec left it out of that commit; splitting into its own commit rather than amending). - AGENTS.md, README.md, CONTRIBUTING.md: drop the packages/mcp table rows and "CLI/MCP run on plain Node" phrasing (MCP is now part of the Bun server, not a separate Node package); add apps/server/src/mcp/ and apps/server/src/oauth/ to AGENTS.md's Source-of-Truth table plus a Server-architecture bullet describing the MCP/OAuth subsystems, matching how every other subsystem there is documented. - docs/public/cli/init.md: the two .mcp.json mentions now describe the bundled-server / HTTP form instead of the npx package. - pnpm-lock.yaml: regenerated by `pnpm install` after `git rm -r packages/mcp`. Historical CHANGELOG.md entries and docs/public/changelog.md's past-release notes are left untouched (they describe what shipped at the time). docs/public/mcp.md itself is rewritten in the next commit. Part of plan 003 (bundled MCP server + OAuth 2.1).
Full rewrite of docs/public/mcp.md around the server-bundled MCP endpoint instead of the removed @devicesdk/mcp npm package: HTTP .mcp.json quickstart, per-host config verified against each host's current docs (Claude Code's `claude mcp add --transport http`, Cursor's url-only mcpServers shape, VS Code's servers/type:http shape at .vscode/mcp.json, OpenCode's opencode.json mcp/type:remote shape), OAuth-recommended / API-token-alternative authentication section, the real 15-tool table (replacing the old CLI-wrapper 7-tool list), an honest note that devicesdk_device_logs currently always surfaces the REST endpoint's 410 deprecation, and an mDNS section. Frontmatter (title/description/weight/ social_image) keeps the same keys the site's build-content.ts expects. Verified: apps/website's own build-content.ts (gray-matter + markdown-it, not my server-side indexer) parses the new frontmatter and generates 68 pages including /docs/mcp/ with the expected title/description; apps/server's docs-index builder re-indexes it correctly at the same path; `pnpm lint --filter @devicesdk/website` and `pnpm check-types --filter @devicesdk/website` both exit 0. Part of plan 003 (bundled MCP server + OAuth 2.1).
…mage Adds the docs-index build step (Step 3a) to the serverbuild stage and copies docs-index.sqlite + sets DOCS_INDEX_PATH in the runtime stage. .dockerignore excluded the whole docs/ directory from the build context, so stage 1's `COPY . .` never actually brought docs/public in - the RUN step failed with "Docs directory not found: /repo/docs/public". Removed the bare `docs` line; docs/public is 61 small markdown files (384K) with nothing sensitive, and now the only place it's used to build a Docker image is compiled into docs-index.sqlite - no raw markdown ships in the runtime image (verified: `find / -name '*.md'` in a running container only turns up the pre-existing migrations/README.md). Verified end-to-end: full `docker build` succeeds, and inside the running container /mcp lists all 15 tools and devicesdk_docs_search returns real BM25-ranked results against the built index.
Minor bump for @devicesdk/server (new /mcp + OAuth subsystem) and @devicesdk/cli (init scaffolds .mcp.json for the bundled endpoint instead of @devicesdk/mcp), patch for @devicesdk/website (MCP docs page rewrite). No entry for @devicesdk/mcp - the package is deleted, so changesets cannot reference it.
…test src/tests gotchas Two things this task hit that cost real debugging time and will bite again: (1) .dockerignore's blanket `docs` exclusion silently breaks any future Docker build step that reads docs/public, no matter how many COPY --from stages sit between it and the RUN that needs it; (2) apps/server's plain `test` script (`bun test src`) never runs tests/e2e or tests/unit - only `test:e2e`/`test:coverage` do - so a green `pnpm test --filter @devicesdk/server` is not proof those suites passed.
…e filter missed The plan's own verify command (grep --include='*.ts' --include='*.json' --include='*.md') is blind to plain-text and .txt files. Broadening the grep past that filter turned up two live, stale references: - NOTICE: the npm-published-packages list still named @devicesdk/mcp (removed, so no longer accurate). - apps/website/src/llms.txt: the hand-authored LLM-crawler manifest (copied verbatim into static/llms.txt by build-content.ts) still pointed agents at `npx @devicesdk/mcp` instead of the bundled /mcp endpoint - the same fix already applied to docs/public/mcp.md. Left untouched (correctly, per the plan's own grep exclusions and by the same logic applied to CHANGELOG.md files): docs/public/changelog.md (historical release-announcement record), init.test.ts's negative assertion, and this branch's own new prose in tools.ts's comment and the changeset.
Reviewed and approved: OAuth 2.1 flow, MCP tool loopback, and the auth.ts token-expiry change all verified independently (re-ran the full test suites, lint, typecheck, and a scoped build rather than trusting the executor's report). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s-read quota to protect The 410 dated from the Cloudflare Durable Object era, where polling this endpoint burned a per-device daily rows-read quota. The self-host refactor replaced that with local SQLite, so there is no quota left to protect; the endpoint now returns a real cursor-paginated page of persisted logs (newest first, optional level filter), scoped to the caller's project. This resolves the one open item from the bundled-MCP-server review: devicesdk_device_logs was a permanently-410ing stub. The watcher WebSocket stays the dashboard's live-tailing path, unchanged.
…-reading process.env devicesdk_docs_search called searchDocs(), which lazily loaded its own config via a bare loadConfig() (defaulting to process.env). The e2e test harness sets DOCS_INDEX_PATH via an object passed to loadConfig(overrides), never touching process.env, so this second, disconnected config read always resolved to the default dist/docs-index.sqlite - present locally only by accident of a stale build artifact, absent in CI (which never builds apps/server), causing devicesdk_docs_search to deterministically fail with docs_index_missing in CI while appearing to pass locally. Fix: pass c.env.config.docsIndexPath through ToolDeps into searchDocs instead of having docsSearch.ts re-derive config independently - one source of truth for the path, matching how every other MCP tool already gets its context via the loopback. searchDocs() now takes the path as a parameter and caches its DB handle keyed by path, so multiple servers in one process (e2e test harness) get independent handles instead of racing over a single global cache. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@devicesdk/serveratPOST /mcp- 15devicesdk_*tools covering projects, devices, env vars, script versions, commands, and offline docs search (SQLite FTS5 index built at server-build time, shipped in the Docker image, no network call at runtime).app.request), so tool behavior always matches the server's own REST handlers - one source of truth, no duplicated query logic./oauth/*,/.well-known/oauth-*) additive to existing static API tokens: PKCE (S256 required), open rate-limited dynamic client registration (RFC 7591), no refresh grant. Consent approval mints amanagedAPI token capped at 30 days, revocable from the dashboard's existing Tokens page.authenticateUsergains a nullabletokens.expires_atcheck (NULL = never expires, preserving all pre-existing tokens' behavior).@devicesdk/mcpnpm package outright (owner-confirmed no users rely on it) -devicesdk initnow scaffolds.mcp.jsonpointing at the server's own/mcpendpoint instead ofnpx @devicesdk/mcp.GET /v1/projects/:projectId/devices/:deviceId/logs: it was a permanent410 Goneleft over from the Cloudflare Durable Object era (polling it burned a per-device rows-read quota). That quota doesn't exist on the self-hosted SQLite server, so it now returns a real cursor-paginated page of persisted logs (newest first, optional level filter). This is what makesdevicesdk_device_logsa working tool instead of a stub; the watcher WebSocket (/watch?backfillLimit=N) remains the dashboard's live-tailing path, unchanged.Generated from
plans/003-bundled-mcp-server.mdvia the/improve executeworkflow: implemented by a dispatched executor in an isolated worktree, then independently reviewed (full diff read, security-critical paths re-derived by hand, test suites/lint/typecheck/build re-run rather than trusted from the executor's report). The logs un-deprecation was a follow-up fix for the one open item the review flagged.Human-only follow-up (not attempted here, needs npm credentials)
Deprecate or unpublish the already-published
@devicesdk/mcpversions on the npm registry, e.g.:Test plan
pnpm test --filter @devicesdk/server- 347 pass (incl.mcp.test.ts,oauth.test.ts,docs-index.test.ts, rewrittenlogs.test.ts)pnpm test --filter @devicesdk/cli- 228 passpnpm lint/pnpm check-types --filter @devicesdk/server --filter @devicesdk/dashboard- cleanpnpm buildof every in-scope package - clean; docs FTS5 index builds 60 pages in the real build pipeline/health,/mcp401 +WWW-Authenticate, 15 tools, docs search)auth.tstoken-expiry regression tested in both directions against live DB state: an OAuth-minted token stops working after forced expiry (both REST and MCP paths), a NULL-expiry token created via the realPOST /v1/tokensendpoint keeps workingdevicesdk_list_projectsnever includes user A's projectCookieheader (not justAuthorization, which is all the plan specified) - verified safe: the dashboard session cookie isSameSite=Lax(pre-existing), which blocks it from attaching to any cross-site POST, so this doesn't open a CSRF path to the mutating tools (env_set,send_command,upload_script, etc.)logs.test.ts: empty page, newest-first ordering, level filter, cursor pagination across 3 pages, malformed-cursor 400, cross-project 404, unauthenticated 401🤖 Generated with Claude Code