fix(edge): stop agent-edge 404ing every real /api/* route

[sarthakagrawal927]

The bug

agent-edge.mjs gained a catch-all in 4c67733 ("Add Clarity, an OpenAPI spec, and Accept-aware caching"):

// JSON error for unknown /api/* paths
if (path.startsWith('/api/')) {
  return jsonError(404, 'not_found', `Unknown API path: ${path}`, path);
}

worker.mjs calls handleAgentEdge(request) before delegating to OpenNext, and that handler engages for GET/HEAD with only /api/ai allow-listed above the catch-all. So every other GET /api/* was answered by the edge and never reached Next.js. It shipped 2026-08-23 and went unnoticed for six days.

Reproduction — the GET/POST split

The same path answers differently by method, which proves the route handler exists and only GETs are being swallowed at the edge:

$ curl -s -o /dev/null -w '%{http_code}' https://starboard.codevetter.com/api/discover
404          # {"error":{"code":"not_found","message":"Unknown API path: /api/discover", ...}}

$ curl -s -o /dev/null -w '%{http_code}' -X POST -d '{}' https://starboard.codevetter.com/api/discover
405          # Next.js answering "method not allowed" — the route is alive

$ GET /api/health   -> 404   |  POST /api/health   -> 405
$ GET /api/tools    -> 404   |  POST /api/tools    -> 405
$ GET /api/ai       -> 200   (the one allow-listed path)
$ POST /api/definitely-not-a-real-path -> 404 HTML   (a genuinely absent path)

Every route that was being shadowed

All 31 src/app/api/**/route.ts handlers were behind the catch-all; /api/ai is not one of them (it is edge-owned), so 30 of 31 real handlers were unreachable on GET. The ones that export GET were outright broken; the rest were shadowed but only observable as a wrong status code.

Path	Exports GET	Broken on GET
/api/auth/[...nextauth] (session, csrf, providers, signin, signout, callback/github)	yes	yes
/api/catalog-updates	yes	yes
/api/discover	yes	yes
/api/github/projects	yes	yes
/api/growth	yes	yes
/api/health	yes	yes
/api/internal/embed-pending	yes	yes
/api/lists	yes	yes
/api/lists/public/[slug]	yes	yes
/api/project-preview	yes	yes
/api/projects	yes	yes
/api/projects/[slug]/intelligence	yes	yes
/api/projects/[slug]/recommendations	yes	yes
/api/repos/[repoId]	yes	yes
/api/repos/[repoId]/comments	yes	yes
/api/repos/[repoId]/similar	yes	yes
/api/repos/[repoId]/star-history	yes	yes
/api/repos/[repoId]/tools	yes	yes
/api/stars	yes	yes
/api/tools	yes	yes
/api/embeddings/generate	no (POST)	shadowed on GET
/api/internal/external-reviews/ingest	no (POST)	shadowed on GET
/api/internal/project-intelligence/run	no (POST)	shadowed on GET
/api/lists/[id]	no (PATCH/DELETE)	shadowed on GET
/api/lists/[id]/share	no (POST)	shadowed on GET
/api/projects/[slug]	no (DELETE)	shadowed on GET
/api/repos/[repoId]/comments/[commentId]/vote	no (POST)	shadowed on GET
/api/repos/[repoId]/likes	no (POST)	shadowed on GET
/api/repos/[repoId]/list	no (PUT)	shadowed on GET
/api/repos/[repoId]/save	no (PUT)	shadowed on GET
/api/stars/sync	no (POST)	shadowed on GET

Blast radius beyond the four MCP tools flagged in the report:

• GitHub sign-in was broken. NextAuth's whole surface is GET-based — /api/auth/session, /api/auth/csrf, /api/auth/providers, /api/auth/signin, and critically /api/auth/callback/github, the OAuth redirect target. All 404'd.
• /api/health was down, so any uptime/health probe pointed at it has been reporting the site as broken (or the check itself was silently mis-reading a JSON 404).
• All four tools of the public Starboard MCP connector in sass-maker/chatgpt-connections.

The fix, and why this shape

The root cause is not the allow-list being short — it is that the edge was answering a question only Next.js can answer. Any allow-list has to be re-edited every time a route is added, and nothing makes that failure loud; the bug shipped precisely because someone added the catch-all without a mechanism that notices new routes.

So this inverts who decides existence:

• handleAgentEdge becomes a strict pre-handler. It answers only for surfaces the edge genuinely owns — /llms.txt, /llms-full.txt, /index.md, /openapi.json, /api/ai (now via an explicitly named EDGE_OWNED_API_PATHS set), and homepage markdown negotiation — and returns null for everything else. It can no longer conclude that a path does not exist.
• The JSON 404 moves to a post-handler, withApiJsonNotFound. worker.mjs calls OpenNext first and passes the response through it on the way out. Only when Next.js itself returns 404 for an /api/* path does the HTML error page get swapped for the JSON envelope.

There is still a one-entry allow-list (EDGE_OWNED_API_PATHS), but it can no longer rot the way the old one did, because it is additive, not subtractive. It grants the edge one path; it does not deny anything. A new route handler under src/app/api/ is reachable with zero edge changes. The only way to break a route now is to add its exact path to EDGE_OWNED_API_PATHS on purpose — and the filesystem-derived test below fails immediately if anyone does.

Two smaller pieces of the same shape:

• The markdown-404 branch now skips /api/*. Previously the catch-all ran first so this never triggered for API paths; with the catch-all gone, an Accept: text/markdown header would have diverted a real API request into a markdown 404. An Accept header must never stop a request from reaching its handler.
• withApiJsonNotFound leaves a route handler's own JSON 404 body alone (checks content-type), and is scoped to GET/HEAD, matching 4c67733's original scope — POST /api/unknown still gets the Next.js HTML 404 exactly as it does today.

Everything 4c67733 intended still works: the Clarity/agent surfaces, the OpenAPI spec at /openapi.json, Accept-aware caching, /api/ai, and a JSON (not HTML) 404 for a genuinely unknown /api/* path. All of that is asserted in the new tests.

Regression tests, and the non-vacuousness proof

Two levels, both verified to fail against the current main code by stashing only agent-edge.mjs + worker.mjs and re-running.

1. src/__tests__/agent-edge-api-routing.test.ts — drives the real worker.mjs entrypoint (not a reimplementation of it) with a stubbed OpenNext handler. .open-next/worker.js is a gitignored build artifact absent in CI, so vitest.config.ts aliases that import to src/__tests__/fixtures/open-next-worker-stub.mjs.

The guarded route list is read off the filesystem, not hard-coded — the exact rot the old allow-list suffered:

const API_ROUTE_PATHS = collectApiRoutePaths(resolve(__dirname, '../app/api'));

it.each(API_ROUTE_PATHS)('lets GET %s reach Next.js', async (path) => { ... });

A route handler added under src/app/api/ is covered the moment it lands.

BEFORE the fix — 35 failed / 7 passed

$ git stash push -- agent-edge.mjs worker.mjs
$ pnpm vitest run src/__tests__/agent-edge-api-routing.test.ts

     × reaches a real API route through the edge on GET 10ms
     × lets GET /api/auth/sample/segment reach Next.js 1ms
     × lets GET /api/catalog-updates reach Next.js 0ms
     × lets GET /api/discover reach Next.js 0ms
     × lets GET /api/embeddings/generate reach Next.js 0ms
     × lets GET /api/github/projects reach Next.js 0ms
     × lets GET /api/growth reach Next.js 0ms
     × lets GET /api/health reach Next.js 0ms
     × lets GET /api/internal/embed-pending reach Next.js 0ms
     × lets GET /api/internal/external-reviews/ingest reach Next.js 0ms
     × lets GET /api/internal/project-intelligence/run reach Next.js 0ms
     × lets GET /api/lists/sample reach Next.js 0ms
     × lets GET /api/lists/sample/share reach Next.js 0ms
     × lets GET /api/lists/public/sample reach Next.js 0ms
     × lets GET /api/lists reach Next.js 0ms
     × lets GET /api/project-preview reach Next.js 0ms
     × lets GET /api/projects/sample/intelligence reach Next.js 0ms
     × lets GET /api/projects/sample/recommendations reach Next.js 0ms
     × lets GET /api/projects/sample reach Next.js 0ms
     × lets GET /api/projects reach Next.js 0ms
     × lets GET /api/repos/sample/comments/sample/vote reach Next.js 0ms
     × lets GET /api/repos/sample/comments reach Next.js 0ms
     × lets GET /api/repos/sample/likes reach Next.js 0ms
     × lets GET /api/repos/sample/list reach Next.js 0ms
     × lets GET /api/repos/sample reach Next.js 0ms
     × lets GET /api/repos/sample/save reach Next.js 0ms
     × lets GET /api/repos/sample/similar reach Next.js 0ms
     × lets GET /api/repos/sample/star-history reach Next.js 0ms
     × lets GET /api/repos/sample/tools reach Next.js 0ms
     × lets GET /api/stars reach Next.js 0ms
     × lets GET /api/stars/sync reach Next.js 0ms
     × lets GET /api/tools reach Next.js 0ms
     × does not let an Accept header divert an API route to the markdown 404 0ms
     × replaces the Next.js HTML 404 with JSON 0ms
     × leaves a route handler's own 404 JSON body untouched 1ms
⎯⎯⎯ Failed Tests 35 ⎯⎯⎯
 Test Files  1 failed (1)
      Tests  35 failed | 7 passed (42)

AFTER the fix:

$ pnpm vitest run src/__tests__/agent-edge-api-routing.test.ts
 Test Files  1 passed (1)
      Tests  42 passed (42)

2. e2e/public-app.spec.ts — the same assertions through the real workerd runtime via wrangler dev, so the fix is proven in the environment that actually broke, not just in Node.

BEFORE (fix stashed):

  ✘  1 [desktop] › public API routes are reachable through the Worker edge on GET (70ms)

    Error: /api/health was 404'd — the edge is shadowing the Next.js route again
    expect(received).not.toBe(expected)
    Expected: not 404
  1 failed

AFTER:

  ✓  1 [desktop] › public API routes are reachable through the Worker edge on GET (309ms)
  1 passed (25.8s)

Verification

Ran pnpm check, pnpm typecheck, pnpm test, pnpm docs:check, pnpm quality:unused, and the desktop project of pnpm test:e2e.

$ pnpm check
Checked 243 files in 338ms. No fixes applied.
Found 1 warning.

$ pnpm typecheck
> tsc --noEmit          # clean

$ pnpm test
 Test Files  48 passed (48)
      Tests  291 passed (291)

$ pnpm docs:check
✓ 57 Markdown file(s) checked — no broken links or missing required files.

$ pnpm quality:unused
Unused: files=0, exports=0, types=0, dependencies=0, devDependencies=0, unlisted=0, unresolved=0.

$ pnpm test:e2e --project=desktop -g "public API routes are reachable"
  1 passed (25.8s)

Pre-existing, not touched: the single pnpm check warning is src/app/layout.tsx:75 — an ineffective biome-ignore lint/security/noDangerouslySetInnerHtml suppression on fleet-generated JSON-LD. It is present on main, unrelated to this change, and left alone.

Needs a deploy

This does not fix production on merge. Starboard deploys manually (pnpm deploy:cf), so starboard.codevetter.com keeps 404ing every GET /api/* — including the GitHub OAuth callback and /api/health — until someone runs a deploy. No deploy was performed as part of this PR.

Follow-up outside this repo (not fixed here)

agent-edge.mjs is listed in docs/development/conventions.md as generated by the fleet agent-edge generator. The fix here is in the generated output, so regenerating from the unfixed upstream template reintroduces the outage. I added a carry-forward note to that section of the conventions doc, and the new test fails loudly if the catch-all returns — but the generator template itself still needs the same change.

The same Unknown API path catch-all pattern is present across the fleet (calorie/src/agent-edge.mjs, free-ai/src/agent-edge.mjs, anime-list/src/agent-edge.mjs, everythingrated/apps/web/agent-edge.mjs, codevetter/apps/landing-page-astro/worker.mjs, and the functions/_middleware.ts variants in ~14 more repos). Any of those fronting an app with real /api/* routes has the same bug. I did not touch sibling repos; this is worth a fleet-wide sweep.

🤖 Generated with Claude Code