Skip to content

fix(frontier-digest): stop the proposals phase blocking the cron's HTTP response - #446

Merged
github-actions[bot] merged 1 commit into
mainfrom
fix/frontier-digest-cron-timeout
Aug 30, 2026
Merged

fix(frontier-digest): stop the proposals phase blocking the cron's HTTP response#446
github-actions[bot] merged 1 commit into
mainfrom
fix/frontier-digest-cron-timeout

Conversation

@catomean

Copy link
Copy Markdown
Collaborator

Root cause

fc-cron@frontier-digest.service on the box failed this morning with curl exit 28 — a 120s timeout with 0 bytes received. scripts/install-hetzner-crons.sh generates /opt/fleetcrown/fc-cron.sh, which calls this route with curl -m 120.

The route awaited runFrontierProposals() synchronously before responding. That phase's model calls (generateProposals + the judge panel in src/lib/frontier/propose.ts) retry across a multi-vendor fallback chain (src/lib/groq.ts's fallback: true default), and each link burns its full per-link timeout before falling through on a degraded link. That alone can exceed two minutes, even though the public-facing digest half (ingest + rank, ingest.ts + digest.ts) reliably finishes in well under 30s.

Confirmed empirically rather than just from reading the timeouts: production's /frontier page already showed today's (2026-08-30) real digest content while systemctl status fc-cron@frontier-digest.service still showed failed. That proves the digest half succeeded fast — the proposals phase was what held the HTTP response past the cron's 120s ceiling.

Fix

Detach the runFrontierProposals() call — fire-and-forget, chained into a new logProposalsOutcome helper via .catch().then() — so the route returns as soon as the digest is saved. Proposals still run to completion and are still logged via logDebug exactly as before, just asynchronously. This is safe because the app runs as a persistent systemd-managed Next.js server, not serverless — the process keeps running after the response is flushed.

The JSON response's proposals field now reads "queued — outcome logged separately" instead of the full result object, since the result isn't known synchronously anymore. Nothing else in the repo parses that field (grepped) — only fc-cron.sh logs the raw response body as text.

Why no box-side change

scripts/install-hetzner-crons.sh / the box's /opt/fleetcrown/fc-cron.sh are untouched. Deploying this route.ts change alone fixes it: the wrapper's curl -m 120 will now always get a fast reply regardless of how slow/degraded the proposals-phase model chain is that day.

Verification

  • npx tsc --noEmit: 0 errors.
  • Pre-commit (npx eslint on staged files) and pre-push (full local verify gate, including test:home, check:schema, dogfood-skip) both passed.
  • Confirmed no other caller depends on the synchronous shape of the proposals response field.

🤖 Generated with Claude Code

…TP response

fc-cron@frontier-digest.service was failing with curl exit 28 (120s
timeout, 0 bytes received): the route awaited runFrontierProposals()
synchronously, and that phase's model calls retry across a multi-vendor
fallback chain (lib/groq.ts's fallback: true default), each burning its
full per-link timeout on a degraded link — easily exceeding the cron
wrapper's fixed `curl -m 120` even though ingest+digest (the public-facing
half) finish in well under 30s. Confirmed live: production's /frontier
page already showed today's real digest while the systemd unit still sat
failed, proving the digest half was fast and the proposals phase was what
blocked the response.

Detach the proposals call (fire-and-forget into a new logProposalsOutcome
helper) so the cron gets its response as soon as the digest is saved.
Proposals still run to completion and get logged via logDebug exactly as
before — just asynchronously, which is safe here since this is a
persistent systemd-managed server, not serverless. No box-side change
needed; deploying this alone fixes it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions
github-actions Bot merged commit 0c4e36a into main Aug 30, 2026
3 checks passed
@github-actions
github-actions Bot deleted the fix/frontier-digest-cron-timeout branch August 30, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant