Skip to content

CI can now smoke-check a rendered page BEFORE merge, and nothing wires it up — prevention is still missing from the deploy pipeline #274

Description

@os-project-manager

Filed by the repo:objectos os-dev seat working #261 round 3, as an out-of-scope finding from PR #273. Unassigned and unlabelled, for PM triage.

Dedup-checked against open issues first: #265 asks for local preview to be able to serve a page at all (PR #273 makes it do so); #262 is the bundle size budget; #266 (the human gate) was closed not planned. None of them wires a rendering check into CI before merge. This card is the prevention half.

The gap

#269 / PR #270 built detection and recovery: the deploy is gated on CI, the tested artifact is published, the live site is smoke-checked after deploying, a bad deploy auto-rolls back, and a failure files a card. #266 was closed on the reasoning that the automated layer answers "deploys must not reach production unverified" without a human clicking on every prose fix.

That reasoning is sound, and it leaves one thing standing: the only environment in which a rendering defect is detected is production. The exposure window is short and recovery is automatic, but the first observer of a broken render is still a live visitor.

The reason it was built that way is #265: nobody could render a page in the Workers runtime before merge, so there was nothing to check.

What changed

PR #273 removes that constraint. With an incremental cache configured, opennextjs-cloudflare preview serves real prerendered pages under real workerd, with no Cloudflare credentials. Measured on that branch, against http://localhost:PORT:

node .github/scripts/smoke-docs.mjs --base http://localhost:8792
✓ smoke: 4 page(s) rendered, negative control demonstrated red

and, on main at d6f5dda with the same command, 21 findings — the outage of 2026-09-04 reproduced offline, from a pull request, before anything is published.

So the check that currently runs only after a deploy can run as a pull-request gate, using the same script, the same rules, and the same live negative control.

Shape of the work (the dev's call, not prescribed here)

  • Run opennextjs-cloudflare preview in the build job against the bundle that job already packages, then point smoke-docs.mjs at it.
  • ⚠️ It has to be the same script, not a second copy of its rules. Two implementations of "does the site render" drift, and the one that drifts is the one nobody watches.
  • The negative control matters more here than in production: a preview server that failed to start answers nothing, and "no findings" from a dead server is indistinguishable from a pass. smoke-docs.mjs already fetches a path that must not render and fails the run if it comes back clean — keep that, and check the preview reached Ready on before judging anything.
  • Cost is one extra preview boot on top of a build the job already does. Worth measuring before committing to it; if it is disproportionate, a measured "not worth it" is a fine outcome.

Why it is worth doing rather than leaning on the backstop

The chain deploy succeeds → smoke fails → rollback fires has still never executed. It was not exercised on 2026-09-04 (the rollback was manual, on maintainer instruction) and it has not been exercised since, because no deploy has succeeded. Whether or not it works, a check that fires before merge costs a red pull request; the same check firing after merge costs a live outage plus an untested recovery path.

Re-check

git fetch origin claude/issue-261-worker-size-second-attempt
cd apps/docs && pnpm exec opennextjs-cloudflare build && pnpm exec opennextjs-cloudflare preview -- --port 8792
node .github/scripts/smoke-docs.mjs --base http://localhost:8792   # green on that branch
git switch main && ...                                             # 21 findings

Related: #261 (the outage), #265 (the constraint PR #273 removes), #262 (the size budget), #269 / PR #270 (detection and recovery), #266 (the human gate, declined).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions