Skip to content

[finding] publish-smoke's log scan cannot name a WARN-level boot failure — it runs after the probes and matches only error/fatal #16793

Description

@claude

The registry canary's server-log scan cannot name a WARN-level boot failure, so when the boot is the defect the job reports a downstream probe instead. Filed from #16500, where this cost the triage seat a full log read to reach the root cause.

Evidence

Run 34084559243, job 101626009369. The defect was one line, at boot:

AuthPlugin failed to load: The requested module '@better-auth/core/db'
  does not provide an export named 'createLocalAccountIssuer'

The job did not exit on that. It ran every auth and CRUD probe against a server with no auth, and exited 1 on a probe. The log scan that could have named the cause runs after those probes.

Mechanism

scripts/publish-smoke.sh section 4 (log scan) is the last thing in the script, after section 3 (probes), and its pattern matches error and fatal only:

grep -nE '^\[(error|fatal)\]|"level":"(error|fatal)"|^\S+Z ERROR |Failed to register OIDC discovery routes'

Two properties combine:

  1. Severity. A plugin that fails to load logs at WARN, not ERROR, so the scan does not match it.
  2. Order. Even at a matching severity the scan runs after the probes, so a boot defect is always reported as a probe failure first.

The section's own comment states the intent this misses: "The #3091 breakage announced itself at startup ... and would have been caught by ANY error-level line." That held for #3091 because that breakage logged at error level. This one did not, and the scan was structurally unable to see it.

Why it matters

The canary's whole value is naming which of several owners a failure belongs to. A boot-load failure and a genuine auth-behaviour regression are different owners, and today they produce the same job output — a red probe.

Not in this card

Suggested shape

Scan the boot log for failed to load between the boot and the probes, and fail there with the plugin named. That is a new assertion on the smoke's contract, not a mechanical edit, which is why it is filed rather than carried into #16500's PR.


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions