Fix nightly rolling-upgrade rehearsal

[hardbyte]

Summary

• use migrations::CURRENT_VERSION in every rolling-upgrade schema assertion
• recheck queue-storage readiness when an atomic migration commit races runtime startup diagnostics
• include compat-matrix and rolling-upgrade-rehearsal in nightly failure notifications
• remove stale wording that described v043 as the current schema

Root cause

Migration v044 advanced the current schema while the release-gate rehearsal still asserted the literal version 43, making every cell fail deterministically after a successful migration.

Repeated local validation also exposed a startup race in the overlap cell. The readiness probe could observe the pre-migration schema, then the version probe could observe the committed current schema and misclassify the transient state as an unprepared installation. The startup path now carries the observed version and rechecks readiness when those probes straddle the commit.

Validation

• cargo fmt --all -- --check
• SQLX_OFFLINE=true cargo clippy --workspace --all-targets --all-features -- -D warnings
• SQLX_OFFLINE=true cargo build --workspace
• cargo test --package awa-worker — 49 passed, 2 ignored
• exact formerly failing rolling-upgrade cell
• complete five-cell rolling-upgrade rehearsal twice — 5 passed each run
• nightly workflow YAML parsed and every job is included in failure notification dependencies

Summary by CodeRabbit

• Bug Fixes

  • Improved queue storage readiness handling during migrations, reducing false stale-schema errors when migrations complete between checks.
  • Error messages now report the observed schema version for clearer troubleshooting.
  • Nightly workflow failure notifications now include compatibility-matrix and rolling-upgrade rehearsal results.

• Tests

  • Updated rolling-upgrade validation to track the current migration version automatically, improving resilience as migrations evolve.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-08-30T04:23:40.787341Z	de4f32b	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 918a8ba3-3c12-4783-8bfe-88a8c5d749c8

📥 Commits

Reviewing files that changed from the base of the PR and between 9d1196f and de4f32b.

📒 Files selected for processing (3)

• .github/workflows/nightly-chaos.yml
• awa-worker/src/client.rs
• awa/tests/rolling_upgrade_rehearsal_test.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Changes

The worker now re-probes queue-storage readiness when a migration commits between probes. Rehearsal tests use migrations::CURRENT_VERSION instead of version 43. Nightly failure issues now include compatibility and rolling-upgrade job results.

Queue readiness and rehearsal validation

Layer / File(s)	Summary
Queue-storage readiness re-probe awa-worker/src/client.rs	resolve_effective_storage rechecks readiness after observing CURRENT_VERSION. Error construction now receives the observed schema version synchronously.
Current-schema rehearsal assertions awa/tests/rolling_upgrade_rehearsal_test.rs	Rehearsal assertions, comments, and phase messages now use current-schema terminology and migrations::CURRENT_VERSION.

Nightly failure reporting

Layer / File(s)	Summary
Nightly failure issue updates .github/workflows/nightly-chaos.yml	notify-failure waits for compat-matrix and rolling-upgrade-rehearsal and includes both results in the issue body.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to de4f3

The PR corrects rolling-upgrade schema assertions, handles a startup race without bypassing readiness checks, and updates nightly failure notifications; no actionable merge-blocking risk remains after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant resolve_effective_storage
  participant current_version_readonly
  participant queue_storage_schema_ready
  resolve_effective_storage->>current_version_readonly: read observed schema version
  current_version_readonly-->>resolve_effective_storage: return CURRENT_VERSION
  resolve_effective_storage->>queue_storage_schema_ready: re-probe queue-storage readiness
  queue_storage_schema_ready-->>resolve_effective_storage: return readiness result

Loading

Poem

A rabbit checks the schema gate

Migrations finish, readiness waits
Current versions guide the way
Nightly failures list the day
Tests follow what changes bring
And carrots cheer the upgraded thing

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the primary change: fixing the nightly rolling-upgrade rehearsal. It is concise, specific, and relevant to the changeset.
Docstring Coverage	✅ Passed	Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. (1 skipped: 1 u…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}