Skip to content

ci: the merge queue's Test Core runs the affected package set; selection moves into a script with a self-test #16471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
os-steve merged 3 commits into from
Sep 7, 2026
Merged

ci: the merge queue's Test Core runs the affected package set; selection moves into a script with a self-test #16471

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e6fe28e
ci: move Test Core package selection into a script; the merge queue r…
claude Sep 7, 2026
8991c1a
ci: the selection self-test declares its one repo path and spells no …
claude Sep 7, 2026
05cb57b
ci: the dispatch-gates specimen roster drops the invocation that move…
claude Sep 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 28 additions & 127 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,15 +350,17 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# PRs: only test packages affected by the diff against the PR base.
# spec sits at the root of the dependency graph, so spec-touching PRs
# still run (close to) everything — but the many PRs that don't touch
# spec skip the bulk of the 75-package matrix. Push to main and
# merge-queue builds partition the FULL package list instead: the queue
# result IS the next main, so it gets main's validation, not the PR's
# affected-only subset. (Spec's suite runs here plain / uninstrumented;
# the coverage-instrumented pass lives in the nightly coverage-nightly
# workflow.)
# PRs and merge-queue builds: only test packages affected by the diff —
# against the PR base on pull_request, against the group's own
# `base_sha` on merge_group (#16453). spec sits at the root of the
# dependency graph, so spec-touching diffs still run (close to)
# everything — but the many that don't touch spec skip the bulk of the
# 75-package matrix, and at ~120 queue merges a day that is the bulk of
# the queue's test compute. Push to main partitions the FULL package
# list: it gates nobody, and its full run is the ground truth the
# shard-timings refresh reads. (Spec's suite runs here plain /
# uninstrumented; the coverage-instrumented pass lives in the nightly
# coverage-nightly workflow.)
#
# !@objectstack/dogfood: the ~7½-minute dogfood suite is the dedicated
# Dogfood job's whole purpose, and both jobs run under the same `core`
Expand Down Expand Up @@ -415,127 +417,26 @@ jobs:
# `turbo ls`'s internal choice of dot-ness — undocumented, and
# `turbo ls` is experimental (see above). Resolving to a commit here
# leaves turbo no choice to make.
# The selection itself lives in scripts/ci/select-shard-packages.sh:
# environment in, `$RUNNER_TEMP/turbo-ls.json` out. This step only
# exports the event's fields, so the same script -- every branch of
# it -- runs locally and under its self-test (#16453). Per event:
# pull_request affected set against merge-base(origin/<base>, HEAD)
# merge_group affected set against the group's `base_sha`, unioned
# with the same cross-package scans. An EMPTY set is a
# legitimate docs-only group: it reaches the "No packages
# on this shard" exit below, every shard still attests,
# and Test Core is an honest green. It is NOT the #10057
# case, which is pull_request-only (the script says why).
# push unchanged: the FULL list.
- name: Compute this shard's package set
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
PINNED_BASE_SHA: ${{ github.event.pull_request.base.sha }}
OS_SHARD_EVENT_NAME: ${{ github.event_name }}
OS_SHARD_PR_BASE_REF: ${{ github.event.pull_request.base.ref }}
OS_SHARD_PR_PINNED_BASE_SHA: ${{ github.event.pull_request.base.sha }}
OS_SHARD_MERGE_GROUP_BASE_SHA: ${{ github.event.merge_group.base_sha }}
run: |
SCM_BASE=''
if [ "${{ github.event_name }}" = "pull_request" ]; then
if [ -z "$BASE_REF" ]; then
echo "::warning::This pull_request event carries no base branch, so the affected-set diff base cannot be computed."
else
# `fetch-depth: 0` above already makes this resolve — the fetch is
# the guard for the day that changes, not the normal path.
#
# `git cat-file -e` rather than the more idiomatic strict
# `git rev-parse --verify` spelling. That is history rather than
# style, and it is written down because it used to be a live
# hazard: the pre-#6589 check-shard-attestation.mjs classified a
# job as an aggregate GATE when the script's basename and its
# `--verify` flag merely CO-OCCURRED as substrings anywhere in
# the job's joined `run:` text. This job always carries the
# basename (its `--emit` step at the bottom), so spelling
# `--verify` anywhere in this step — comments included, since
# they were part of `run:` — silently reclassified the shard job
# as a gate.
#
# #6589 closed that by construction. Classification is now by
# INVOCATION: within one command the flag must follow the
# script's own name as an argument, the test is applied per STEP
# and never over the job's joined text, and the lexer drops shell
# comments — all three pinned by that script's `--self-test`,
# which is why this comment can now name the flag at all. Either
# spelling is safe here; `git cat-file -e` stays because
# churning it would buy nothing.
if ! git cat-file -e "refs/remotes/origin/$BASE_REF^{commit}" 2>/dev/null; then
git fetch --no-tags --quiet origin "+refs/heads/$BASE_REF:refs/remotes/origin/$BASE_REF" \
|| echo "::warning::Could not fetch origin/$BASE_REF; the merge-base resolution below will decide."
fi
# `if !` rather than a bare assignment on purpose: these steps run
# under `bash -e`, where a failing command substitution kills the
# step with no message at all.
if ! SCM_BASE=$(git merge-base "refs/remotes/origin/$BASE_REF" HEAD); then
SCM_BASE=''
fi
fi
fi
if [ -n "$SCM_BASE" ]; then
# The drift is printed, not just corrected: nothing in this log ever
# said which commit the affected diff started from, which is why the
# decay was invisible.
DRIFT=$(git rev-list --count "$PINNED_BASE_SHA..$SCM_BASE" 2>/dev/null || echo '?')
echo "Affected-set diff base: $SCM_BASE (merge-base of origin/$BASE_REF and HEAD)"
echo "Frozen payload base.sha: $PINNED_BASE_SHA -- $BASE_REF has moved $DRIFT commit(s) since it was frozen, and that drift is exactly what this step used to charge to this PR."
TURBO_SCM_BASE="$SCM_BASE" pnpm exec turbo ls --affected --output=json > "$RUNNER_TEMP/turbo-ls.json"
# `turbo ls --affected` answers "which packages does the dependency
# GRAPH reach from this diff" — and some suites read files the graph
# does not connect them to. spec's api-methods-batch-conformance scan
# walks every `*.object.ts` in the monorepo while spec declares no
# dependency on the packages it judges (nor should it: the scan reads
# source text precisely to avoid inverting the spec -> * direction).
# A platform-objects-only diff therefore left it unrun, and #7769
# landed a violation on `main` that only PRs touching `spec` ever saw.
# Measured on turbo 2.10.7: 51 packages affected by that diff, spec
# not among them.
#
# So packages that declare a cross-package input radius are unioned
# back in when the diff touches it. The declarations, and the static
# detector that refuses to let a new cross-package scan go
# undeclared, live in the script (`pnpm check:cross-package-test-inputs`).
#
# Failure here falls back to the FULL package list, never to the
# affected-only set: same posture as the merge-base fallback below
# (#6195), and the same reason — the full list is a strict superset,
# so doubt costs minutes rather than coverage. This is the FILTER
# CONTRACT's half 1 applied one layer down.
if ! git diff --name-only "$SCM_BASE" HEAD > "$RUNNER_TEMP/changed-files.txt" \
|| ! node scripts/check-cross-package-test-inputs.mjs \
--union-into "$RUNNER_TEMP/turbo-ls.json" \
--changed "$RUNNER_TEMP/changed-files.txt"; then
echo "::warning::Could not union cross-package scans into the affected set; falling back to the full package list for this shard."
pnpm exec turbo ls --output=json > "$RUNNER_TEMP/turbo-ls.json"
# A fourth signal the failure branches do not cover: a producer that
# exits 0 with a wrong, plausible, EMPTY answer. Only that `git
# diff`'s exit STATUS was checked, never its emptiness, so a
# merge-base resolving to something wrong-but-valid gave an empty
# changed-file list -> zero affected packages -> a green shard that
# tested nothing, with every log line reading like a normal quiet PR
# (#10057). The shard attestation (#6082) does not cover it: it
# attests "shard N ran and every step passed", which is exactly what
# a shard that tested nothing does.
#
# Empty is decidable as BROKEN here, and only here: a pull_request
# always differs from its merge-base. At the partitioner zero is
# frequently the CORRECT answer (a docs-only PR genuinely affects no
# package, and with 6 shards a small change legitimately leaves
# shards empty), so a blanket "red on empty" belongs there least of
# all -- this is the one place selection-failed and nothing-selected
# can be told apart.
#
# Scoping this to pull_request needs no `github.event_name` test:
# SCM_BASE is assigned only inside the `pull_request` guard above, so
# this whole `[ -n "$SCM_BASE" ]` branch is unreachable on push and
# merge_group. Those keep taking the full-list path in the `else`
# below, by design.
elif [ ! -s "$RUNNER_TEMP/changed-files.txt" ]; then
echo "::warning::The diff against merge-base $SCM_BASE listed no changed files, which a pull_request cannot legitimately produce; falling back to the full package list for this shard rather than selecting nothing (#10057)."
pnpm exec turbo ls --output=json > "$RUNNER_TEMP/turbo-ls.json"
fi
else
# Falling back to the FULL package list, never to the frozen
# base.sha. This is not the #4690 silent-skip anti-pattern: that is
# about a gate PASSING on input it could not read, and the full list
# is a strict superset of the affected one — this shard still runs
# everything it would have run and more. Cost is minutes; the
# alternative is a red Test Core on a PR with nothing wrong with it.
# Push and merge-queue builds take this branch by design (the queue
# result IS the next main, so it gets main's validation).
if [ "${{ github.event_name }}" = "pull_request" ]; then
echo "::warning::Could not resolve merge-base(origin/$BASE_REF, HEAD); falling back to the full package list for this shard rather than diffing from the frozen base.sha (#6195)."
fi
pnpm exec turbo ls --output=json > "$RUNNER_TEMP/turbo-ls.json"
fi
bash scripts/ci/select-shard-packages.sh
node scripts/partition-test-shards.mjs "$RUNNER_TEMP/turbo-ls.json" \
--shard ${{ matrix.shard }}/6 --exclude @objectstack/dogfood \
> "$RUNNER_TEMP/shard-packages.txt"
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3235,6 +3235,20 @@ jobs:
- name: Shard attestation gate
run: pnpm check:shard-attestation

# Test Core package selection self-test (#16453). ci.yml's "Compute this
# shard's package set" step is a thin call into
# scripts/ci/select-shard-packages.sh — environment in, turbo-ls.json
# out — so that the `merge_group` branch, which by construction cannot
# run before a PR is already in the queue, is proved here instead: the
# self-test drives every branch of every event (pull_request, merge_group,
# push) offline, against a throwaway upstream + clone and a fake `turbo`
# on PATH, and pins which `turbo ls` calls were made and with what base,
# whether the cross-package union ran, every `::warning::` fallback and
# the absence of the ones that must not fire (an empty docs-only merge
# group selects nothing WITHOUT the #10057 complaint). Offline; ~2s.
- name: Test Core package selection self-test
run: pnpm check:select-shard-packages

# Aggregator roster gate (#10490). Three required contexts are aggregate
# jobs standing in for a set of real jobs — `Test Core` and `Dogfood
# Regression Gate` in ci.yml, `TypeScript Type Check` in this file — and
Expand Down
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@
"check:node-version": "node scripts/check-node-version.mjs",
"check:pnpm-acquisition": "node scripts/check-pnpm-acquisition.mjs --self-test && node scripts/check-pnpm-acquisition.mjs",
"check:workflow-status-functions": "node scripts/check-workflow-status-functions.mjs --self-test && node scripts/check-workflow-status-functions.mjs",
"check:select-shard-packages": "bash scripts/ci/select-shard-packages.selftest.sh",
"check:shard-attestation": "node scripts/check-shard-attestation.mjs --self-test && node scripts/check-shard-attestation.mjs",
"check:required-contexts": "node scripts/check-required-contexts.mjs --self-test && node scripts/check-required-contexts.mjs",
"check:cross-package-test-inputs": "node scripts/check-cross-package-test-inputs.mjs --self-test && node scripts/check-cross-package-test-inputs.mjs",
Expand Down
Loading
Loading