ci: the merge queue's Test Core runs the affected package set; selection moves into a script with a self-test

[os-steve]

Fixes #16453

The merge queue's Test Core runs the affected package set instead of the full list. The "Compute this shard's package set" step in ci.yml becomes a thin call into a new script that reads plain environment variables, so every branch of every event runs locally and under a self-test — which is how the merge_group branch, which cannot run before a PR is already in the queue, is proved here before it lands. push on main is unchanged: the full list.

Maintainer-directed (part A of the test-cost programme; the authority is quoted on the card). Not a governed surface: check-governed-merges --test on this PR's six paths printed governed-surface predicate: 0 of 6 path(s) hit the register (5 surfaces, repo-agnostic). NOT governed — ordinary queue landing applies to a PR with exactly this file list. (exit 0).

What changed

file	change
scripts/ci/select-shard-packages.sh (new)	the selection: environment in (OS_SHARD_EVENT_NAME, OS_SHARD_PR_BASE_REF, OS_SHARD_PR_PINNED_BASE_SHA, OS_SHARD_MERGE_GROUP_BASE_SHA, RUNNER_TEMP), $RUNNER_TEMP/turbo-ls.json out; exit 0 on every decided outcome, 2 on a usage error, turbo's own status when turbo ls fails (the bash -e behaviour the inline step had)
scripts/ci/select-shard-packages.selftest.sh (new)	21 cases / 97 checks, hermetic and offline: a throwaway upstream + clone + shallow clone, a fake pnpm/turbo pair on PATH that records every call, a stub union script that records its argv; pins which turbo ls calls were made and with what TURBO_SCM_BASE, the union's changed-file list, every ::warning:: fallback text, and the ABSENCE of the ones that must not fire
.github/workflows/ci.yml	the step exports the four event fields and calls the script; the partition call stays in the step verbatim (partition-test-shards.mjs pins its spelling); the header comment states the per-event posture and the empty-set rule
.github/workflows/lint.yml	one step, Test Core package selection self-test, runs pnpm check:select-shard-packages (wiring the self-test into CI is what check:self-test-wired exists for)
package.json	check:select-shard-packages
scripts/pm/dispatch-gates.mjs	one self-test roster entry removed, reason recorded beside it: the case the card's named specimens all classify as VARIABLE from the workflow text listed scripts/check-cross-package-test-inputs.mjs, whose only value-bearing invocation was the inline step's --union-into … --changed … line that this PR moved into the script; with it gone from the workflow text check:pm-dispatch-gates failed 1 of 1534 on this branch (the rot its own block comment predicts for a typed list). Measured before the edit with the tool's exported extractCheckInvocations over the live workflows: the other five specimens still classify VARIABLE, check-cross-package-test-inputs.mjs no longer does anywhere. After the edit: ✓ dispatch-gates self-test: 1534 cases pass.

File-surface increment against the claim: lint.yml and package.json carry the self-test wiring only (one step, one script line); scripts/pm/dispatch-gates.mjs is the one-entry repair above, a red this PR itself causes. Flight X (#16455) edits a different ci.yml step and none of these files; git merge-tree --write-tree HEAD d2fab7ab77 against its pushed head exits 0 both plain and with -c merge.os-regen.driver=false (final head: both write tree 4c84c99cf3).

Per event

• pull_request — moved VERBATIM: the affected set against merge-base(origin/BASE_REF, HEAD), unioned with the cross-package scans; fallbacks ci.yml 的 TURBO_SCM_BASE 吃同一个冻结 base.sha —— turbo --affected 在 merge ref 上把 main 漂移算成本 PR 改动(方向保守:多跑,不会少跑) #6195 and [finding] Nothing distinguishes "zero packages affected" from "the affected-set computation silently produced nothing" — an empty changed-files.txt is never checked #10057 with their exact warning texts.
• merge_group — SCM_BASE is github.event.merge_group.base_sha; if the checkout cannot resolve it the script fetches it by sha (mirroring the PR branch's fetch of origin/BASE_REF), then TURBO_SCM_BASE=base_sha pnpm exec turbo ls --affected --output=json and the union over git diff --name-only base_sha HEAD, through the same code path the PR branch runs. Any failure to resolve the base, to fetch it, or to run the scans falls back to the FULL list with a ::warning:: naming the reason. An EMPTY selection is legitimate (a docs-only group) and is not the [finding] Nothing distinguishes "zero packages affected" from "the affected-set computation silently produced nothing" — an empty changed-files.txt is never checked #10057 case: that guard is now an explicit pull_request test, and the comment beside it says why the two cannot be told apart anywhere else.
• push — unchanged, the full list (the step comment says so in one line).

Byte-for-byte equivalence of the pull_request branch

The old run: block (121 lines at 6eba38f5a3, extracted and dedented; the round-trip re-indents to the identical bytes) was spliced into the script by line range, never retyped. diff -b old-inline new-script-body removes exactly 15 lines and adds 45; the 15 removed are, in full:

- if [ "${{ github.event_name }}" = "pull_request" ]; then        (plumbing, becomes "$EVENT_NAME", 2 sites)
-   # Scoping this to pull_request needs no `github.event_name` test:
-   # SCM_BASE is assigned only inside the `pull_request` guard above, so
-   # this whole `[ -n "$SCM_BASE" ]` branch is unreachable on push and
-   # merge_group. Those keep taking the full-list path in the `else`
-   # below, by design.                                              (comment rewritten: no longer true)
-   elif [ ! -s "$RUNNER_TEMP/changed-files.txt" ]; then           (gains `[ "$EVENT_NAME" = "pull_request" ] &&`)
-   # Push and merge-queue builds take this branch by design (the queue
-   # result IS the next main, so it gets main's validation).        (comment rewritten: push only)
-   if [ "${{ github.event_name }}" = "pull_request" ]; then        (plumbing)
- node scripts/partition-test-shards.mjs ...  (5 lines)             (the partition tail stays in the step)

Every other old line survives verbatim: 106 of 121 are present byte-for-byte in the script (indentation aside — the six drift/echo lines sit one level deeper under the new pull_request guard, with a merge_group else printing Affected-set diff base: SHA (the merge group's base_sha)). The added lines are the merge_group resolution block, the two event guards, the merge_group fallback warning, and the plumbing header.

Behavioural proof — the OLD step's run: block (with ${{ github.event_name }} = pull_request, ${{ matrix.shard }} = 1) and the NEW step's run: block were both executed under bash -e in a scratch clone against the same tree and the same event variables, real turbo, real union script, real partitioner:

pull_request shape	old vs new stdout+stderr	turbo-ls.json	shard-packages.txt	selected
(a) leaf change, base ref resolves	IDENTICAL (18 lines)	IDENTICAL 86f09bcdd964	IDENTICAL	53
(a2) empty base ref	IDENTICAL (17 lines, both #6195 warnings)	IDENTICAL e1ac2440598f	IDENTICAL	79 (full)
(a3) unresolvable base ref nope	IDENTICAL (19 lines, fetch + #6195 warnings)	IDENTICAL	IDENTICAL	79 (full)
(a4) HEAD == merge-base (#10057)	IDENTICAL (20 lines, the #10057 warning)	IDENTICAL	IDENTICAL	79 (full)

And live: this PR's own pull_request run of the new step on a runner — Test Core (1/6) … (6/6) and the aggregate Test Core all success on 05cb57b43c.

The four scenario runs (real commits in a scratch clone, git diff against the synthetic base T = e6fe28e94e; never pushed)

raw is turbo ls --affected alone at the same HEAD; selected is what the step wrote after the union.

scenario	changed file	raw turbo	union adds	selected	shards 1..6
(a) pull_request, leaf (packages/drivers/driver-sql/src/index.ts)	1	50	spec, core, types	53 (== old inline, see above)	—
(b) merge_group, same leaf change	1	50	spec, core, types	53	9 / 10 / 9 / 8 / 9 / 8
(b2) merge_group, TRUE leaf, .ts (packages/plugins/knowledge-ragflow/src/index.ts; zero dependents)	1	1	spec, core, types, objectql, runtime	6	—
(b3) merge_group, TRUE leaf, .md only (packages/plugins/knowledge-ragflow/CHANGELOG.md)	1	1	spec	2	—
(c) merge_group, packages/spec/src/index.ts	1	75	none	75	16 / 13 / 13 / 11 / 11 / 11
(d) merge_group, docs-only (.claude/agents/os-dev.md, the 01:18Z shape)	1	0	none	0	0 / 0 / 0 / 0 / 0 / 0
(d2) merge_group, docs-only (root README.md)	1	0	none	0	—
(d3) merge_group, docs-only under content/** (content/docs/concepts/north-star.mdx)	1	0	spec, rest, create-objectstack	3	1 / 1 / 1 / 0 / 0 / 0

(d): no warning, exit 0, shard-packages.txt is 0 bytes on every shard, so each shard takes the existing No packages on this shard — nothing to test exit. Every merge_group run printed zero ::warning:: lines. The (d) reading is the docs-only group the card measured at 15–20 minutes of shard time; it now costs six ~1-minute shards.

Seat addendum — does turbo ls --affected read the $TURBO_ROOT$ inputs?

No. Measured on turbo 2.10.10: --affected reads package directories plus the dependency graph only; the $TURBO_ROOT$ inputs on the heavy test tasks feed the task hash and never widen the affected verdict. What pulls those packages into the queue run is the cross-package union (check-cross-package-test-inputs.mjs --union-into), by declared glob.

change	turbo.json $TURBO_ROOT$ inputs that match it	raw turbo ls --affected	pulled in by the union (declared glob)
leaf .ts (b2)	spec#test, core#test, types#test, runtime#test, objectql#test all list packages/**/*.ts	knowledge-ragflow only (1)	spec, core, types, objectql, runtime (packages/**/*.ts)
leaf .md (b3)	spec#test (packages/**/*.md)	knowledge-ragflow only (1)	spec (packages/**/*.md)
docs under content/** (d3)	spec#test, rest#test, create-objectstack#test list content/**	none (0)	spec, rest, create-objectstack (content/**)
.claude/agents/*.md / root README.md (d, d2)	none	none (0)	none

So the floor for a leaf .ts merge is six packages — the leaf plus the five whose declared radius is every .ts under packages/; spec's suite (~500s) dominates that shard. That radius is a declaration in scripts/cross-package-test-inputs.mjs, not a turbo.json input, and nothing here moves it (no turbo.json input was changed).

The attestation reading (empty roster on every shard)

check-shard-attestation.mjs --emit carries no package field at all (payload: attestation, job, shard, total, run_id, run_attempt, sha, workflow_job, attested_at); the emit step has no if:, and on an empty shard the test step exits 0 before creating a log while the completeness guard exits 0 on No test log. Dry run: six --emit calls over six empty shard-packages.txt files, then --verify --gate 'Test Core' --dir ... --filter-result success --leg test/6:success:

    attested 6 / 6 declared shard(s)
Test Core: satisfied — all 6 declared shard(s) published a positive attestation.   (exit 0)

Control, one credential removed: - test-4-of-6 MISSING / ::error::Test Core: 1 of 6 declared shard(s) of test published no positive attestation (test-4-of-6) (exit 1). Its --self-test: 144 assertions green. Nothing in that script needed to change.

Gates (final head 05cb57b43c, every exit captured before any pipe)

• node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (path-less, derived from the merge base): 58 commands (the three scripts/pm/ families joined at the last commit: check-declaration-mirrors, its self-test, check:ratchet-remedy-authority, all exit 0). 55 ran to exit 0, including check:select-shard-packages (all 21 cases passed (97 checks)), check:pm-dispatch-gates (✓ dispatch-gates self-test: 1534 cases pass., run detached as its header requires), check:shard-attestation, check:cross-package-test-inputs, check:workflow-status-functions, check:required-contexts, check:aggregator-roster, check:ci-filter-parity, check:self-test-wired, check:self-test-workflow-commands, check:step-collectors, check:declared-population-live (224 of 270 famil(ies) declare a path population, and every one of them reaches this tree's 8087 tracked file(s)), check:entry-guard, check:parse-guard, check:scripts-symbol-anchors, check:bash32-floor (29 tracked shell file(s) ... name no bash 4+ construct), check:nul-bytes (OK (scanned 8080 text file(s) ...)).
• 3 NOT MEASURED, each a PREREQUISITE NOT MET exit 3 on an unbuilt closure: check:dts-closure, check:dual-build-cjs-loads, check:type-check-debt. Declared narrowing: this diff touches no package source, tsconfig or build input (workflows, two shell scripts, one root script line, one self-test roster entry), so none of the three properties can move; CI ran them on a built closure (Type Check · debt ledger, TypeScript Type Check: success).
• --ran reconciliation: ✓ dispatch-gates --ran: 58 derived famil(ies) accounted for — 55 run, 3 NOT-MEASURED. (exit 0; the three carry their stated reason).
• Whole-repo pnpm lint through scripts/pm/os-verify-lock.sh (slot issue-16453): VERDICT command-exit 0 · held the lock 138s (2m18s) · waited 0s.
• check-governed-merges --test on the six paths: exit 0, NOT governed (quoted above).
• CI on 05cb57b43c: 33 check runs completed, 31 success, 2 skipped (Check Changeset under skip-changeset; Packed-tarball smoke (opt-in)), 0 failures — Lint & Repo Gates, TypeScript Type Check, Test Core, Dogfood Regression Gate, Build Core, Temporal Conformance (live PG + MySQL) all success.

skip-changeset: nothing here publishes from any package.

Noted, not filed (below the #16351 threshold): two prose readings are now stale for merge_group — the header of scripts/check-cross-package-test-inputs.mjs ("the merge-queue and push builds ... partition the FULL package list") and AGENTS.md multi-agent §7 ("the queue runs the full suite; the PR ran affected-only"); the second is a governed surface and both are outside this card's file surface.

Authored in Claude Code session session_019RfFHiRCSs3JXLK4cwcfox (os-steve).

🤖 Generated with Claude Code

https://claude.ai/code/session_019RfFHiRCSs3JXLK4cwcfox

[claude]

Verdict: ACCEPT (contract tier — Clause-② yes, maintainer-directed)

Implemented-by: os-dev flight V (fable), branch claude/issue-16453-merge-group-affected-set, head 05cb57b43c
Reviewed-by: pm-dispatch skills seat (contract-tier read in a detached compare worktree at 05cb57b43c, merge-base 245c6a23cd)

What was checked

• Six files, +740/−128: the "Compute this shard's package set" step is a thin call exporting four event fields into the new scripts/ci/select-shard-packages.sh (environment in, turbo-ls.json out, exit 0 on every decided outcome, a ::warning:: per fallback); its self-test select-shard-packages.selftest.sh (21 cases / 97 checks: usage, push ×3, pull_request ×8 including ci.yml 的 TURBO_SCM_BASE 吃同一个冻结 base.sha —— turbo --affected 在 merge ref 上把 main 漂移算成本 PR 改动(方向保守:多跑,不会少跑) #6195 and [finding] Nothing distinguishes "zero packages affected" from "the affected-set computation silently produced nothing" — an empty changed-files.txt is never checked #10057, merge_group ×9 including a base fetched by sha into a shallow clone, an unfetchable base, the empty docs-only set, the union failure fallback) is wired into Lint & Repo Gates; dispatch-gates.mjs's specimen roster drops the one entry whose invocation moved into the script (measured before and after, 1534/1534).
• The pull_request branch is spliced in by line range: 106 of 121 old lines byte-identical, the rest plumbing and comments; the old inline block and the new step produce identical stdout, turbo-ls.json and shard-packages.txt on four pull_request shapes in a scratch clone (the control the ruling asked for).
• merge_group: base_sha resolved and fetched when absent, the same --affected + cross-package union, full-list fallback with a warning on any resolution/fetch/union failure; an EMPTY set flows to the existing no-packages exit and six empty-shard attestations verify "satisfied — all 6 declared shard(s)".
• Scenario table on real scratch commits: PR leaf 53 (= control); merge_group leaf 53, true-leaf .ts 6, true-leaf .md 2; spec 75; docs-only .claude/agents 0 on all six shards with zero warnings, root README 0, content/** 3 by the union. The seat's addendum answered: turbo 2.10.10 --affected reads package directories + the graph only; the $TURBO_ROOT$ inputs never widen it, the cross-package union does (spec/core/types on any leaf .ts).
• Gates in the compare worktree: the selection self-test 21/97 green; check-shard-attestation --self-test green; check-governed-merges --test on all six paths NOT governed. Merge-tree against flight X's pushed head clean twice. CI on the head: 31 success / 2 skipped / 0 failure — the pull_request path of the new step ran live.

Rulings

• Open question 1 (the model-free co-author trailer): A — the trailer form on today's landed commits is the same; the session URL is the durable attribution.
• Open question 2 (a turbo failure reds merge_group exactly as it reds pull_request): A — one code path, one failure posture; a toolchain fault should be seen.
• The two stale sentences the dev noted (the cross-package gate's header and AGENTS.md §7 both still say the queue runs the full list) are true today and false the moment this merges: one p3 card, filed by the seat and blocked behind this PR's merge, so the corpus never states a falsehood without a card naming it.

Landing

Not governed, ordinary queue landing: the seat flips the draft to ready and arms auto-merge (SQUASH). The PR's own merge-group run is the live proof: the seat reads its selected set (this PR touches .github/**, scripts/** and package.json — an empty or near-empty selection is the expected reading) and the Test Core verdict before closing the loop. On merge the seat strips pm:dispatched and clears the assignee on #16453, then dispatches #16467 (push runs use the affected set + the hourly full run), which shares this script.

---

Generated by Claude Code