Skip to content

check-self-test-workflow-commands walks only the root scripts/, so the one package-local gate CI self-tests is in no population and its output is never scanned #15414

Description

@baozhoutao

Found while measuring #15342, out of that card's scope (which is the two anchors in scripts/pm/dispatch-gates.mjs and scripts/check-self-test-wired.mjs). Filed rather than fixed: closing it changes which self-tests that gate SPAWNS, which is new verification surface and a different card.

The gap

scripts/check-self-test-workflow-commands.mjs imports the shared extraction from check-self-test-wired.mjs (collectInvocations, carriesSelfTest, codeOf) — "one definition, two gates", as its header puts it — but it does not import the population. It builds its own:

const files = walkScripts(scriptsDir);   // scriptsDir = join(ROOT, 'scripts')
...
const population = [...carriers].filter((s) => named.has(s)).sort();

walkScripts is anchored at the repo-root scripts/ directory, so a gate CI runs by a package-local path can never enter carriers and therefore never enters population, however CI names it.

Measured on 2c22f5716

lint.yml runs one such gate, twice:

- name: Relationship carriers are spelled as the string the spec declares
  run: |
    node packages/lint/scripts/check-reference-carrier-shape.mjs --self-test
    node packages/lint/scripts/check-reference-carrier-shape.mjs

Read through the shared extraction:

named by CI: true | self-tested by CI: true | carries --self-test in code: true
in the workflow-commands population (root walk only): false

The two gates' own scope lines disagree by exactly one, which is this file:

check-self-test-wired:            169 of those are run by 30 workflow(s)
check-self-test-workflow-commands: 168 script(s) CI runs ship a `--self-test`

Why it matters

That gate's whole subject is what a self-test PRINTS on a runner: a line the Actions runner parses as a workflow command mints a failure-level annotation on a green required check, and its header records that the transformation happens on a runner and nowhere else. check-reference-carrier-shape.mjs --self-test is run by CI on every pull request, and its output is in no sweep. The blind spot is one script wide today and grows with the package-local gate lane.

Note the asymmetry that keeps this quiet: the gate's #4690 refusals all fire on an EMPTY population or an empty candidate set. A population that is complete-minus-one refuses nothing and prints a confident scope line.

Direction, not a prescription

check-self-test-wired now admits a second population source on the root walk's own terms — the file must exist and its code must carry the literal — and deliberately does NOT add a second walk. The obvious repair is to export that population from the one gate that owns it rather than to widen a second walk here, which is what the "one definition, two gates" header already promises for the extraction half. Whoever takes it should also decide whether isCandidate then selects the file, and price the extra subprocess.

Related: #15342 (the same root-anchoring class, in the two tools that card names).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions