Skip to content

test(ci): cover the missing-upload-step branch with a synthetic job set (#303) - #333

Merged
phmatray merged 8 commits into
devfrom
feat/303-the-ci-guard-suite-can-only-run-against-
Aug 13, 2026
Merged

test(ci): cover the missing-upload-step branch with a synthetic job set (#303)#333
phmatray merged 8 commits into
devfrom
feat/303-the-ci-guard-suite-can-only-run-against-

Conversation

@phmatray

@phmatray phmatray commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Implements #303.

Closes #303.

Gives TestReportingTests a seam so its own assertions can be exercised against synthetic jobs. Test
infrastructure only — one file. No library code, no workflow files, no build/Build.cs change, no new
package.

Plan

  • Task 1: Introduce the job-set seam and cover the absence path

What changed

TestReportingTests discovers its guard set once, statically, from the repository's real workflows.
Every real test-running job has the Publish: test-results upload step — that is what the suite
exists to keep true — so the five assertions' "the step is missing" branch could never execute in a
green run. #267 verified it by hand (delete the step from ci.yml, watch the tests name the job,
revert), which proved the behaviour once and could not prove it again.

  • UploadOffenders(jobs, claim) takes the job set instead of reading JobsThatRunTests() itself.
    That parameter is the whole point: a test that cannot supply its own jobs cannot reach the branch.
  • Each content claim is now defined onceRunsOnFailure,
    PointsAtTheDirectoryTheBuildFills, AvoidsThePathTheBuildNoLongerFills, ToleratesAMissingPath
    referenced both by the [Fact] that holds it against the real workflows and by the coverage
    theories. A dictionary of re-typed copies would let the two drift apart silently.
  • UploadMissing(jobs) names the presence check, so the test whose subject is "the step is there"
    no longer reads as jobs that fail an always-true claim.
  • Three coverage tests: every claim reports a job with no step; every claim reports a step that is
    present and violates it; and a conforming step offends nothing.

Code review

A review raised 11 findings; all 11 addressed. One was decisive and two more were real gaps in the
coverage I had written — the first draft of this PR was half-built:

Finding Why it mattered
No fixture had the step present but violating a claim A UploadStepFails that ignored claim entirely would have passed every test, silently reducing all four content assertions to "the step exists" — the exact vacuity #303 was filed about, one branch over
Both fixtures were passed as single-element lists An implementation returning the whole list whenever any job offended would have passed — and in production (3 jobs) that destroys the "names the offending job" property this file is built around
UploadClaims duplicated the predicates rather than being what the [Fact]s read, while its doc claimed the opposite guarantee 165 lines of separation between the copies; drift would have been invisible in both directions
The conforming fixture omitted name: test-results and pinned @v4 That name:/path: pair is the one discrimination the path claim exists to make, so the fixture could not exercise it. Now copied from ci.yml verbatim
A foreach over claims in one [Fact] aborts at the first failure Now [Theory], so each claim reports independently — the form the sibling WorkflowSourceTests already uses

The rest were accuracy fixes, again mostly in my prose: a doc promising a guarantee nothing enforced,
"drives every claim over a synthetic job" (the delegate is never invoked on that path — ||
short-circuits), and "cannot fire in a normal run" said of a branch that fires exactly when the guard
does its job.

Verification

The coverage was mutation-tested, not assumed. Three mutations, each of which a reviewer identified
as previously shipping green:

Mutation Before After
UploadStepFails ignores claim entirely passed 4 tests fail
Absence branch inverted (is { } step &&) caught 4 tests fail
UploadOffenders returns the whole list if any job offends passed 8 tests fail

All restored afterwards; the tree is clean.

@phmatray
phmatray marked this pull request as ready for review August 13, 2026 13:16
@phmatray

Copy link
Copy Markdown
Owner Author

Merge blocked — not by this PR. dev is red, and merging it in makes this PR's CI red too.

  • This PR's own CI was green at 62225e4 (build-and-test, ubuntu-latest, lint-pr-title all passed).
  • Syncing dev in (87d84d2) brought in the failure: CollectionItemShapeGuardTests.No_Suite_Should_Re_Declare_A_Collection_Item_Shape_The_Fixture_Provides, in FormCraft.ForMudBlazor.UnitTests — a project this PR does not touch.
  • Reproduced on a pristine checkout of origin/dev with none of this branch's commits: Failed: 1, Passed: 582. dev has been red on every commit since 0ad0d49.

Cause and remedies are in #345 — a collision between #306 (added the guard) and #308 (added the model it rejects), each green on its own branch.

Holding this PR rather than merging over a red bar. Once dev is green, re-run /merge-pr #333: the branch is already synced, reviewed and verified, so it should land without further work.

@phmatray

Copy link
Copy Markdown
Owner Author

Unblocked. dev is green again — #346 (a68cf29) fixed the shape-guard collision that #345 tracked, and #345 is closed.

Re-synced this branch onto dev (8d23d45) and re-verified, including a gate that did not exist when this PR was written: #307 landed ci(build): enforce dotnet format, so the build now runs dotnet format --verify-no-changes over the whole solution.

  • ./build.sh Test1625 passed, 1 skipped, 0 failed (884 / 584 / 158)
  • ./build.sh Format (the new gate) — exit 0, no violations in this branch

Ready to land.

@phmatray
phmatray merged commit 01e1d65 into dev Aug 13, 2026
2 checks passed
@phmatray
phmatray deleted the feat/303-the-ci-guard-suite-can-only-run-against- branch August 13, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The Ci guard suite can only run against the real workflows, so its absence paths are untested

1 participant