test(ci): cover the missing-upload-step branch with a synthetic job set (#303) - #333
Conversation
…suite-can-only-run-against-
…suite-can-only-run-against-
…suite-can-only-run-against-
…suite-can-only-run-against-
|
Merge blocked — not by this PR.
Cause and remedies are in #345 — a collision between #306 (added the guard) and #308 (added the model it rejects), each green on its own branch. Holding this PR rather than merging over a red bar. Once |
…suite-can-only-run-against-
|
Unblocked. Re-synced this branch onto
Ready to land. |
Implements #303.
Closes #303.
Gives
TestReportingTestsa seam so its own assertions can be exercised against synthetic jobs. Testinfrastructure only — one file. No library code, no workflow files, no
build/Build.cschange, no newpackage.
Plan
What changed
TestReportingTestsdiscovers its guard set once, statically, from the repository's real workflows.Every real test-running job has the
Publish: test-resultsupload step — that is what the suiteexists to keep true — so the five assertions' "the step is missing" branch could never execute in a
green run. #267 verified it by hand (delete the step from
ci.yml, watch the tests name the job,revert), which proved the behaviour once and could not prove it again.
UploadOffenders(jobs, claim)takes the job set instead of readingJobsThatRunTests()itself.That parameter is the whole point: a test that cannot supply its own jobs cannot reach the branch.
RunsOnFailure,PointsAtTheDirectoryTheBuildFills,AvoidsThePathTheBuildNoLongerFills,ToleratesAMissingPath—referenced both by the
[Fact]that holds it against the real workflows and by the coveragetheories. A dictionary of re-typed copies would let the two drift apart silently.
UploadMissing(jobs)names the presence check, so the test whose subject is "the step is there"no longer reads as jobs that fail an always-true claim.
present and violates it; and a conforming step offends nothing.
Code review
A review raised 11 findings; all 11 addressed. One was decisive and two more were real gaps in the
coverage I had written — the first draft of this PR was half-built:
UploadStepFailsthat ignoredclaimentirely would have passed every test, silently reducing all four content assertions to "the step exists" — the exact vacuity #303 was filed about, one branch overUploadClaimsduplicated the predicates rather than being what the[Fact]s read, while its doc claimed the opposite guaranteeconformingfixture omittedname: test-resultsand pinned@v4name:/path:pair is the one discrimination the path claim exists to make, so the fixture could not exercise it. Now copied fromci.ymlverbatimforeachover claims in one[Fact]aborts at the first failure[Theory], so each claim reports independently — the form the siblingWorkflowSourceTestsalready usesThe rest were accuracy fixes, again mostly in my prose: a doc promising a guarantee nothing enforced,
"drives every claim over a synthetic job" (the delegate is never invoked on that path —
||short-circuits), and "cannot fire in a normal run" said of a branch that fires exactly when the guard
does its job.
Verification
The coverage was mutation-tested, not assumed. Three mutations, each of which a reviewer identified
as previously shipping green:
UploadStepFailsignoresclaimentirelyis { } step &&)UploadOffendersreturns the whole list if any job offendsAll restored afterwards; the tree is clean.
./build.sh Test(the exact CI target) — Restore/Compile/Test all succeeded on the merged tree.dotnet test -c Release— 1601 passed, 1 skipped, 0 failed (FormCraft.UnitTests876;ForMudBlazor567;ForFluentUI158).devmoved three times during the run. The last merge overlapped this file — The .html test report is promised but unenforced, so half the artifact can vanish silently #276 (PR build(ci): guard every promised report kind, not just the trx (#276) #325)also edited
TestReportingTests.cs. Git merged it without conflict, and I checked the result wassemantically clean rather than trusting that: The .html test report is promised but unenforced, so half the artifact can vanish silently #276's additions are confined to the
Build.csreport-kind assertions, touch nothing in the upload region, and all four production call sites still
route through the named delegates.