Skip to content

fix(tooling): stop coverage.yml conflating file counts with coverage - #42

Open
koniz-dev wants to merge 1 commit into
mainfrom
fix/coverage-output-clarity
Open

fix(tooling): stop coverage.yml conflating file counts with coverage#42
koniz-dev wants to merge 1 commit into
mainfrom
fix/coverage-output-clarity

Conversation

@koniz-dev

Copy link
Copy Markdown
Owner

Two pieces of confusion in the coverage workflow. One of them cost a bogus issue,
which is why it is worth removing rather than tolerating.

1. File counts printed as if they were coverage

📊 Coverage Metrics:
  Overall: 89.8%
  Domain files: 15

15 is a count of SF: records, sitting directly beneath a real percentage
under a heading saying "Coverage Metrics". I read that block, concluded the
per-layer thresholds were unenforced, and filed #25 on it. They were enforced all
along — by a different step (layer_coverage). #25 is closed as invalid.

Now the counts are labelled as counts and point at the step that reports real
percentages:

📊 Overall line coverage: 89.8%

Source files measured per layer (counts, not coverage):
  domain: 15   data: 12   presentation: 11   core: 70
Per-layer coverage percentages are reported by the
'Calculate coverage by layer (architecture gates)' step above.

2. Prose was being written into $GITHUB_OUTPUT

calculate_layer_coverage.sh mixed labels and key=value on stdout, and the
workflow appended every line via while IFS='=' read -r key value. So
Summary of coverage by layer became a step-output key with an empty value.

The script now has two explicit modes:

./scripts/test/calculate_layer_coverage.sh                  # human summary
./scripts/test/calculate_layer_coverage.sh --github-output  # ONLY key=value

Verified locally against real lcov.info--github-output emits eight
key=value lines and nothing else, and no emitted key contains a space:

domain=100.0  data=96.2  presentation=93.5  core=85.2
domain_display=100.0%  ...

The workflow uses --github-output for $GITHUB_OUTPUT and the human mode for
log readability.

Docs corrected

docs/guides/testing/test-coverage.md described output the script has never
produced: per-file low-coverage listings and a Shared layer. There are four
layers and no per-file output. Since this PR changes that output, its description
had to follow.

Not touched

The Enforce coverage thresholds step. It still gates on the same five numbers
(overall 80, domain 100, data 90, presentation 80, core 80). Criterion 5 asks for
a dispatched run proving it still passes — reported on the issue.

Refs #39

Two pieces of confusion, one of which cost a bogus issue.

1. The metrics step printed per-layer FILE COUNTS under a "📊 Coverage
   Metrics:" heading, directly beneath a real percentage:

     Overall: 89.8%
     Domain files: 15

   `15` is a count of SF: records. I read that block, concluded the per-layer
   thresholds were not enforced, and filed #25 on it. They were enforced all
   along, by a different step. The counts are now clearly labelled as counts
   and point at the step that reports real percentages.

2. calculate_layer_coverage.sh mixed prose and key=value on stdout, and the
   workflow appended every line to $GITHUB_OUTPUT via
   `while IFS='=' read -r key value`. So "Summary of coverage by layer" became
   a step-output key with an empty value.

   The script now has two explicit modes: default prints a human summary,
   --github-output prints ONLY key=value. The workflow uses the latter for
   $GITHUB_OUTPUT and the former for log readability. Verified no emitted key
   contains a space.

Also corrected docs/guides/testing/test-coverage.md, which described output the
script never produced: per-file low-coverage listings and a "Shared" layer.
Neither exists; there are four layers and no per-file output.

The Enforce coverage thresholds step is untouched and still gates on the same
five numbers.

Refs #39
@koniz-dev

Copy link
Copy Markdown
Owner Author

Not merging this yet: GitHub Actions is not scheduling runners.

No Quality gate or Strip smoke run was created for this PR, ~10 minutes after
opening it. That is not the documented docs-only case - this PR touches
.github/workflows/coverage.yml and scripts/test/calculate_layer_coverage.sh,
neither of which is in ci.yml's paths-ignore, and strip-smoke.yml has no
path filter at all.

Evidence that it is upstream, not this branch:

  • A dispatched Coverage Analysis run concluded failure at run level while its
    only job never left queued (no started_at completion, no steps executed).
  • gh run rerun was accepted and the run went back to queued - and stayed there.
  • An E2E Android (Patrol) run has been queued since 15:25Z with nothing else
    occupying runners.
  • The last run that actually executed was at 14:42Z.
  • All eight workflows report state: active, so nothing is disabled.

Merging on "no checks reported" is exactly the trap documented in CLAUDE.md
after it nearly caught me on #18/#19: that message is identical whether checks
are legitimately absent or simply have not been created. Leaving this open.

Locally verified in the meantime: ./scripts/dev/audit_template.sh exits 0, and
--github-output emits eight key=value lines with no key containing a space.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant