Skip to content

feat(debug): make --debug emit the probe-level diagnostics SPEC promises - #49

Open
sroberts wants to merge 1 commit into
mainfrom
feat/debug-diagnostics
Open

feat(debug): make --debug emit the probe-level diagnostics SPEC promises#49
sroberts wants to merge 1 commit into
mainfrom
feat/debug-diagnostics

Conversation

@sroberts

Copy link
Copy Markdown
Owner

The gap

SPEC §8.2.7 calls --debug "the load-bearing affordance for trust": a user who can see which paths were probed and which patterns ran can file a precise issue against a signal instead of reshaping their repo to guess at a matcher.

DiagEntry existed. Result.Diag existed and was serialized. Zero signals populated either. --debug printed one summary line per signal and nothing else.

That gap has a cost with a name. The report behind #42 was a Go repo scoring 0.67 on both L3 gates while implementing both, with no way to see which probe failed — and a fix hint telling them to add a lint step to a workflow that had three. One --debug run would have answered it. Now it does:

$ plumbline assess ../decant --debug
[debug] l3.build-lint-gate: trigger `push` or `pull_request` .github/workflows/ci.yml hit=true
[debug] l3.build-lint-gate: linter command or lint action .github/workflows/ci.yml    hit=true
[debug] l3.build-lint-gate: build command .github/workflows/ci.yml                    hit=true
[debug] l3.coverage-gate: stat codecov.yml                                            hit=false
[debug] l3.coverage-gate: trigger `pull_request` .github/workflows/ci.yml             hit=true
[debug] l3.coverage-gate: threshold flag in a coverage step .github/workflows/ci.yml  hit=true (step: Coverage gate)

Design

acmm.Diagnostics. Detectors call NewDiagnostics(ctx) and wrap conditions in d.Probe(path, action, hit, detail...), which returns hit unchanged so it reads inline:

if d.Probe(w.Path, "trigger `pull_request`", w.HasPullRequestTrigger()) { ... }

Recording is a no-op unless --debug marked the context, so the normal scan path allocates nothing. The perf budgets in internal/signals are unmoved.

Scope, stated rather than implied

Instrumented: l3.build-lint-gate, l3.coverage-gate — the two signals this thread was about. Everything else emits only its summary line. SPEC now says exactly that, rather than describing a universal trace that doesn't exist. An uninstrumented signal is visibly uninstrumented rather than silently so, and adding probes is a local change that can't alter a verdict.

Two contract bugs found while wiring it

The repo's own integration test caught the first one. Diag leaked into assess's stdout JSON, violating the flag's documented promise that --debug "does not change stdout — gate output stays stable." A CI gate adding --debug to a failing run would have seen its JSON move. Probes now go to stderr only for assess.

Second: inspect --debug was _ = debug — a flag that parsed, did nothing, and advertised "Include detection diagnostics in output." It works now, printing on stdout, where there's no gate output to keep stable:

Probes:
  · stat codecov.yml
  · stat .codecov.yml
  ✓ trigger `pull_request` .github/workflows/ci.yml

Tests

  • pkg/acmm: disabled-by-default, records-when-enabled, nil-safe.
  • internal/signals/l3: verdicts byte-identical with and without diagnostics across five fixtures, and probes recorded only under --debug. Probe wraps the conditions detectors branch on, so a mistake there would change behaviour under --debug only — the worst kind to find later.

Full suite, go vet, gofmt, perf budgets clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XtNwLDzRYmPrXcYeQypTf9

SPEC.md §8.2.7 calls --debug "the load-bearing affordance for trust": a
user who can see which paths were probed and which patterns ran can file a
precise issue against a signal instead of reshaping their repo to guess at
a matcher. The DiagEntry type existed, Result.Diag existed and was
serialized — and zero signals populated either. --debug printed one summary
line per signal and nothing else.

That gap has a cost with a name. The report that started #42 was a Go repo
scoring 0.67 on both L3 gates while implementing both, with no way to see
which probe failed; the fix hint told them to add a lint step to a workflow
that had three. One --debug run would have answered it.

Adds acmm.Diagnostics: detectors call NewDiagnostics(ctx) and wrap
conditions in d.Probe(path, action, hit, detail...), which returns hit
unchanged so it reads inline. Recording is a no-op unless --debug marked
the context, so the normal scan path allocates nothing — the perf budgets
in internal/signals are unmoved.

Instruments l3.build-lint-gate and l3.coverage-gate, the two signals that
thread was about. Other signals emit only their summary line, and SPEC now
says which are instrumented rather than implying all are.

Two contract details found while wiring it:

- The repo's own integration test caught Diag leaking into assess's stdout
  JSON, which violates the flag's documented promise that --debug "does not
  change stdout — gate output stays stable". Probes now go to stderr only
  for assess.
- `inspect --debug` was `_ = debug`, a flag that parsed and did nothing
  while advertising "Include detection diagnostics in output". It now does,
  on stdout, where there is no gate output to keep stable.

A test asserts verdicts are byte-identical with and without diagnostics
across five fixtures. Probe wraps the conditions detectors branch on, so a
mistake there would change behaviour under --debug only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtNwLDzRYmPrXcYeQypTf9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant