You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hand-run mutation testing found a decorative test nearly every time it was tried during 0.18 work. Tests that could not fail for the reason they stated:
Not per-PR: it is slow, and the signal needs a human to triage. A scheduled run (weekly, or pre-release) with the surviving mutants reported is the right cadence.
Acceptance
cargo-mutants runs against the listed modules and the output is readable.
A baseline is recorded — the point is the trend, not a one-off number.
Known-acceptable survivors are documented so triage is about what changed, not re-litigating the same ones.
Caveat worth stating
Mutation score is a means, not a target. A test suite optimised for killing mutants can still be one that nobody can read. The value here is specifically finding assertions that cannot fail, which is a narrower and more useful signal than coverage.
Fourth of the trustworthiness items.
Why
Hand-run mutation testing found a decorative test nearly every time it was tried during 0.18 work. Tests that could not fail for the reason they stated:
EVENT_VARIANT_COUNT— compared two hand-written lists; a new variant appeared in neither (EVENT_VARIANT_COUNT does not guard what its failure message claims #137)unpriced_presets_report_unknown_rather_than_free— survived flipping||to&&file_backend_evicts_oldest_to_stay_under_its_cap— keys sorted in write order, so the tiebreak agreed with mtime and it passed either wayassert_eq!(seen.len(), samples.len())— tautological, in the PR that fixed decorative assertionsclaude_sonnet_5price assertion pinning one field of four, which is how a 1.5x error survived 18 releasescontext::message_text, which returns empty for anything but aToolResult— so one passed having inspected nothingSeven, found by hand, in one release cycle. The pattern is not going to stop on its own.
Proposal
cargo-mutantsover the modules where a silent failure costs the most:src/context.rs— compaction, truncation, execution limitssrc/agent_loop.rs— the loop itselfsrc/llm_compaction.rs— the state machinesrc/provider/model.rs— pricingNot per-PR: it is slow, and the signal needs a human to triage. A scheduled run (weekly, or pre-release) with the surviving mutants reported is the right cadence.
Acceptance
cargo-mutantsruns against the listed modules and the output is readable.Caveat worth stating
Mutation score is a means, not a target. A test suite optimised for killing mutants can still be one that nobody can read. The value here is specifically finding assertions that cannot fail, which is a narrower and more useful signal than coverage.