Skip to content

Test gaps: the bash -n safety guard and the --config override have no tests #11

Description

@catinspace-au

Ranked test gaps from an adversarial review of the new integration suite (during the Rust-migration wrap-up). The suite is 38 integration + 65 unit tests and green; these are the behaviours it does NOT pin, ordered by consequence.

  1. The bash -n skip-the-write guard has zero tests at any level. src/cli.rs warns and preserves the original when a fix would produce invalid syntax, and ARCHITECTURE.md calls that the reason -w is safe to ship. Reachable in one command with a --config pack whose fix_template is if then fi (((. The complementary fail-open branch (bash absent -> Ok(())) is also untested.
  2. A --config rule reusing a built-in id replaces it, and the new severity flips the exit code. This is the headline --config behaviour in docs/rules.md. Verified working, entirely untested.
  3. Here-doc bodies are not rewritten by -w. tests/fixtures/stress-test-fixer.sh exists partly for this and nothing checks it.
  4. -o <existing dir> with a single input joins the file name, and -o <the input's own directory> is refused. Both were added in response to real bugs.
  5. A non-UTF-8 / unreadable / directory argument -- see -o drops the executable bit; one unreadable file aborts the whole run #9; at minimum a test pinning today's behaviour so the choice is deliberate.
  6. --config pointing at a directory, and --dry-run passed without -w/-o (currently ignored silently).
  7. Byte vs character column on a non-ASCII line. src/scanner.rs reports m.start() + 1, a byte offset: on X="eee" ; echo -e hi with accented characters the match is at character 11 but the JSON reports column 14. The JSON schema is a declared public contract, so whichever is intended should be pinned.

Two loose assertions worth tightening

  • tests/cli_fix.rs asserts finding_count(wide) <= finding_count(narrow) for the severity filter in fix mode. That passes unchanged if --severity were ignored entirely; the real relation is strict (measured 6 vs 4 on the same fixture).
  • insta::assert_snapshot! silently rewrites and passes under INSTA_FORCE_UPDATE=1. insta disables this when CI=true, so it is a local-only hazard, but it is the suite's one environment dependency.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions