Skip to content

Assert what the tag reader does where a page ends and at the edges [#44] - #205

Merged
iderex merged 1 commit into
mainfrom
tests/what-the-tag-reader-does-at-the-end-of-a-page
Aug 26, 2026
Merged

Assert what the tag reader does where a page ends and at the edges [#44]#205
iderex merged 1 commit into
mainfrom
tests/what-the-tag-reader-does-at-the-end-of-a-page

Conversation

@iderex

@iderex iderex commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Under #44, over the third scoped area. It does not close it: what that issue
waits on is where a mutation run lives, and this change adds no leg to the gate.

What was wrong

The strict read of a produced page walks the bytes itself rather than handing
them to a parser. Nine places in that walk test an index and then read the byte
after it, and no case in the suite ended a document at any of them. So a
mutation turning i < len(rest) into i <= len(rest), or i >= len(rest) into
i > len(rest), read past the end of the input and every test still passed.

What the bound guards is not a wrong verdict. It is the checker panicking on the
page it exists to judge, which takes the gate down rather than naming one broken
tag, and the input that reaches it is a template that stopped writing.

Three more sites had the same shape without an index behind it. An empty raw
text element puts its end tag at offset zero, so a bound of end < 0 moving to
end <= 0 refuses <style></style>. A page whose first heading is not the top
level is not a skip, so lastHeading > 0 moving to >= 0 refuses it. And the
deepest heading level stops being a heading at all if level > 6 moves to
>= 6, which quietly removes the level a jump to it is measured against.

How it was found

By re-taking the mutation run #44 records over the one scoped area that had no
score in it, in a clone outside the working tree because the tool rewrites
source files where it finds them.

gremlins --version
gremlins version dev windows/amd64
gremlins unleash --timeout-coefficient=20 --workers=2 ./internal/markup
Mutation testing completed in 6 minutes 46 seconds
Killed: 60, Lived: 30, Not covered: 3
Timed out: 11, Not viable: 0, Skipped: 0
Test efficacy: 66.67%
Mutator coverage: 96.77%

Run 2026-08-26 at ad8b25c. Every figure reproduces the reading of that package
recorded on #44 in August, so the escaping path had not moved: the lowest
efficacy of the three scoped areas and the only one with survivors in double
figures.

What the change does

Three cases, and no production code.

A table of fragments that end inside a tag, each of which has to come back as
one structure problem rather than as a crash or a silence. Those are raw
fragments rather than wrapped ones, deliberately: the wrapper this file uses
writes a closing body after the byte each case is about, so wrapping would put
the input past the end of the thing being tested.

A table of well-formed shapes, which is the half that matters more. A reader
that refused one of these would be repaired by somebody loosening the rule.

A heading table holding the three shapes the rule must leave alone beside the
jump it exists for, so the pair is one case rather than two files apart.

The proofs

Each mutation applied by hand at the commit being pushed, with the tree restored
between. The first is the family the fragments are for, and it is the reason
they are not decoration:

for i < len(rest) ...  ->  for i <= len(rest) ...
--- FAIL: TestAPageThatEndsInTheMiddleOfATagIsRefusedRatherThanReadPastItsEnd
panic: runtime error: index out of range [4] with length 4

end < 0  ->  end <= 0    (the raw text element)
--- FAIL: TestTheEdgesOfAWellFormedTagAreReadRatherThanRefused
    markup_test.go:230: a raw text element with nothing inside: "    <style></style>" was refused: [line 8: <style> is opened and never closed]

level > 6  ->  level >= 6
--- FAIL: TestOnlyASkippedHeadingLevelIsRefused
    markup_test.go:251: a jump from the top level to the deepest was read as 0 problem(s): []

lastHeading > 0  ->  lastHeading >= 0
--- FAIL: TestOnlyASkippedHeadingLevelIsRefused
    markup_test.go:247: a first heading that is not the top level: "    <h2>A section</h2>" was refused: [line 8: <h2> follows <h0>, so a level is skipped and a reader moving by heading cannot tell what is under what]

i = at + end + len("-->")  ->  i = at + end - len("-->")
--- FAIL: TestTheEdgesOfAWellFormedTagAreReadRatherThanRefused
    markup_test.go:230: a comment ending on a stray bracket: "    <!-- a rule with a < --><p>A paragraph.</p>" was refused: [line 8: a tag is opened and never closed]

The last one is the near miss worth reading. The step past a comment is six
bytes wrong under that mutation, which changes nothing unless a bracket sits in
those six bytes, so it takes a comment whose last character before the closer is
one. This package's own design note says that a bracket inside a stylesheet is a
character and not a tag; inside a comment it is the same character, and that is
the case the arithmetic had nothing standing on it.

The score after it

gremlins unleash --timeout-coefficient=20 --workers=2 ./internal/markup
Mutation testing completed in 6 minutes 26 seconds
Killed: 77, Lived: 14, Not covered: 2
Timed out: 11, Not viable: 0, Skipped: 0
Test efficacy: 84.62%
Mutator coverage: 97.85%

Run 2026-08-26 in the same clone at the same commit with the cases in. Sixteen
survivors dead, one mutant out of Not covered.

What is left alive, said rather than left to be found

Fourteen. Eight of them are bounds no input can distinguish, because a guard
above them already refuses everything the mutation would change the verdict for,
or the fall-through reaches the same return: 119:8, 130:11, 138:11,
261:9, 276:21, 281:8, 300:8, 351:8. That is a reading of those lines
rather than a measurement.

Five are arithmetic or an increment whose mutation moves an index by two bytes
in a direction that only matters if a bracket sits in exactly those two bytes:
142:17, 264:62, 294:16, and the two decrementing space skips at 283:5
and 316:5. Reaching them means writing a page no template writes, and a
fixture built to trip one of those proves less than the fragments above.

One is 282:9 negated, which produces an attribute with an empty name and
changes no verdict this package reads, because nothing asks about an attribute
called nothing.

Eleven mutants time out at this coefficient rather than getting a verdict, which
is the same eleven the August reading recorded and is not a pass. One of them,
134:11, was killed on one of the three runs taken tonight and timed out on the
other two, so a timeout here is a run that did not finish rather than a property
of the mutant.

The gate at the commit being pushed

go run . ci
gate: 7 legs, in order: format, vet, test, build, links, sitemap, invariants
  format: ok, 74 file(s)
  vet: ok
  test: ok, 37 test file(s)
  build: ok, 22 file(s)
  links: every reference that stays inside this site resolves to a file the build wrote
  sitemap: every page the build wrote is listed once, and every entry has a page behind it
  invariants: ok, 39 rule(s) decided, 1 owed and not decided
7 of 7 legs ran. None was skipped.

The run also prints that the needs-network harness was not asked for and what
asking would cost, which is unchanged by this branch.

go run . hygiene origin/main HEAD
hygiene: 1 non-merge commit(s) in origin/main..HEAD, origin internal
  5112832fe1e8: subject carries its reference
1 commit(s) judged, none refused.

git diff --name-only origin/main...HEAD
internal/markup/markup_test.go

One file, one topic, no production code touched.

No second reader

This change carries no second reader. The evidence in place of one is the five
mutations above, each applied at this commit and each producing a red run naming
the case that caught it, and the before and after scores from the same tool in
the same clone at the same commit.

The strict read of a produced page walks the bytes itself. Nine places in
that walk test an index and then read the byte after it, and no case ended
a document at any of them, so a mutation turning one bound into its
neighbour read past the end of the input and every test still passed. What
that guards is the checker crashing on the page it exists to judge, which
takes the whole gate down instead of naming one broken tag.

Three more places had the same shape without the index: an empty raw text
element, whose end tag sits at offset zero; a page whose first heading is
not the top level; and the deepest heading level, which stops being a
heading if the bound moves by one. All three are pages a template writes.

Found by re-taking the mutation run this issue records, over the third
scoped area, which had no score in it. Thirty mutants survived at ad8b25c
and fourteen do now, with efficacy at 84.62% against 66.67%.

The cases are raw fragments rather than wrapped ones where the point is
the end of the input, because the wrapper writes a closing body after the
byte each of those cases is about. The well-formed neighbour of every
refusal is asserted beside it: a reader that refused one of these is
repaired by somebody loosening the rule.

Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com>
@iderex
iderex merged commit 63b4c2b into main Aug 26, 2026
17 checks passed
@iderex
iderex deleted the tests/what-the-tag-reader-does-at-the-end-of-a-page branch August 26, 2026 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant