test(#1356): assert the rendered label, not where identifiers sit in map.js - #1933
test(#1356): assert the rendered label, not where identifiers sit in map.js#1933TeTeHacko wants to merge 1 commit into
Conversation
…ers sit in map.js
V3.e, V3.f and V3.g all describe what `makeRepeaterLabelIcon` produces, but all
three asserted it by grepping `public/map.js`. V3.e went further and bounded the
distance between two identifiers to 200 characters. Three problems with that:
1. It fails on edits that change nothing. Any statement inserted in the
neighbourhood pushes the two identifiers apart and the build goes red while
the markup is byte-identical.
2. It cannot tell code from prose about code. A comment mentioning both
identifiers satisfies the grep, so the assertion reports green no matter what
the function does. Reproduced: with such a comment present, moving the hash
assignment away from the glyph still passed.
3. It does not assert the thing it is named after. On master the match is not
the declaration order at all -- it is `MB_GLYPHS[...]` reaching the later
`shortHash` inside `ariaStatus`, 212 characters downstream. The ordering the
assertion claims to check is incidental to whether it passes.
This loads map.js in the same DOM-less vm sandbox test-map-clustering.js already
uses, exposes `makeRepeaterLabelIcon` through the existing
`window.__meshcoreMapInternals` hook, and asserts the emitted markup:
* glyph, U+2009 thin space, hash, in that order and adjacent;
* no glyph and no thin space when there is no multi-byte status;
* aria-label exactly "multi-byte <status>, hash <ID>", and "repeater hash
<ID>" without one;
* the visible span carries aria-hidden.
Still no browser, so it stays in the JS-unit-tests CI step.
Mutation-tested rather than eyeballed. Old assertions vs new, same mutations:
mutation old V3.e/f/g new
glyph moved after the hash all silent caught
plain space instead of U+2009 all silent caught
span loses aria-hidden V3.g caught caught
aria-label loses its comma all silent caught
200 chars inserted between the
two identifiers (no behaviour
change) V3.e FAILS passes
The sandbox loader deliberately has no try/catch that warns and continues: if
map.js stops loading, the suite must fail rather than quietly skip every
assertion below it.
Full JS unit list from .github/workflows/deploy.yml: 65/65.
|
Reviewing this properly shortly; flagging the CI result first so you are not left guessing. It failed on something unrelated to what this PR changes: That is a customizer persistence check, not one of the three a11y assertions you are replacing. Two things worth knowing before you go looking:
Worth one rerun before treating it as real. If it fails the same way twice, it is worth a look and I will help chase it. On the PR itself: replacing a source grep with a check on what the function renders is the right direction, and the mutation table showing what each version catches is exactly the evidence I would want. Full review to follow. |
|
Recycling both of your PRs, and it is worth saying why, because it is a trap that has cost this queue a lot of time today. Both of your runs are That fully explains #1912. Its re-run at 11:44 still tests a base from before #1932, which is the fix for the master breakage I caused this morning by merging #1929 and #1930 together. So it keeps failing on For #1933 it is less clear-cut. Its run started at 09:31, which is the same minute #1932 merged, so its base may or may not have included the fix. The failure there was I said I would look properly if it failed twice rather than send you chasing it, so: I read your Closing and reopening both, which forces a new merge ref against current master. Nothing needed from you. |
Second instance of the same race, found by re-running the master pipeline for 859173f. That re-run failed on: ✗ setting persists across reload: multibyte toggle should restore checked=true from localStorage with none of the PR code that first surfaced it (Kpa-clawbot#1933) present, which settles that it is a master-level flake and not that PR's doing. The step waited for the element to exist and then read .checked on the next tick: await persistPage.waitForFunction(() => !!document.getElementById('liveMultibyteToggle')); const checked = await persistPage.evaluate(() => ...checked); The element existing does not mean the code that reads localStorage and applies it has run. Waiting on existence is a proxy for the thing under test — the same mistake Kpa-clawbot#1317 documented in test-channel-color-picker-e2e.js. Now waits for the toggle to exist AND be checked, with the same fall-through-to-the-assertion shape as the swatch fix in this PR, and the read reports "(toggle absent)" rather than throwing if the element is genuinely missing. No product code changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wzwr3eXseyNM7Xj598djjE
|
Settled, and in your favour: your PR was not the cause. I re-ran the master pipeline for The same assertion that failed on your PR, on plain master. That is as clean a control as we are going to get. I said earlier that I had read your The cause is in the test: it waits for the toggle element to exist and then reads Your #1933 stands on its own merits and I still like it: replacing a source grep with a check on what the function renders is the right direction, and it is what would have spared us both the 312-versus-277-character detour this morning. |
Follow-up to the review on #1912, where this assertion cost a round trip. Independent of that PR — this branch is off current
masterand touches no code path it changes.The problem
#1356 V3.e,V3.fandV3.gall describe whatmakeRepeaterLabelIconproduces, but all three assert it by greppingpublic/map.js. V3.e also bounds the distance between two identifiers:Three separate failure modes, all observed:
1. It fails on edits that change nothing. #1912 inserts one variable declaration in that function; the markup is byte-identical and the build went red.
2. It cannot tell code from prose about code. My first attempt at fixing #1912 added a comment explaining the constraint — and the comment mentioned both identifiers, so it satisfied the grep by itself. With that comment present I moved the hash assignment away from the glyph, reintroducing the exact defect, and the test still reported green. A check that a comment can satisfy is worse than one that is merely brittle.
3. It does not assert the thing it is named after. On
masterthe match is not the declaration order at all. It isMB_GLYPHS[...]reaching the latershortHashinsideariaStatus, 212 characters downstream. Whether the glyph is actually prepended to the hash is incidental to whether this passes.That third point also corrects something I said on #1912, and it corrects it against myself: both the 312 you quoted and the 299 I "corrected" it to are the distance between the two declarations, which is not the distance the regex uses. Measuring the one it does use:
MB_GLYPHS[→ nextshortHashmasterunknownWidthbelow the glyphshortHashbelow the glyphSo moving
unknownWidthdown does not merely fall short — it makes the gap worse, because it lands between the glyph andariaStatus. My earlier "233, still 33 over" was the wrong metric on the wrong pair. Apologies; the conclusion happened to hold but the reasoning did not.What this does
Loads
map.jsin the same DOM-lessvmsandboxtest-map-clustering.jsalready uses, exposesmakeRepeaterLabelIconthrough the existingwindow.__meshcoreMapInternalshook, and asserts the emitted markup:U+2009thin space, hash — in that order and adjacent;aria-labelexactlymulti-byte <status>, hash <ID>, andrepeater hash <ID>without one;aria-hidden.No browser, so it stays in the JS-unit-tests step rather than moving to Playwright.
Mutation-tested, not eyeballed
U+2009aria-hiddenaria-labelloses its commaFull JS unit list from
.github/workflows/deploy.yml: 65/65.Notes for review
try/catchthat warns and continues. Ifmap.jsstops loading, the suite must fail rather than silently skip every assertion below it.