Skip to content

dash/scripts: the last two literal NUL bytes become escapes - #316

Merged
nyblnet merged 1 commit into
mainfrom
claude/silly-hofstadter-a4e7c2
Aug 30, 2026
Merged

dash/scripts: the last two literal NUL bytes become escapes#316
nyblnet merged 1 commit into
mainfrom
claude/silly-hofstadter-a4e7c2

Conversation

@nyblnet

@nyblnet nyblnet commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Two tracked sources carried a literal NUL byte inside a sentinel string. grep
answers a query against such a file by saying nothing at all — not "no match",
just silence — because one control byte makes it treat the whole file as binary.

file sentinel
dash/src/filter.ts:147 matchKey's "(Blanks)" key
scripts/test-clipboard.ts:314 the colour-agreement rig's fallback marker

Both are un-collidable prefixes — the same idea as the U+001F separator in the
CRDT elKey — so the intent was right and only the encoding was wrong. Written as
\u0000 they are the byte-identical string, which is already this repo's idiom
for \u001f in kernel/src/sync/crdt.ts and spaces/src/fields.ts.

-  if (isBlank(v)) return '<NUL>blank'
+  if (isBlank(v)) return '\u0000blank'

-  const FB = '<NUL>fallback'
+  const FB = '\u0000fallback'

filter.ts was hidden twice over

Git calls a diff binary when it finds a NUL in the first 8000 bytes, and that
one sat at 6911. So git grep answered Binary file HEAD:dash/src/filter.ts matches — no line, no content — and every diff of the file came back as Binary files differ, which reads as a generated blob nobody should be reading.
test-clipboard.ts's NUL was at 17979, past that window, so only grep lost it
and code review still worked. Both now grep and diff normally.

That is also why the filter.ts hunk shows as Bin 21884 -> 21889 bytes in this
PR's diffstat: the old blob is the binary side. Diffs from here on are text.

Verification

  • grep -c "" gives 535 / 343 instead of silence; grep -n "blank" dash/src/filter.ts finds line 147 along with 20 other hits that were invisible.
  • Zero NUL bytes in either file. A sweep of every tracked file now finds NULs only
    in genuine binaries (fonts, images, media, the gradle wrapper) — these were the
    last two in the repo.
  • Values confirmed byte-identical by evaluating the literals as they stand in the
    source
    : both begin at codepoint 0.
  • tsc -b clean in dash/; test-dash-filter 86/86; test-clipboard 73/73
    (bundled with esbuild exactly as CI does — it is not standalone, it imports six
    slides/src modules).

No behaviour change and no test moved.

Notes

Scoped to its own PR per docs/PARALLEL-WORK.md: dash/ is an app zone and
scripts/ is ops.

This class of bug is already documented in docs/DECISIONS.md — DECISIONS.md
itself carried one until a61b211, and the entry that repair points at says to
write these as an escape. This applies that to the two remaining instances.

Not done here, as it would touch a third zone: a CI gate rejecting a NUL byte in
any tracked text file would stop this recurring, rather than fixing instances
three and four.

Two tracked sources carried a literal NUL inside a sentinel string, and grep
answers a query against such a file by saying nothing at all - not "no match",
just silence, because one control byte makes it treat the whole file as binary:

  dash/src/filter.ts:147         matchKey's "(Blanks)" sentinel
  scripts/test-clipboard.ts:314  the colour-agreement rig's fallback marker

Both are un-collidable prefixes, the same idea as the U+001F separator in the
CRDT elKey, so the intent was right and only the encoding was wrong. Written as
\u0000 they are the byte-identical string - which is already the repo's idiom
for \u001f in crdt.ts and fields.ts. Nothing moves: dash tsc -b is clean,
test-dash-filter 86/86, test-clipboard 73/73.

filter.ts was hidden twice over. Git calls a diff binary when it finds a NUL in
the first 8000 bytes, and that one sat at 6911 - so `git grep` answered "Binary
file ... matches" with no line and no content, and every diff of the file came
back as "Binary files differ", which reads as a generated blob nobody should be
reading. test-clipboard.ts's NUL was at 17979, past that window, so only grep
lost it and review still worked.

These were the last two. DECISIONS.md carried one until a61b211, and the entry
that repair points at already says to write these as an escape - this applies
that to the two remaining instances. A sweep of every tracked file now finds
NULs only in genuine binaries: fonts, images, media, the gradle wrapper.
@nyblnet
nyblnet merged commit 7422f4f into main Aug 30, 2026
1 check passed
nyblnet added a commit that referenced this pull request Aug 30, 2026
Two things in one push, because splitting them turns `main` red in between.

FILTER.TS, AGAINST #316. Git reported `UU` with ZERO conflict markers — a
BINARY conflict, because the file carried a literal NUL byte and git had been
treating it as binary, which is the very defect #316 landed to fix. So there was
nothing to read: the sides had to be compared by hand.

#316's whole change to this file is ONE LINE — a literal NUL inside the blank
sentinel replaced by its escape. Verified by forcing the diff through `cat -v`,
and verified as the ONLY commit touching filter.ts on main since this branch
diverged. This branch had already made the identical change: the filter work hit
the same binary-diff problem from the other side and fixed it the same way. Line
147 is byte-identical on both sides and neither carries a literal NUL any more.
Ours is a strict superset — the same fix plus 73 lines of filter work — so it is
resolved to ours on that evidence rather than on "ours usually wins", and the
file is UTF-8 text again instead of binary.

NOT_RUN IS EMPTY. #399 (48945fb) registered all three exempted rigs, so all
three reasons expired at once and the rig failed by design: 288/291, exactly
those three, with failure text naming the fix. Deleted together with the comment
blocks that justified them, because a reason for an entry that no longer exists
is the same rot the list warns about. 288/288 now.

Verified after: tsc clean, 64/64 dash rigs, build:single builds, shell-gate
passes, and test-doc-index.mjs — which this branch briefly and wrongly repaired
inside ops' zone — passes on main's version of it. test-spaces.mjs needs
slides/node_modules, which CI has and this checkout does not.

I did not rebase. This branch is a PR head with 100+ commits and merge commits;
replaying it would rewrite published history and force-push under an open PR to
avoid one three-line resolution. Merged instead. Say if the integrator wants a
true rebase and I will do it deliberately rather than as a side effect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant