Skip to content

An offset table you can also look at: every format page gets a block view - #284

Merged
dhobi merged 1 commit into
masterfrom
byte-maps
Aug 23, 2026
Merged

An offset table you can also look at: every format page gets a block view#284
dhobi merged 1 commit into
masterfrom
byte-maps

Conversation

@dhobi

@dhobi dhobi commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Every format page describes bytes with a table. That is the right shape when you are implementing a reader and the wrong one when the question is what is this file mostly made of? — a table gives a 4-byte field and a 4-megabyte payload the same row.

So the same region list now renders two ways behind a Table / Block switch: the offset table as before, or blocks drawn to scale from byte 0 at the top left, one row worth 64 B or 64 KB or 1 MB as the file's own scale asks. Hover a block for what that stretch of bytes is for.

What is here

  • docs/.vitepress/theme/ — a custom theme, for the one component markdown cannot express. withMermaid is unaffected: it patches VitePress's client entry, not the theme.
  • Struct layouts, hand-authored (bytemap/layouts.ts): the 1024-byte file header, a container record, an STG flat record.
  • Whole-file maps, generated by tools/blockmap.ts: it opens a real file with the real reader and walks it — SET (both engines), STG/FLT, MOV (both), TRK/SFX/11K, SND, SHP, PUP, CST, BOOTFILE, both save formats. What the walk misses is sniffed; an audio chunk, a script and a picture each announce themselves. Coverage runs 100% on most (extra.cst 332/333, cuff.shp 62/62, both saves complete) and is honest where it is not.
  • The maps are committed — the game data is not in this repo — and carry offsets and roles, never content. 214 KB total, lazily loaded one chunk per map so a page pays only for the map it shows.

Pointers between containers

The walk records which container named which, so a map carries the file's pointer graph: hovering a table rings every container it addresses. The pointer itself is four bytes inside somebody else's payload — invisible at file scale — so the map draws the relation between the two containers instead, which is the same fact at a size you can hover.

Three things that fell out of building it

  • The containers a SET walk cannot name are not random: equal-sized pairs plus one singleton, one pair per scene and per road, on every set checked — wireless (1 scene) through hallf2c (14 scenes, 13 roads). Documented on the formats index as the open question it is.
  • A BOOTFILE has no pointer graph at all. Its handlers are reached by name through the resolution chain, so nothing in the file points at anything else in it — which is what "standard library" means in bytes.
  • v1Index's banks is a count, not a container index, and the first draft of the save annotator read it as one.

Also

Corrects a stale claim the maps made me re-read: set.md and set-v1-to-v4.ts still said Dust's field of view was "unmeasured" and that the viewer takes max(w, h) / 2 for every set. It was recovered from DF.EXE — 310, written at 0x4331e5 / 0x433418, read by the projection at 0x433c60 — and v1 sets have carried it since; max(w, h) / 2 is the v4 default only.

Colours are the first three categorical slots of the data-viz palette, the ones that clear the all-pairs CVD and normal-vision gates in both light and dark (a byte map is all-pairs by construction: reading one means matching a block against every legend swatch). Audio inside "pictures & sound" gets a hatch rather than a fourth hue.

Checks

tsc --noEmit clean · npm test 534/534 · npm run docs:build clean · looked at every map in light, dark, table view and at 390 px.

🤖 Generated with Claude Code

…view

The format pages describe bytes with a table, which is right when you are
implementing a reader and wrong when the question is "what is this file mostly
made of?" — a table gives a 4-byte field and a 4-megabyte payload the same row.
So the same region list now renders two ways behind a Table/Block switch: the
offset table as before, or blocks drawn to scale from byte 0 at the top left,
one row worth 64 B or 64 KB or 1 MB as the file's own scale asks, hover for what
a stretch of bytes is for.

<ByteMap> is the one component this doc set needs that markdown cannot express,
so docs/.vitepress/theme/ exists now. Struct layouts are hand-authored beside it
(the 1024-byte file header, a container record, a flat record); whole-file maps
are generated by tools/blockmap.ts, which opens a real file with the real reader
and walks it — SET (both engines), STG/FLT, MOV (both), TRK/SFX/11K, SND, SHP,
PUP, CST, BOOTFILE and both save formats. What the walk misses is sniffed: an
audio chunk, a script and a picture each announce themselves. The maps are
committed because the game data is not in this repository, and what they carry
is offsets and roles, never content.

The walk also records WHICH CONTAINER NAMED WHICH, so a map carries the file's
pointer graph and hovering a table rings every container it addresses. The
pointer itself is four bytes inside somebody else's payload — invisible at file
scale — so the map draws the relation between the two containers instead.

Three things fell out of building it:

  - the containers a SET walk cannot name are not random. They are equal-sized
    pairs plus one singleton: one pair per scene AND per road, on every set
    checked, from wireless (1 scene) to hallf2c (14 scenes, 13 roads).
  - a BOOTFILE has no pointer graph at all. Its handlers are reached by name
    through the resolution chain, so nothing in the file points at anything
    else in it — which is what "standard library" means in bytes.
  - v1Index's `banks` is a count, not a container index, and the first draft of
    the save annotator read it as one.

Also corrects a stale claim the maps made me re-read: set.md and
set-v1-to-v4.ts still said Dust's field of view was "unmeasured" and that the
viewer takes max(w, h)/2 for every set. It was recovered from DF.EXE (310, at
0x4331e5 / 0x433418, read by the projection at 0x433c60) and v1 sets have
carried it since; max(w, h)/2 is the v4 default only.

Colours are the data-viz palette's first three categorical slots, which are the
ones that clear the all-pairs CVD and normal-vision gates in both light and
dark — a byte map is all-pairs by construction, because reading one means
matching a block against every legend swatch. Audio inside "pictures & sound"
gets a hatch rather than a fourth hue.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dhobi
dhobi merged commit 77675dc into master Aug 23, 2026
2 checks passed
@dhobi
dhobi deleted the byte-maps branch August 23, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant