Skip to content

The loader's rate was a number that was never true - #285

Merged
dhobi merged 1 commit into
masterfrom
dust-loader-rate
Aug 23, 2026
Merged

The loader's rate was a number that was never true#285
dhobi merged 1 commit into
masterfrom
dust-loader-rate

Conversation

@dhobi

@dhobi dhobi commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Dust's fuse measured its transfer rate per completed file, on the arrival that completed it — and the first thing it fetches is a film. recent got one sample carrying the file's whole size, the window it divided by floored at 0.25 s, and 13 MB / 0.25 s is 52 MB/s. The caption showed some tens of MB/s for one tick and then held that figure, unchanged, for the minute the next film spent arriving.

The bar had the same problem from the same cause: it counts fetches, Dust's boot is fourteen of them, so on a slow connection it sat at one mark for minutes.

Why

The store never streamed. HostFiles.load has always promised that onBytes "reports each chunk where the source streams", and taoot/src/files.ts does exactly that. DustFiles buffered the whole body and reported it once, so both the rate and the bar were drawn from an event that fires fourteen times a boot.

What changed

dust/src/files.ts

  • reads the body chunk by chunk, like its Titanic sibling, with the same arrayBuffer() fallback where a response has no body;
  • keeps the manifest's byte sizes — it was throwing the values away and keeping only the keys;
  • partialProgress(): how far the in-flight fetches have got, in whole-file units;
  • a joiner on a shared fetch still gets its single total report, as before.

dust/src/main.ts — one meter for the whole loader, fed by every byte the page pulls (the head phase's own two fetches included), read on a 250 ms ticker over a 3 s window:

  • no number until the window is wide enough to divide by;
  • the rate only while bytes are arriving — when the wire goes quiet the caption falls back to what the loader was saying, because a rate that outlives its transfer is the misleading part;
  • · 8.4 MB so far beside it, a fact that cannot go stale;
  • the bar counts completed fetches plus the fraction of those in flight, and the head phase's two fetches move it as they arrive.

Measured, not argued

Throttled via CDP:

before after
caption at 300 KB/s one wrong figure, then frozen 300 KB/s · 5.4 MB so far, ticking
bar over the first 30 s stuck at 16% 13% → 59%, continuous
full boot at 4 MB/s 100% / "ready", no page errors

tsc --noEmit clean · npm test 534/534.

One thing the honest counter surfaces: a cold Dust boot moves ~95 MB (the intro films plus town.set). That is not new, it was just never visible.

🤖 Generated with Claude Code

Dust's fuse measured its transfer rate per COMPLETED FILE, on the arrival that
completed it — and the first thing it fetches is a film. `recent` got one sample
carrying the file's whole size, the window it divided by floored at 0.25 s, and
13 MB / 0.25 s is 52 MB/s. So the caption read some tens of MB/s for one tick
and then held that figure, unchanged, for the minute the next film spent coming
down the wire. The bar had the same shape of problem from the same cause: it
counts fetches, and Dust's boot is fourteen of them, so on a slow connection it
sat at one mark for minutes at a time.

The store never streamed. `HostFiles.load` has always promised that `onBytes`
"reports each chunk where the source streams", and taoot/src/files.ts does
exactly that; DustFiles buffered the whole body and reported it once at the end,
so a rate and a bar were both being drawn from an event that fires fourteen
times a boot. Now it reads the body chunk by chunk like its sibling does, keeps
the manifest's byte SIZES (it was throwing the values away and keeping only the
keys), and can say how far the fetches in flight have got.

On the page that becomes one meter for the whole loader, fed by every byte it
pulls — the two fetches the head phase makes itself as well as the store's — and
read on a 250 ms ticker over a three-second window:

  - no number at all until the window is wide enough to divide by, instead of
    one manufactured from a quarter-second floor;
  - the rate only while bytes are arriving. When the wire goes quiet the caption
    falls back to what the loader was saying (which room it is opening, what the
    boot just loaded), because a rate that outlives its transfer is the
    misleading part;
  - "· 8.4 MB so far" beside it, which is a fact that cannot go stale;
  - the bar counts completed fetches PLUS the fraction of those in flight, and
    the head phase's two fetches now move it as they arrive rather than jumping
    on completion.

Measured against a throttled connection rather than argued: at a 300 KB/s cap
the caption reads 300 KB/s and the bar climbs 13% → 59% without a stall, where
before it showed one wrong number and stood still. A full boot at 4 MB/s reaches
100% with no page errors — and says out loud that it moved 95 MB to get there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dhobi
dhobi merged commit b03945a into master Aug 23, 2026
2 checks passed
@dhobi
dhobi deleted the dust-loader-rate branch August 23, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant