Decode images without blocking the caller and within a stated memory budget

[iderex]

Decoding is the part of artwork that costs real time, and it is the part that
most easily lands on the thread a client draws with. The concurrency record in #9
promised which calls may block. This is where that promise is kept or broken for
the most expensive operation in the repository.

Memory is the other half. Two hundred decoded tiles held at once is enough to be
killed by the operating system on a television, and that failure looks like a
crash rather than like a cache that was too large.

What this produces

Decoding that never runs on the caller's thread, matching the guarantee in #9.

A bound on how many decodes run at once, with a stated default and its reason,
since unbounded parallel decode on a four-core television is slower than a bounded
one as well as heavier.

A bound on decoded bytes held, separate from the byte cache in #54, because
decoded images and stored bytes have very different costs.

Decoding handed to the platform where the platform has a better decoder, through
an interface, with a core implementation as the fallback.

Done when

A test proves the caller's thread is not used for decode. A test drives 200
images through and proves the decoded-bytes bound holds throughout. Both run
headless. Blocked on #9.

[iderex]

Two of the four things this issue asks for have been answered elsewhere since it was written, and one of the two is answered in the opposite direction.

Handing decoding to the platform, through an interface with a core implementation as the fallback, is refused by docs/decisions/0003-what-the-core-does-not-do.md under the heading Image decoding is inside. Its reason is that turning bytes into pixels is a parse of untrusted input that arrived over a network, in the same class as parsing a response body, and that it is the most attacked surface this repository will carry. It names the exact shape this bullet asks for: eleven clients each reaching for a platform decoder is eleven answers to which formats are accepted, eleven bounds on a declared dimension, and eleven chances that one of them decodes whatever it is handed. docs/decisions/0112-where-the-platform-decoder-begins.md puts image decoding on the core's side of its line for the same reason and the two records name each other. So that bullet is not an implementation detail this issue can settle. Building it would mean superseding 0003, which needs a record and an argument that answers the untrusted-input reason rather than the performance one.

It also runs against docs/decisions/0101-what-the-core-trusts.md, which treats a declared image dimension as untrusted, bounds it before a decode, and expects the refusal in #55 to be on content rather than on the claim. A platform decoder reached through an interface is a decoder the core cannot make do either.

A bound on how many decodes run at once already exists. docs/decisions/0009-the-concurrency-model.md puts image decoding on the processing lane, sizes that lane to the host's reported usable processor count less one with a floor of one, and lets a client set the size and not set it to zero. A decode runs on a lane thread, so the number that can run at once is that size. A second bound underneath it is a different number for the same question, and stating one here and another there is how two answers to one question get written down. If a lower bound is wanted for a reason that is about memory rather than about processors, that reason is what the record has to carry, because the number without it will read as a duplicate of the lane size and be removed by whoever notices.

What is untouched by both records is the rest of this issue, and it is the part worth keeping. Decoding never running on the caller's thread is 0009's guarantee rather than this issue's, and #117 and #76 are the two routes that check it. The bound on decoded bytes held, separate from the byte tier in #54, is decided nowhere and is the number this issue still owes, because decoded images and stored bytes have costs that are not comparable.

Both conditions here are untouched. A test proving the caller's thread is not used for decode, and a test driving 200 images through against the decoded-bytes bound, are statements about running code, and there is no language chosen, no build and no test command in this tree to assert them against.

[iderex]

The number this issue still owed landed as docs/decisions/0050-the-decoded-bytes-budget.md, on 638ed89.

Most of the work was deciding what the quantity is, because the phrase reads as a cache and is not one. 0009 hands a decoded image to the caller and never reads it again, and 0006 caches artwork bytes and their dimensions rather than pixels, so there is no decoded image anywhere in the core that somebody is not waiting for. What the budget bounds is the pixel buffers of decodes that are running plus those whose outcome has not yet reached the caller. Read without that, the natural build is a cache of decoded images, which is a second store with a second eviction policy for data 0006 deliberately does not keep, and the memory it occupies is the memory the budget exists to bound.

Sixty four mebibytes. At three hundred by four hundred and fifty and four bytes a pixel a poster is 540000 bytes, so the budget holds about 124 of them, and a wall of two hundred at 103 mebibytes does not fit. That is the point of the number rather than an accident of it: what is on a screen is a fraction of two hundred, #53 cancels a tile that scrolled off, and a budget large enough for every tile somebody scrolled past holds a library in pixels.

The floor a client may set is derived rather than chosen. It is the per-image bound in 0055, which is sixteen million pixels at four bytes each, because below that an image 0055 admits could never be decoded and the accept set would depend on a memory setting, which is what 0055 exists against. A client wanting a smaller ceiling is asking for 0055's dimension bound to be lowered, and that is a change to 0055 with its own argument about what an attacker can make the core allocate.

At the bound a decode does not start. It waits, in the order it was asked for, and no sixteenth kind in 0004 is asked for because nothing failed. A client that never consumes its completions stalls only its own further decodes, and after five seconds with a decode waiting the core reports how much is held and how many are waiting through 0100.

No second bound on how many decodes run at once, for the reason already on this issue. The lane size in 0009 and this budget interact rather than duplicate: the budget also counts finished decodes waiting to be handed over, which is the larger number on a client that consumes slowly, so on a four-processor television the lane is three and the budget still binds first for large images.

Both conditions here are unchanged, on the ground already recorded above.