Implement partial refreshing protocol

[LordMike]

Introduces partial image sending to OpenDisplay. The code has been written by a mix of Claude Code and OpenAI Codex, at my direction.

The process

Roughly, a full image is done by sending 0x70 to start new imagery, and then following it up with 0 to many 0x71's with chunks of (possibly) compressed data. Once done, a single 0x72 is sent with mode=full, which triggers the refresh.

A partial refresh however, is done by sending both old and new pixels to the display panel. The display controller then picks the appropriate update waveforms based on what the diff is. As our MCU deep sleeps between updates, and also turns the panel effectively off (for SSD16xx variants which do hold the current image in memory), we need to re-send old+new images on each partial update. The protocol is extended to allow a partial update to start, and then to re-use the 0x71 image data packets to send both the old and new images over the wire. Finally, the 0x72 is reused to complete the image upload, and do the partial refresh.

The etag

To help avoid state desynchronizations, because the client is now responsible for sending the old image along, which should be truthful to what is currently displayed, in order for the waveforms picked to be correct (to avoid force pushing a pixel beyond its acceptable range for example), an etag is added. This is a nonce, a tag, attached to the image shown, and is just a 32 bit integer. If the client presents the same etag that is currently shown, the client is trusted to pass in the proper old image so the updates are correct.

There are immediately two scenarios in which a client would be wrong about the "old image" currently displayed:

• There are two clients in play, with each their own images to display
• The display is rebooted unknownst to the client, so it no longer shows what the client thinks it shows

The etag is a quick way for the client to discover that the panel has drifted from its state, and to then perform a full refresh to reset it to a new known good state. It is up to the client to determine how to persist the etag and old image, if it needs to. The firmware only stores the new etag after a successful refresh; rejected or failed partial updates clear it, so the next update naturally falls back to a full refresh.

On ESP32, the etag is stored in RTC memory so it can survive deep sleep. On other targets it is a regular global and resets on reboot, which is fine; the next partial will fail the etag check and the client can recover with a full update.

Protocol shape

Full image uploads keep the existing shape:

0x70 DIRECT_WRITE_START
0x71 DATA...
0x72 DIRECT_WRITE_END + refresh

Compressed full-image uploads are buffered as compressed bytes while 0x71 packets arrive. They are decompressed during 0x72, in chunks, and those decompressed chunks are streamed to the display panel. We do not buffer a decompressed full image in memory.

Partial image uploads add a new start command:

0x76 PARTIAL_WRITE_START
0x71 DATA...
0x72 DIRECT_WRITE_END + partial refresh

The 0x76 payload is:

[version:1 = 0x01]
[flags:2 BE]
[old_etag:4 BE]
[x:2 BE][y:2 BE][width:2 BE][height:2 BE]
[uncompressed_size:4 LE]
[optional initial stream bytes...]

Flags:

0x0004: stream is zlib-compressed
0x0008: 0x72 includes a new etag to store after successful refresh
all other bits: reserved, must be 0

The partial stream contains two controller-plane images, concatenated:

old rectangle bytes for PLANE_1
new rectangle bytes for PLANE_0

This is controller RAM data, not a higher-level drawing command. For the supported 1bpp path, each controller plane is ceil(width / 8) * height bytes, so uncompressed_size must be plane_bytes * 2.

Raw partial 0x71 bytes are streamed directly into the panel driver. Once the old-image byte count is reached, the firmware opens the same rectangle on the new plane and continues writing there. Compressed partial data follows the same pattern as compressed full images: 0x71 buffers compressed bytes, and 0x72 inflates them in chunks through that same old/new plane sink.

The 0x72 payload is:

[refresh_mode:1]
[new_etag:4 BE]  only when 0x76 had flag 0x0008 set

For partial streams, the default refresh mode is partial. A refresh_mode of 1 requests fast refresh instead. A full-refresh request is deliberately treated as partial for partial streams.

Return messages and errors

Responses keep the existing direct-write shape:

ACK:  [0x00, opcode]
NACK: [0xFF, opcode, error, 0x00]

There is also still the older generic direct-write error:

[0xFF, 0xFF]

That is used in a few allocation/buffer cases where the old direct-write code already behaved that way.

0x70 direct write start

Possible responses:

[0x00, 0x70]                 start accepted
[0xFF, 0x70, 0x02, 0x00]     mixed full/partial data
[0xFF, 0xFF]                 compressed buffer unavailable or compressed start data exceeds capacity

0x76 partial write start

Possible responses:

[0x00, 0x76]                 start accepted
[0xFF, 0x76, 0x01, 0x00]     etag mismatch
[0xFF, 0x76, 0x03, 0x00]     rectangle out of bounds
[0xFF, 0x76, 0x04, 0x00]     unsupported partial protocol version or too-short start payload
[0xFF, 0x76, 0x05, 0x00]     rectangle x or width is not byte-aligned
[0xFF, 0x76, 0x06, 0x00]     unsupported/reserved flags, or currently unsupported backend
[0xFF, 0x76, 0x07, 0x00]     uncompressed_size does not match rectangle geometry
[0xFF, 0x76, 0x08, 0x00]     stream byte count or stream content error
[0xFF, 0x76, 0x09, 0x00]     partial update unsupported for this panel mode
[0xFF, 0xFF]                 compressed buffer unavailable

0x71 data

Possible responses:

[0x00, 0x71]                 data accepted
[0xFF, 0x71, 0x08, 0x00]     partial stream byte count or content error
[0xFF, 0xFF]                 compressed buffer unavailable

For raw full-image uploads, a final 0x71 can also complete the image and make the firmware enter the same end/refresh path as 0x72.

0x72 direct write end

Possible responses:

[0x00, 0x72]                 end accepted, refresh has been started
[0xFF, 0x72, 0x08, 0x00]     partial stream byte count/content error, decompression error, or missing new etag when requested
[0x00, 0x73]                 refresh completed successfully
[0x00, 0x74]                 refresh timed out

For both full and partial uploads, a successful refresh stores the new etag if one was supplied in the end payload. If refresh fails, the etag is cleared.

Future work

Decompression on the fly - While developing this, it occurs to me that in compressed imagery, the way the compression library (uzlib) is used means that the full image (compressed) is stored in MCU memory. Once done, it is decompressed during the 0x72 packet, and then streamed to the display panel. Cursory searching leads to other libraries, like miniz and tinf which should be two mini/tiny zlib inflaters. We don't need compression support, so thats a win. What I really want is a small inflater API that can receive a packet like a 0x71, decompress whatever it can, and then suspend while keeping the sliding dictionary state until the next packet arrives. This way, we can avoid buffering anything, besides the mandatory decompression dictionary.

Confusion about refresh modes - I've added 0x76 to do partials, which basically implies that the final 0x72 will be a partial refresh. If you submit a 0x72 with full after doing partial data, you'll get a garbled image out - I think. This also bleeds into the python library, where refresh mode is a parameter on the CLI, and on other tools, but you should not set these. It may be that future work should remove/deprecate the refresh mode from all packets, and let it be implicit based on the starting packet (0x70 or 0x76).

Caveats

• Only 1 bit per pixel panels are supported - during work, I found that bb_epaper apparently has no panels that have both 2 bits per pixel (color support) whilst also having a partial refresh initialization code. It may be that no color panels support partial refreshing. The current protocol design should be robust enough that its not an issue, in that we're sending images over. The 0x76 packet is rejected if the panel is >1bpp.
• Pixel alignment - as the updates are actually byte aligned, updates must be aligned to 8-pixels. That means both x and width must be multiples of 8, ie. x=0/8/16/24/...

Avenues checked and disregarded

I've been through a few designs. The commit history reveals this, but notable variants are:

• Sending very small dirty segments - I initially thought that less data at all costs is a benefit. So I had a protocol that send small segments, series of x/y/w/h+data so the MCU could write just those bits. It worked, but it was very slow, as compression was non-existent, which lead to many packets. And latency is the largest bottleneck of this system. Even adding compression did not fix it, because the overhead per segment was just too large, and that compression never really took off due to the small segments it worked on.
• Interleaving old & new images - I moved to reusing 0x71 to just send one big dirty rectangle to the device, and then use compression to win anything lost by the larger images. I then thought that the larger rectangle might hold more unchanged areas (what would previously be saved by not sending those segments), and that these unchanged areas would appear as series of bits that are identical, but across two images. So if we interleaved the images, like 256 bytes of A, 256 bytes of B, an so on, compression would be better. Empirical testing showed no change across a series of tests, so it was removed in favour of just flat out sending image A and then image B.

Changes

This PR adds these protocol changes:

• 0x76 is added to start a new partial image (much like 0x70)
• 0x72 is amended, to accept a "new etag"
• 0x71 is changed to also support sending two images in one stream, this has no protocol change

References:

• Client changes: Implement partial refreshing protocol py-opendisplay#42