vnc_worker: add ZRLE and a configurable tile size, reduce redundant screen-update work, identify clients by bitranox · Pull Request #3665 · microsoft/openvmm

bitranox · 2026-06-04T18:44:29Z

This is the existing vnc_worker improvements PR, now also carrying the ZRLE
encoding and a configurable dirty-tracking tile size. It is squashed to one commit
and rebased on current main.

Screen-update work

Idle suppression: when no client is connected, the synthetic video device is
told to stop reporting screen and pointer updates, re-asserted on a
re-handshake. The device-side synth-video channels are bundled into one handle
so they stay paired across worker restart and OpenHCL paravisor wiring.
Partial-width dirty reads: only the dirty columns of each scanline are read,
not the whole row.
Full-refresh reuse: a non-incremental request reuses the device-maintained
framebuffer copy instead of re-reading all of VRAM, except on a first frame, a
resolution change, or a dropped broadcast.

Configurable tile size

--vnc-tile-size sets the dirty-tracking tile edge length (4, 8, or 16),
replacing the hardcoded 16. It defaults to 8, which measured best for bandwidth
and CPU across resolutions and workloads. The dirty bitmap and update state hold
the size as a field; a resolution change keeps it, and the paravisor console uses
the same default. A cycle value rotates through 2, 4, 8, 16, 32 every 30 seconds
and logs the bytes and CPU at each, for re-measuring; 2 and 32 are left out of the
offered set as dominated.

ZRLE encoding

A new framebuffer encoding (RFB encoding 16). Each rectangle is split into 64x64
tiles, and each tile is sent as the smallest applicable subencoding (solid, raw,
packed palette, plain RLE, palette RLE) through one continuous per-connection
zlib stream. CPIXELs drop the unused byte of a 32bpp true-colour pixel in the
client's byte order. The server prefers ZRLE over Zlib for any client that
offers it, then Zlib, then Raw, so a client that offers no plain zlib (TigerVNC)
no longer falls back to uncompressed Raw.

Client identification and diagnostics

guess_client matches the offered SetEncodings list against known viewer
signatures and logs a best-effort client_guess (or "unknown"), with the peer
address.
The dirty-rect backlog is drained and coalesced into a single broadcast per
wakeup, with a high-water-mark gauge and a rate-limited error if the queue
climbs past a ceiling. Each client connection runs in a span tagged with its
id.
The RFB encoding numbers move into one EncodingType open_enum so a number
and its name cannot drift apart.
A --vnc-max-clients of zero is now rejected.

Docs

Reference pages under the VNC guide: a client-capability matrix, a comparison
with other VNC servers, and the dirty-tracking tile-size measurements behind the
default.

Testing

75 unit tests in the vnc crate, including an in-test ZRLE reference decoder
that round-trips every subencoding (big-endian and high-3-byte CPIXEL layouts,
multi-byte packed rows, RLE runs over 255, multi-update continuous-stream
cases), and tile-to-pixel reconstruction at non-default tile sizes.
Live-validated on a Windows guest with five viewers (noVNC, TigerVNC, RealVNC,
MobaXterm, UltraVNC): each selects ZRLE and renders correctly, including video
playback and text-heavy screens. The tile-size default was set from per-size
bandwidth and CPU measurements at four resolutions, under static and
full-screen-video workloads.

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR improves VNC console efficiency and observability by plumbing a bidirectional synth-video control channel, reducing VRAM reads to dirty spans, and adding richer negotiation/dirty-path diagnostics.

Changes:

Bundle synth-video dirty-rect + “updates needed” channels into a single optional struct and plumb it through OpenVMM/Underhill/device wiring.
Optimize update collection to avoid unnecessary full VRAM rereads and support partial scanline reads (dirty columns only).
Add diagnostic logging (security type, pixel format, encodings) plus tests and updated guide docs.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
workers/vnc_worker_defs/src/lib.rs	Introduces `SynthVideoChannels` and updates `VncParameters` to carry both channels together.
workers/vnc_worker/vnc/src/update_state.rs	Refines full-refresh vs stale-buffer logic and reads only dirty columns; adds tests.
workers/vnc_worker/vnc/src/traits.rs	Extends `Framebuffer::read_line` to include an `x` offset for partial reads.
workers/vnc_worker/vnc/src/server.rs	Adds debug logging for security type, negotiated pixel format, and encoding negotiation.
workers/vnc_worker/vnc/src/rfb.rs	Adds `encoding_name()` helper for readable encoding diagnostics plus unit tests.
workers/vnc_worker/vnc/examples/vncserver.rs	Updates example framebuffer implementation for new `read_line(line, x, ...)` signature.
workers/vnc_worker/src/lib.rs	Wires `SynthVideoChannels`, client-id tracing spans, dirty backlog draining/coalescing, and updates-needed signaling.
workers/vnc_worker/Cargo.toml	Adds `tracelimit` dependency used for rate-limited logging.
vm/devices/uidevices_resources/src/lib.rs	Introduces `SynthVideoDeviceChannels` and updates `SynthVideoHandle` to carry paired channels.
vm/devices/uidevices/src/video/protocol.rs	Removes unused `FeatureChangeMessageV2` definition.
vm/devices/uidevices/src/video/mod.rs	Adds “updates needed” receiver, sends FeatureChange, gates dirt forwarding, and adds tests.
vm/devices/uidevices/src/resolver.rs	Unpacks `SynthVideoHandle.channels` into device constructor parameters.
vm/devices/framebuffer/src/lib.rs	Adds `View::read_line_at()` to support partial scanline reads.
petri/src/vm/openvmm/construct.rs	Updates synth-video handle construction for new `channels` field.
openvmm/openvmm_entry/src/lib.rs	Adds reverse “updates needed” channel wiring when `--gfx` is enabled and passes synth-video channels to VNC worker.
openhcl/underhill_core/src/worker.rs	Updates Underhill remote console channel wiring to use paired dirty/updates channels.
openhcl/underhill_core/src/lib.rs	Refactors remote console setup return type and wires paired channels into `VncParameters`.
Guide/src/reference/devices/vnc/what-we-are-proud-of.md	Updates documentation to reflect partial dirty-column reads and guest update suppression.
Guide/src/reference/devices/vnc/architecture.md	Updates architecture docs for new channel bundling, stale-buffer semantics, and backlog coalescing.

github-actions · 2026-06-05T21:33:12Z

1 Petri tests failed (1 unstable)

bitranox · 2026-06-06T16:20:21Z

rate-limit an error if the unbounded mesh queue ever climbs past a ceiling, which

for performance reasons i propose to change it to a non-rate limited debug message instead.

Copilot

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 2 comments.

bitranox · 2026-06-08T20:56:11Z

@smalis-msft - maybe You can take a look, I tested and refractored again. I acknowledge jstarks concerns but the channel is unbound by behaviour - we consume the dirty rects much faster then the device can are produce, so we never where able to provoke more then a few rectangles piling up (single digits)

Copilot

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.

bitranox · 2026-06-09T13:27:59Z

@smalis-msft @jstarks would be great if you could skim this and merge it in. There's still a lot left to do. Next up I want to tackle ZRLE encoding so remote viewers aren't stuck on just ZLIB or RAW. That's the biggest win for remote video performance right now.
Whether we also add Tight is up for discussion. My take is that VNC is best for quick admin tasks, while people doing serious remote work will reach for RDP, AnyDesk, TeamViewer, Chrome Remote Desktop, RustDesk, whatever fits.
I don't want to push for anything unreasonable, but a lot of the code overlaps across the changes I have planned, and that's slowing me down. So I'd rather not treat this as a final state. If you could merge it in, it would make the next steps a good bit easier for me.

smalis-msft · 2026-06-09T15:18:47Z

I will take a pass on both of your current PRs sometime today.

bitranox · 2026-06-09T15:35:32Z

let me sneak in ZRLE compression - almost finished

Copilot

Pull request overview

Copilot reviewed 27 out of 28 changed files in this pull request and generated 4 comments.

bitranox · 2026-06-09T19:39:47Z

@smalis-msft : adopted the minimal docstring style and took out quemu references - except in the VNC server feature comparison documentation - it should be ready for review now.

maybe You discuss internally if we need Tight encoding or not (mostly for jpeg picture frames where ZRLE falls back to ZLIB compression performance) - or what to tackle next ? Authentication ? Mouse Pointer ?

github-actions · 2026-06-09T20:32:53Z

25 Petri tests failed (4 unstable)

smalis-msft · 2026-06-09T20:34:37Z

I think our SNP runner broke...

bitranox · 2026-06-09T20:38:24Z

I think our SNP runner broke...

thanks - was just about looking into it.

Copilot

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 4 comments.

…creen-update work, identify clients Combines several vnc_worker changes. The dirty-rect ones started as separate branches that overlapped heavily and conflicted on every rebase, and they were largely reviewed already, so they are folded in here together with the new ZRLE encoding. ZRLE: a framebuffer encoding (RFB encoding 16) that splits a rectangle into 64x64 tiles and sends each tile as the smallest applicable subencoding, all through one continuous per-connection zlib stream: solid (one CPIXEL), raw (one CPIXEL per pixel), packed palette (up to 16 colours, indices packed MSB-first and padded to a byte per row), and plain or palette RLE (runs of CPIXELs or palette indices, using the 255-continuation run-length scheme). A CPIXEL drops the unused byte of a 32bpp true-colour pixel (3 bytes instead of 4) in the client's byte order, falling back to the full pixel for other formats. The server prefers ZRLE over Zlib whenever a client offers it (ZRLE, then Zlib, then Raw), which also stops TigerVNC, which offers no plain zlib, from falling back to uncompressed Raw. An in-test reference decoder exercises every subencoding, including big-endian and high-3-byte CPIXEL layouts, packed rows spanning multiple index bytes, RLE runs longer than 255, and two updates decoded through one continuous stream. Idle suppression: when no VNC client is connected, ask the synthetic video device to stop reporting screen and pointer updates, and re-assert that if the device re-handshakes while idle. The device-side synth-video channels (dirty rects plus the updates-needed signal) are bundled into one handle so they stay paired across worker restart and OpenHCL paravisor wiring. Partial-width dirty reads: read only the dirty columns of each scanline rather than the whole row when building an update. Full-refresh reuse: when a client requests a non-incremental update and the device is already keeping our framebuffer copy current, send that copy instead of re-reading all of VRAM. A first frame, a resolution change, or a dropped broadcast still forces a fresh full read. Configurable tile size: --vnc-tile-size sets the dirty-tracking tile edge length (4, 8, or 16), replacing the hardcoded 16 and defaulting to 8, which measured best for bandwidth and CPU across resolutions and workloads. The dirty bitmap and update state hold the size as a field rather than a constant; a resolution change keeps it, and the paravisor console uses the same default. A cycle value rotates through 2, 4, 8, 16, 32 every 30 seconds, logging the bytes and CPU at each for re-measuring; 2 and 32 are otherwise left out of the offered set as dominated. The flag and the measurements behind the default are in the VNC guide. Diagnostics: drain the whole dirty-rect backlog each wakeup, coalesce it into a single broadcast, track its high-water mark, and rate-limit an error if the unbounded mesh queue ever climbs past a ceiling, which would mean a wedged consumer. Each client connection runs in a span tagged with its id so the log lines from concurrent clients can be told apart. Client identification: RFB carries no client name, but the SetEncodings list is characteristic of each viewer. guess_client matches the offered set against known signatures (noVNC, TigerVNC, RealVNC, MobaXterm, UltraVNC) and logs the result as client_guess on the client encodings line, or "unknown" when nothing matches. The peer address is logged on that line too, and the full decoded offered list stays at debug. A reference page under the VNC guide records the encodings each client advertises next to what the server implements. Encoding numbers: the RFB encoding and pseudo-encoding numbers now live in one EncodingType open_enum instead of a few u32 constants sitting beside a separate integer match in encoding_name, so a number and its name cannot drift apart. The wire field stays unsigned big-endian; the handful of wire sites reinterpret the bits through a single helper. The worker's test client shares the type instead of keeping its own copies of the numbers, and ZRLE adds its number to the same open_enum. Also reject a --vnc-max-clients of zero, which would otherwise open the port but refuse every connection.

Copilot

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 2 comments.

github-actions · 2026-06-10T16:18:15Z

1 Petri tests failed (1 unstable)

bitranox · 2026-06-10T18:58:59Z

@smalis-msft gh runners look a bit flaky today

smalis-msft · 2026-06-10T19:04:52Z

Yeah was probably https://www.githubstatus.com/incidents/fcj3088jg1wx

bitranox requested a review from a team as a code owner June 4, 2026 18:44

Copilot AI review requested due to automatic review settings June 4, 2026 18:44

github-actions Bot added the Guide label Jun 4, 2026

Copilot AI reviewed Jun 4, 2026

View reviewed changes

Comment thread workers/vnc_worker/src/lib.rs Outdated

Comment thread vm/devices/framebuffer/src/lib.rs Outdated

Comment thread workers/vnc_worker/src/lib.rs

This was referenced Jun 4, 2026

vnc: diagnostics for client negotiation and the dirty-rect channel #3628

Closed

vnc: read only the dirty columns, not whole scanlines #3627

Closed

vnc: stop guest screen/pointer reporting when no client is connected #3614

Closed

bitranox force-pushed the vnc-worker-improvements branch from 75b3554 to 2d8045d Compare June 4, 2026 19:18

Copilot AI review requested due to automatic review settings June 8, 2026 20:31

bitranox force-pushed the vnc-worker-improvements branch from 2d8045d to 29f6dc0 Compare June 8, 2026 20:31

Copilot AI reviewed Jun 8, 2026

View reviewed changes

Comment thread workers/vnc_worker/src/lib.rs

Comment thread workers/vnc_worker/src/lib.rs

bitranox force-pushed the vnc-worker-improvements branch from 29f6dc0 to a481297 Compare June 8, 2026 23:16

Copilot AI review requested due to automatic review settings June 9, 2026 12:48

bitranox force-pushed the vnc-worker-improvements branch from a481297 to 18e3ce0 Compare June 9, 2026 12:48

bitranox changed the title ~~vnc_worker: reduce redundant screen-update work and add channel diagnostics~~ vnc_worker: reduce redundant screen-update work, identify clients, add diagnostics Jun 9, 2026

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread workers/vnc_worker/src/lib.rs

Comment thread Guide/src/SUMMARY.md

bitranox force-pushed the vnc-worker-improvements branch from 18e3ce0 to b689f5e Compare June 9, 2026 17:10

bitranox changed the title ~~vnc_worker: reduce redundant screen-update work, identify clients, add diagnostics~~ vnc_worker: add ZRLE, reduce redundant screen-update work, identify clients Jun 9, 2026

Copilot AI review requested due to automatic review settings June 9, 2026 19:03

bitranox force-pushed the vnc-worker-improvements branch from b689f5e to 64228ee Compare June 9, 2026 19:03

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread workers/vnc_worker/src/lib.rs

Comment thread Guide/src/SUMMARY.md

Comment thread workers/vnc_worker/src/lib.rs

Comment thread workers/vnc_worker/vnc/src/encoder.rs

smalis-msft reviewed Jun 9, 2026

View reviewed changes

Comment thread openhcl/underhill_core/src/worker.rs Outdated

smalis-msft reviewed Jun 9, 2026

View reviewed changes

Comment thread vm/devices/uidevices/src/video/protocol.rs

bitranox force-pushed the vnc-worker-improvements branch from 64228ee to 2c28b72 Compare June 10, 2026 07:53

bitranox requested review from Copilot and smalis-msft June 10, 2026 07:57

bitranox changed the title ~~vnc_worker: add ZRLE, reduce redundant screen-update work, identify clients~~ vnc_worker: add ZRLE and a configurable tile size, reduce redundant screen-update work, identify clients Jun 10, 2026

bitranox force-pushed the vnc-worker-improvements branch from 2c28b72 to e54fada Compare June 10, 2026 12:56

Copilot AI reviewed Jun 10, 2026

View reviewed changes

Comment thread Guide/src/SUMMARY.md

Comment thread openvmm/openvmm_entry/src/cli_args.rs

Comment thread Guide/src/reference/devices/vnc/architecture.md Outdated

Comment thread workers/vnc_worker/vnc/src/encoder.rs

Copilot AI review requested due to automatic review settings June 10, 2026 13:12

bitranox force-pushed the vnc-worker-improvements branch from e54fada to bbbdcaa Compare June 10, 2026 13:12

Copilot AI reviewed Jun 10, 2026

View reviewed changes

Comment thread workers/vnc_worker/vnc/src/update_state.rs

Comment thread workers/vnc_worker/vnc/src/encoder.rs

Conversation

bitranox commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Screen-update work

Configurable tile size

ZRLE encoding

Client identification and diagnostics

Docs

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 5, 2026

Uh oh!

bitranox commented Jun 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

bitranox commented Jun 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

bitranox commented Jun 9, 2026

Uh oh!

smalis-msft commented Jun 9, 2026

Uh oh!

bitranox commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bitranox commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 9, 2026

Uh oh!

smalis-msft commented Jun 9, 2026

Uh oh!

bitranox commented Jun 9, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

bitranox commented Jun 10, 2026

Uh oh!

smalis-msft commented Jun 10, 2026

bitranox commented Jun 4, 2026 •

edited

Loading

bitranox commented Jun 9, 2026 •

edited

Loading