Skip to content

vnc: read only the dirty columns, not whole scanlines#3627

Closed
bitranox wants to merge 2 commits into
microsoft:mainfrom
bitranox:vnc-partial-width-dirty-read
Closed

vnc: read only the dirty columns, not whole scanlines#3627
bitranox wants to merge 2 commits into
microsoft:mainfrom
bitranox:vnc-partial-width-dirty-read

Conversation

@bitranox

@bitranox bitranox commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Summary

When forwarding device dirty rectangles, the VNC server read only the dirty
rows but the full screen width of each, ignoring the rectangle's horizontal
extent. For a tall, narrow update (a dragged scrollbar, a text caret column,
a vertical widget) that reads far more VRAM than needed, up to roughly screen
width over rectangle width, which can approach a full-frame read for a tiny
change. Full-width updates were already optimal.

This reads only the dirty columns. The non-dirty columns of a dirty row are
already correct from the cur_fb/prev_fb swap (they did not change), and the
encoder only emits the merged rectangles, so the output is identical, just
with less VRAM traffic.

What changed

  • framebuffer: add View::read_line_at(line, x, data); read_line(line, data) now delegates to it with x = 0.
  • The vnc Framebuffer::read_line gains a start column x. The device-dirty
    loop reads [r.x, r.x + r.w) into the matching slice of cur_fb; the
    full-frame read and the tile-diff fallback pass x = 0.
  • Update the vncserver example for the new signature. This also fixes a
    latent byte-offset bug in its read_line: it indexed the byte view of the
    Vec<u32> framebuffer with a pixel offset, so every line after the first
    read the wrong region. Example only, no effect on shipping code or tests.

On the API shape

The shared framebuffer crate keeps read_line and adds read_line_at, so
existing callers (for example the screenshot capture in petri) do not change.
The vnc Framebuffer trait is internal to the vnc crate, so it takes the
start column directly rather than carrying two methods. There is an inline
comment on View::read_line noting this.

The byte arithmetic in read_line_at, including the 4-bytes-per-pixel factor
for the 32bpp framebuffer, is carried over unchanged from the previous
read_line (which only read whole lines). The one addition is the
start-column offset that read_line did not allow.

Testing

  • New unit test asserts only the dirty columns are read: a pixel changed
    outside the dirty rectangle on a dirty row is not picked up.
  • Existing vnc and framebuffer tests pass; cargo fmt --all -- --check,
    cargo clippy --workspace --all-targets, and cargo doc --workspace --no-deps are clean.
  • Ran a Windows guest over VNC with a video workload; the screen renders
    correctly with the column-limited reads, no stale or torn columns around
    updated regions.

Docs

Corrected the architecture guide, which described this partial read as if it
were already implemented.

@bitranox bitranox requested a review from a team as a code owner June 2, 2026 11:14
Copilot AI review requested due to automatic review settings June 2, 2026 11:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Optimizes VNC dirty-rect reads to only fetch the dirty columns within dirty rows, instead of full scanlines, reducing VRAM traffic for partial updates.

Changes:

  • Extends Framebuffer::read_line trait with an x start-column parameter and updates all implementations.
  • Adds View::read_line_at in the framebuffer device, with read_line kept as a whole-line wrapper.
  • Updates UpdateState device-dirty path to read only the dirty horizontal span and adds a unit test.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
workers/vnc_worker/vnc/src/update_state.rs Read only dirty columns per dirty rect; new test asserting non-dirty pixels aren't read.
workers/vnc_worker/vnc/src/traits.rs Updates read_line signature/docs to include start column x.
workers/vnc_worker/vnc/src/server.rs Updates mock/forwarding read_line impls for new signature.
workers/vnc_worker/vnc/examples/vncserver.rs Updates example read_line to support starting column.
workers/vnc_worker/src/lib.rs Wires SharedView::read_line to View::read_line_at.
vm/devices/framebuffer/src/lib.rs Adds read_line_at, refactors offset/length math, keeps read_line as wrapper.
Guide/src/reference/devices/vnc/architecture.md Updates docs to describe dirty-column reads.

Comment thread vm/devices/framebuffer/src/lib.rs
Comment thread workers/vnc_worker/vnc/examples/vncserver.rs
@github-actions github-actions Bot added the Guide label Jun 2, 2026
Comment thread vm/devices/framebuffer/src/lib.rs Outdated
Comment thread vm/devices/framebuffer/src/lib.rs
@bitranox

bitranox commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

@smalis-msft : sorry, I am a bit behind, I was chasing the whole day this nasty bug - skimming the new firmware and my own additions : #3646

@smalis-msft

Copy link
Copy Markdown
Contributor

No rush! We're plenty busy too haha, take your time.

When forwarding device dirty rectangles, the VNC server read only the dirty
rows but the full screen width of each, ignoring the rectangle's horizontal
extent. For a tall, narrow update (a dragged scrollbar, a text caret column,
a vertical widget) that reads far more VRAM than needed, up to roughly screen
width over rectangle width, which can approach a full-frame read for a tiny
change. Full-width updates were already optimal.

Read only the dirty columns. The non-dirty columns of a dirty row are already
correct from the cur_fb/prev_fb swap (they did not change), and the encoder
only emits the merged rectangles, so the result is identical, just with less
VRAM traffic.

- framebuffer: add `View::read_line_at(line, x, data)`; `read_line` delegates
  to it with x = 0, so existing callers are unchanged.
- vnc `Framebuffer::read_line` gains a start-column `x`; the device-dirty loop
  reads `[r.x, r.x + r.w)` into the matching slice of cur_fb. The full-frame
  read and tile-diff fallback pass x = 0.
- Add a test asserting only the dirty columns are read (a changed pixel
  outside the dirty rect on a dirty row is not picked up).
- Correct the architecture doc, which described this partial read as if it
  were already implemented.
@bitranox bitranox force-pushed the vnc-partial-width-dirty-read branch from 52f5381 to 01e74d6 Compare June 3, 2026 23:02
@bitranox bitranox requested a review from smalis-msft June 3, 2026 23:10
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

@bitranox

bitranox commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

merged into #3665

@bitranox bitranox closed this Jun 4, 2026
@bitranox bitranox deleted the vnc-partial-width-dirty-read branch June 9, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants