Skip to content

perf: reduce conversation viewer render time by up to 99% - #271

Open
ChakornK wants to merge 1 commit into
tintinweb:masterfrom
ChakornK:perf/md-rendering
Open

perf: reduce conversation viewer render time by up to 99%#271
ChakornK wants to merge 1 commit into
tintinweb:masterfrom
ChakornK:perf/md-rendering

Conversation

@ChakornK

Copy link
Copy Markdown

Summary

The conversation viewer rebuilt its entire transcript on every frame. buildContentLines() re-wrapped and re-truncated every line of every message on every render, and again on every scroll event. The per-message Markdown cache from #259 skips parsing, but nothing cached the assembled lines. At thousands of lines, it becomes extremely costly to render (~200ms/frame) and is guaranteed to go over the frame time budget. The viewer subscribes to the session, so a running agent redraws on every token and pays this cost continuously.

What changed

Cache the render, not just the parse.

ConversationViewer gains a contentCache, a WeakMap keyed by the message object holding that message's rendered block (header + body, lines pre-truncated). Three things invalidate an entry: terminal width, markdown mode, or the message text. A streaming append rebuilds only the streaming message. buildContentLines() is now an assembly loop over cached blocks plus separators. The per-role rendering moved into messageLines(), which runs only on a cache miss.

Performance

Results from npm run bench:ab -- master:

Comparing working tree against master (4f572ea), 3 round(s).

benchmark                                                                       4f572ea       working      delta
-------------------------------------------------------------------------  ------------  ------------  ---------
ConversationViewer.render — cold cache (first frame) > 50 messages              3.220ms       3.090ms      -4.0%
ConversationViewer.render — cold cache (first frame) > 500 messages            33.967ms      33.075ms      -2.6%
ConversationViewer.render — markdown: assistant (default) > 50 messages         2.119ms      184.57us     -91.3%
ConversationViewer.render — markdown: assistant (default) > 500 messages       20.070ms      266.64us     -98.7%
ConversationViewer.render — markdown: assistant (default) > 5000 messages     216.221ms       1.416ms     -99.3%
ConversationViewer.render — markdown: off (raw wrap) > 50 messages              2.331ms      175.50us     -92.5%
ConversationViewer.render — markdown: off (raw wrap) > 500 messages            24.684ms      260.85us     -98.9%
ConversationViewer.render — markdown: off (raw wrap) > 5000 messages          258.652ms       1.193ms     -99.5%

Testing

  • npm run lint: clean
  • npm run typecheck: clean
  • npm run test: 2129 passed, 7 skipped
  • npm run build: clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant