Skip to content

Batch more database operations to improve the performance - #515

Open
MOZGIII wants to merge 6 commits into
mzg/2026-07-24/batch-vm-snapshotsfrom
mzg/2026-07-24/batch-more
Open

Batch more database operations to improve the performance#515
MOZGIII wants to merge 6 commits into
mzg/2026-07-24/batch-vm-snapshotsfrom
mzg/2026-07-24/batch-more

Conversation

@MOZGIII

@MOZGIII MOZGIII commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Goes in after #513


This PR extends the work done in the couple previous PRs to get the performance of the execution to production levels. As of this PR, the benches demonstrate respectable 114% workflows processing rate versus the old DAG-based approach.

  drain (25k) throughput vs DAG
Pre-batching ~26.8s ~935/s 0.36x (2.8x slower)
Legacy DAG ~9.7s ~2,586/s 1.0x
Batchers / median ~10.1s ~2,475/s ~0.96x
Batchers / best runs ~8.5s ~2,940/s ~1.14x

@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-vm-snapshots branch from b0e1415 to 8903d2c Compare July 31, 2026 13:32
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-more branch from 11341cd to 1f9985b Compare July 31, 2026 13:32
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-vm-snapshots branch from 8903d2c to cee2af8 Compare July 31, 2026 14:07
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-more branch 2 times, most recently from a3cfda7 to 3d1e659 Compare July 31, 2026 14:34
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-vm-snapshots branch from cee2af8 to 4e81e9a Compare July 31, 2026 14:34
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-more branch from 3d1e659 to f166cf9 Compare July 31, 2026 14:42
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-vm-snapshots branch from 4e81e9a to 8eb5588 Compare July 31, 2026 14:42
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-more branch from f166cf9 to 201f23f Compare July 31, 2026 14:54
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown

Coverage Report

Python Coverage

Metric Coverage
Lines 72.9%
Branches 58.2%

Download HTML Report

Rust Coverage

Metric Coverage
Lines 72.4% 🟢 (+0.3%)
Branches N/A

Download HTML Report

Compared to main branch

@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-vm-snapshots branch from 8eb5588 to 5a2cc0b Compare July 31, 2026 16:21
@MOZGIII
MOZGIII force-pushed the mzg/2026-07-24/batch-more branch from 201f23f to 1165d2f Compare July 31, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant