consomme: make per-connection TCP buffer sizes configurable#3595
consomme: make per-connection TCP buffer sizes configurable#3595benhillis wants to merge 2 commits into
Conversation
The hardcoded 256 KiB per-connection rx/tx ring buffers cap throughput at ~2 Gbps on paths whose bandwidth-delay product exceeds the buffer size. Expose `tcp_rx_buffer_size` and `tcp_tx_buffer_size` on `ConsommeParams` so embedders can pick a value appropriate for their workload. The default remains 256 KiB, preserving existing behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
There was a problem hiding this comment.
Pull request overview
Exposes per-connection TCP rx/tx ring buffer sizes through ConsommeParams so embedders can raise the previously-hardcoded 256 KiB cap that was bottlenecking TCP throughput on high-BDP paths. Defaults are unchanged.
Changes:
- Adds public
tcp_rx_buffer_size/tcp_tx_buffer_sizefields toConsommeParams, defaulting to a newDEFAULT_TCP_BUFFER_SIZEof 256 KiB. - Threads these values through
Consomme::newintotcp::Tcp::new, replacing the inline256 * 1024literals inConnectionParams.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| vm/devices/net/net_consomme/consomme/src/lib.rs | Adds default constant, public params fields, and propagates them to Tcp::new. |
| vm/devices/net/net_consomme/consomme/src/tcp.rs | Tcp::new now takes rx/tx buffer sizes instead of using hardcoded 256 KiB. |
|
I noticed the same findings (that our TCP buffer sizes in Consommé limit our throughput) in #3413. I think this is the right change to make. |
…tion The doc previously said these take effect for new connections, implying params_mut() updates would apply. In fact Consomme::new snapshots the sizes at construction and never resyncs, so runtime updates are ignored. Clarify the docs to match the actual behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Closing — superseded by #3597, which includes these configurable buffer commits and builds the autotune mechanism on top (now the default). No need to land this separately. |
The hardcoded 256 KiB per-connection rx/tx ring buffers in consomme cap TCP throughput at roughly 2 Gbps on paths whose bandwidth-delay product exceeds the buffer size.
Expose
tcp_rx_buffer_sizeandtcp_tx_buffer_sizeonConsommeParamsso embedders can pick a value appropriate for their workload. The default remains 256 KiB, preserving existing behavior.Testing
cargo build --releasecargo clippy --release -p consomme -p net_consomme --all-targetscargo nextest run -p consomme(55/55 pass)cargo doc --no-deps -p consomme -p net_consommecargo xtask fmt --fixEmpirically, raising the per-connection buffers to 1 MiB took
iperf3throughput across consomme from ~514 Mbps to roughly 3-4 Gbps in the host-to-guest direction. A follow-up will add autotuning so the default can grow without a fixed per-connection memory cost.