[wip/draft] fix(proto): immediately retransmit in-flight data when a path is abandoned#770
[wip/draft] fix(proto): immediately retransmit in-flight data when a path is abandoned#770Frando wants to merge 1 commit into
Conversation
…doned When a path is abandoned, all in-flight packets on that path are now immediately declared as lost, causing their retransmittable frames to be queued for retransmission on another active path. Previously, in-flight packets remained in the abandoned path's sent_packets until the PathDrained timer fired. That timer only arms once we receive the peer's own PATH_ABANDON frame for this path (not an ACK of ours - the incoming PATH_ABANDON handler), 3*PTO after that; and that in turn requires some live path to carry the frame at all. In the meantime, loss detection kept cycling on the dead path via PTO probes that could never be ACKed, and the actual stream data was never retransmitted on the new active path. This caused multi-second stalls (up to ~20s observed) after path migration: the application's stream data was stuck on the abandoned path while a perfectly good new path was available. The LossDetection timer on the abandoned path is now also stopped, since all packets are declared lost immediately and there is nothing left to detect.
|
Documentation for this PR has been generated and is available at: https://n0-computer.github.io/noq/pr/770/docs/noq/ Last updated: 2026-07-24T12:22:21Z |
Performance Comparison Report
|
| Scenario | noq | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5238.3 Mbps | 7883.7 Mbps | -33.6% | 97.9% / 150.0% |
| medium-concurrent | 5324.4 Mbps | 7467.7 Mbps | -28.7% | 92.9% / 99.0% |
| medium-single | 3352.3 Mbps | 4688.9 Mbps | -28.5% | 92.3% / 100.0% |
| small-concurrent | 3726.5 Mbps | 5229.2 Mbps | -28.7% | 98.0% / 153.0% |
| small-single | 3327.2 Mbps | 4749.5 Mbps | -29.9% | 97.5% / 153.0% |
Netsim Benchmarks (network simulation)
| Condition | noq | upstream | Delta |
|---|---|---|---|
| ideal | 3072.9 Mbps | 4134.7 Mbps | -25.7% |
| lan | 782.4 Mbps | 810.3 Mbps | -3.5% |
| lossy | 69.9 Mbps | 55.9 Mbps | +25.0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
noq is 28.8% slower on average
|
Declaring all in-flight packets as lost when the path is abandoned is bad I think. These in-flight packets may still arrive. So you're potentially wasting a lot of resources. This should follow normal retransmission and loss detection as closely as possible IMHO. The problem is of course that loss is only declared once a new packet is received, which won't happen on an abandoned path. I've long been thinking that when the PTO fires for an abandoned path is the right time to start re-transmitting in-flight data on a new path. I also kind of wanted this to be more progressive, so that not all data is re-sent on a different path immediately, but rather more based on sent-time. That is of course not ideal either. Plus that's hard, the PTO timer only fires for the last packet sent. I almost want to set a per-packet PTO timer once path is abandoned, and each time it first send that packet's data on a different path. But that might just be too expensive. I looked for an issue and couldn't find any right away, which surprises me because this has been an issue we've been aware off for a while too. |
2 participants
Description
Experiment / draft
Change: When a path is abandoned, all in-flight packets on that path are now immediately declared as lost, causing their retransmittable frames to be queued for retransmission on another active path.
Previously, in-flight packets remained in the abandoned path's sent_packets until the PathDrained timer fired. That timer is only armed once we receive the peer's own PATH_ABANDON frame for this path, and then runs for 3*PTO. This means stream data is not retransmitted for at least 3*PTO when the previously-active path dies.
Breaking Changes
Notes & open questions
I had claude compare to picoquic and spec (not checked myself yet):
Change checklist
proposed change and wrote an as clear and concise description as
they could.
intented effect.
cargo makepasses locally.