Skip to content

Answer the sender's closing handshake before dropping the socket - #32

Merged
op-q merged 3 commits into
mainfrom
fix/relay-teardown-drain
Aug 19, 2026
Merged

Answer the sender's closing handshake before dropping the socket#32
op-q merged 3 commits into
mainfrom
fix/relay-teardown-drain

Conversation

@op-q

@op-q op-q commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Fixes the relay teardown reset recorded as a follow-up in #30.

The defect

A sender could intermittently see Connection reset by peer after a transfer that actually succeeded — the receiver had confirmed the full byte count and written correct output.

The upload receive task broke out of its loop as soon as the sender sent complete, dropping ws_receiver. The socket stayed open after that, because the send task still owed the sender a transfer_complete status and a Close frame. A WebSocket close is a two-way handshake, so the peer answered that Close — into a socket the relay had stopped reading. Closing a socket with unread data queued sends RST rather than FIN, and the peer reports a reset.

Whether the user saw it was a race: the sender returns as soon as it reads transfer_complete, so the error surfaced only when its next socket operation touched the connection after the reset arrived.

The fix

It cannot live in the send task — that half owns the sink and has nothing to await on. Instead the receive task records that the sender finished cleanly and stays alive through teardown: it waits for the send task to exit (detected through the event receiver being dropped), then reads briefly until the peer's Close arrives.

The wait is deliberately two stages. A single short deadline does not work, because the send task cannot write transfer_complete until the receiver has finished writing the file — unbounded work. Both stages are bounded, so a peer that stops answering cannot hold the task and its per-IP connection slot open. The new WS_CLOSE_DRAIN_TIMEOUT_SECS bounds only the second stage.

Evidence

The defect does not reproduce through separate CLI processes — the sender exits before touching the socket again. It reproduces in the in-process test harness, with two transfers sharing one relay, which is the shape the tests had before #30.

Restoring that pre-#30 test file and running it repeatedly:

Build Runs Failures
main (unfixed) 30 3 (10%)
this branch 30 0
this branch 80 0

Failure text on main, for the record:

transfer should succeed: "IO error: Connection reset by peer (os error 104)"

That matches the ~16% #30 reported.

Why the removed assertions stay out

#30 removed the completion-handshake assertions from two tests and gave each transfer its own relay. I tried restoring the assertions and measured the result: with per-transfer relay isolation they never fail on unfixed main either (0/30). The isolation removes the contention that triggers the race, so those assertions would guard nothing.

The shape that does catch it — two transfers sharing one relay — detects at roughly 10% per run. That is too weak to serve as a CI guard and would reintroduce exactly the flakiness #30 removed. So the tests are unchanged here, and the reproduction stays a manual recipe rather than a CI assertion.

If a stronger guard is wanted later, it should be a targeted test that drives the close handshake directly rather than a probabilistic end-to-end race.

Verification

  • scripts/check-secrets.sh, cargo fmt --check, cargo clippy --workspace --all-targets --all-features -D warnings all pass.
  • cargo test --workspace --all-targets: 56 passed, 0 failed.
  • No protocol change; no client change.

The plan document for this work is in #31 and its evidence checkboxes should be ticked once that merges.

🤖 Generated with Claude Code

A sender could intermittently see `Connection reset by peer` after a transfer
that actually succeeded: the receiver had confirmed the full byte count and
written correct output, but the sender's last socket operation reported an
error.

The upload receive task broke out of its loop as soon as the sender sent
`complete`, which dropped `ws_receiver`. The socket stayed open after that,
because the send task still owed the sender a `transfer_complete` status and a
`Close` frame. A WebSocket close is a two-way handshake, so the peer answered
that `Close` with one of its own — into a socket the relay had stopped reading.
Closing a socket with unread data queued sends RST rather than FIN, and the
peer reports a reset.

Whether the sender saw it was a race: it returns as soon as it reads
`transfer_complete`, so the error surfaced only when its next socket operation
touched the connection after the reset arrived.

The fix cannot live in the send task, which owns the sink and has nothing to
await on. Instead the receive task records that the sender finished cleanly and
stays alive through teardown: it waits for the send task to exit, which it
detects through the event receiver being dropped, then reads briefly until the
peer's `Close` arrives.

The wait is deliberately two stages. A single short deadline would not work,
because the send task cannot write `transfer_complete` until the receiver has
finished writing the file, which is unbounded work. Both stages are bounded, so
a peer that stops answering cannot hold the task and its per-IP connection slot
open.

Evidence and the full diagnosis are in the pull request; the plan document
lands separately in #31.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@vercel

vercel Bot commented Aug 19, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
drop Ready Ready Preview Aug 19, 2026 1:05pm

op-q and others added 2 commits August 19, 2026 15:03
Marks the relay teardown plan done now that its documents are on main, and
writes down what the work actually produced rather than what it predicted.

Three things are recorded because they change what a reader should believe:

- the drain needed two stages, not the single short deadline the plan assumed,
  because the send task cannot write `transfer_complete` until the receiver has
  finished writing the file;
- the defect does not reproduce through separate CLI processes, so the first
  evidence harness measured nothing, and running it only against the fix would
  have produced a clean result that meant nothing;
- the assertions #30 removed stay out for a measured reason, and the download
  socket carries the same latent shape without a user-visible symptom, now
  tracked as its own unscheduled item.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@op-q
op-q merged commit ca9698f into main Aug 19, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant