Feat/deferred disconnect - #17
Merged
Merged
Conversation
disconnectClient() called Channel::broadcastMessage() per channel the departing client belonged to, so a peer sharing N channels with them got the same QUIT line queued N times. Reuse the alreadySent-set pattern from broadcastToChannels() (used for NICK) so each peer gets it once. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
disconnectClient() closed the fd synchronously, discarding whatever was still queued in _out -- most visibly ERR_PASSWDMISMATCH (464) on a wrong PASS, which never reached the client before the silent close. It now announces the departure immediately (teardownClientState: QUIT to peers, leave channels, extension fan-out, log/audit) but only closes the fd once _out drains via the existing EPOLLOUT-gated handleClientOutput(), or a 5s PENDING_CLOSE_TIMEOUT safety net (checkPendingCloseTimeouts(), forcing an abortive SO_LINGER close if a peer never frees enough window). No new send() call site was added; the existing poll-before-send path is reused. SendQ-exceeded and socket-error disconnects still close immediately via the new disconnectClientNow(), since deferring those would recreate the T6 frozen-reader scenario or write to an already-errored socket. A pending-close client stops accepting new input (EPOLLIN dropped, and the handleClientInput() batch loop bails out too) until it's torn down. MAX_CLIENTS "Server full" rejection in acceptClient() remains out of scope and an accepted regression, documented in CLAUDE.md: that client never enters _clients, so the mechanism can't reach it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…precision Addresses the three "should fix" items from the adversarial review of T4 (.claude/workflow/tasks/T4-deferred-disconnect/04-review.md): 1. teardownClientState() is now self-guarding: it marks the client as tearing down (Client::isTearingDown()) as its first statement, before the QUIT broadcast or extension fan-out run. Previously the reentrancy guard was only set at the very end of disconnectClient(), so an extension's onClientDisconnect() calling back into disconnectClient()/ disconnectClientNow() for its own fd — legal per the seam, undocumented as unsafe — re-ran teardown and could delete the Client* out from under the outer call's own fan-out loop. Reproduced the crash on the pre-fix commit with a worktree (segfault right after registration) and added ReentrantDisconnectTest.ReentrantOnClientDisconnectIsNoOp (test_extensions.cpp) to pin the fixed no-op behavior. Documented the hazard on IServerExtension::onClientDisconnect. 2. PENDING_CLOSE_TIMEOUT is now injected via a Server constructor parameter (default unchanged, still the 5s macro) instead of used directly, so tests can shrink it. _pendingCloseSince/now are gettimeofday- based (sub-second) since a sub-second injected deadline needs sub-second resolution to be observable. DeferredCloseDeadlineTest now injects 0.2s and polls via a second, healthy connection to keep the event loop ticking (checkPendingCloseTimeouts only runs once per pass, and idle epoll_wait has its own independent ~1s ceiling) instead of sleeping a fixed 8s (5s deadline + 3s margin). 3. Precised the T4 drain guarantee in Server.cpp and CLAUDE.md: the pending-close deadline is a flat ceiling, not a stuck-peer detector — it can abort a client that's legitimately still draining (slow link), not just T6's frozen reader, silently losing whatever hadn't gone out. Documented as an accepted trade-off rather than left implicit. Verified: full suite 456/456 on a clean `make test`, RobustnessTest (T6) 70/70 under --gtest_repeat=5, all three tiers build -Werror clean, scripts/audit.sh and scripts/normalize.sh clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…mitted) gettimeofday is neither C++98 nor in the subject's External functions list. Revert the sub-second precision introduced for test injection back to time_t/std::time. The injectable PENDING_CLOSE_TIMEOUT stays, now in whole seconds; the deadline test injects 1s, running in ~0.8s instead of 8s.
…ntual FrozenPeerClosedByDeadlineNotDrain only asserted 'eventually closed', so it passed even when the client closed by drain-complete. Now asserts close time falls near the injected deadline. Two further fixes surfaced: the probe tracked a raw fd that could be reused by a new client (now gated on isPendingClose()), and default tcp_rmem (128KiB) exceeds MAX_SENDQ (64KiB) so no sub-SendQ payload ever froze — shrink the client's SO_RCVBUF to 4KiB to force real backpressure. Verified: drain-fast mutation FAILs, frozen PASSes 10/10.
CLAUDE.md still described the reverted gettimeofday/sub-second impl as current; _pendingCloseSince is time_t/std::time now. Also records why the build uses .NOTPARALLEL (concurrent multi-tier builds OOM-freeze machines; forcing -j in MAKEFLAGS emits a jobserver warning audit.sh treats as failure).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: defer client disconnect until queued output drains (T4)
Summary
When the server closed a connection that still had data queued in
_out— themotivating case being
464 ERR_PASSWDMISMATCHon a wrong password — the clientreceived nothing:
disconnectClient()closed the fd before the queuedrained. A wrong-password client saw a silent connection drop, no reason given.
That violates the modern IRC spec's MUST (send 464 and/or
ERRORbeforeclosing) and is visible in HexChat, the evaluation's reference client.
T4 makes the close deferred: the departure is announced immediately, but the
fd is physically closed only once
_outdrains via the existingEPOLLOUTpath, or a safety deadline elapses.
Why this is not a revert of T3
T3 deliberately removed two
send()calls that ran without a prior poll.T4 does not bring them back. The deferred close drains through
handleClientOutput— the onesend()that is alreadyEPOLLOUT-gated — so ithonours T3's rule and still delivers the reply. Zero new un-polled sends.
Mechanism
disconnectClient()marks the client pending-close, removes it from itschannels immediately (so
_outis a fixed snapshot, not growing), and lets thenormal loop drain it. A pending-close client stops reading input — it is
write-only until it dies. Cases that must not defer close immediately via
disconnectClientNow():would re-introduce the frozen-reader scenario T6 guards. Immediate close.
disconnectClientin thefirst place; explicitly out of T4's scope (CLAUDE.md corrected accordingly).
A per-client
time_tdeadline (PENDING_CLOSE_TIMEOUT, 5s prod, injectable fortests) bounds a client that never drains; on expiry the socket is closed
abortively (
SO_LINGER{1,0}), confined tocheckPendingCloseTimeouts.Review history — two rounds, and what they caught
This branch went through two adversarial review rounds. Worth recording because
the fixes are subtle:
onClientDisconnect(use-after-free, reproduced as a real segfault beforefixing — now guarded by
isTearingDown()), and that the deadline test paid afixed 8s sleep (made the timeout injectable).
gettimeofdayintroduced for sub-second test injection was reverted: itis neither C++98 nor in the subject's External-functions list. Back to
time_t/std::time, whole-second injection."eventually closed", so it passed even when the client closed by
drain-complete rather than by the deadline. Fixed to assert close time falls
near the injected deadline. Implementing that surfaced two more real issues:
the probe tracked a raw fd that a new client could reuse (now gated on
isPendingClose()), and defaulttcp_rmem(128KiB) exceedsMAX_SENDQ(64KiB), so no sub-SendQ payload ever actually froze — the client's
SO_RCVBUFis now shrunk to 4KiB to force genuine backpressure.Verification
against pre-T4, passes now.
(close in ~20ms, below the deadline floor); frozen, it PASSES, stable 10/10.
finalizeDisconnectsites traced against 5 trigger paths,guards hold.
--gtest_repeat=5); full suite 456/456; three tiers-Werrorclean viamake verify-tiers; audit and normalize clean.Note on the test count
"456 assertions" is PostMan's assertion counter, not a test-case count: 155 are
real product tests, 301 are
PostManTruncationRegression's own self-checks.