perf(websocket): broadcast to all peers concurrently instead of one at a time - #1186
Conversation
WebSocketConnectionManager.broadcast() awaited send_text() one connection at a time, so every client waited for the ones ahead of it. A single slow or backpressured peer delayed delivery to the entire fleet (head-of-line blocking) and total broadcast latency grew with the connection count. Issue the sends via asyncio.gather(..., return_exceptions=True) over a snapshot of active_connections, then drop only the peers whose send raised. Each WebSocket owns an independent send buffer, so there is no shared pooled resource to bound here and concurrency is naturally limited by the number of connected clients. isinstance(result, Exception) mirrors the previous `except Exception` clause, so BaseException-only failures propagate exactly as before. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited) Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
|
@coderabbitai review Specific challenge questions for this change — please push back hard on any of these:
|
Agent Completion Truth Gate: NOT_APPLICABLEEvidence agrees. Machine-readable verdict{
"details": {},
"reasons": [],
"verdict": "not_applicable"
} |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== Changed files ==='
git diff --name-status HEAD^ HEAD
echo '=== websocket_service outline ==='
ast-grep outline src/youtube_extension/backend/services/websocket_service.py --items all || true
echo '=== websocket_service diff ==='
git diff HEAD^ HEAD -- src/youtube_extension/backend/services/websocket_service.py
echo '=== focused test diff ==='
git diff HEAD^ HEAD -- tests/unit/test_websocket_service.py
echo '=== broadcast definitions and calls ==='
rg -n -C 4 '\bbroadcast\s*\(' . -g '*.py' -g '!**/.git/**' || true
echo '=== connection lifecycle references ==='
rg -n -C 4 '\b(active_connections|connect|disconnect)\b' src tests -g '*.py' || true
echo '=== server concurrency and connection-limit configuration candidates ==='
rg -n -i -C 3 '(limit[-_ ]?concurrency|max[-_ ]?(connections|clients)|websocket|uvicorn|hypercorn)' \
Dockerfile docker-compose.full.yml pyproject.toml src -g '*.py' -g '*.toml' -g '*.yml' -g '*.yaml' || true
echo '=== focused test file outline ==='
ast-grep outline tests/unit/test_websocket_service.py --items all || trueLength of output: 123178 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== websocket_max_connections uses ==='
rg -n -C 3 '\bwebsocket_max_connections\b|WEBSOCKET_MAX_CONNECTIONS' . \
-g '*.py' -g '*.toml' -g '*.yml' -g '*.yaml' -g '!**/.git/**' || true
echo '=== connection manager and broadcast service ranges ==='
sed -n '21,82p' src/youtube_extension/backend/services/websocket_service.py
sed -n '320,347p' src/youtube_extension/backend/services/websocket_service.py
echo '=== production configuration range ==='
sed -n '65,115p' src/youtube_extension/backend/config/production_config.py
sed -n '240,260p' src/youtube_extension/backend/config/production_config.py
echo '=== existing broadcast tests and local test helpers ==='
rg -n -C 8 'test_broadcast_(sends_to_all|removes_failed_connections|empty_connections_no_error)|def _make_ws|class TestBroadcast' \
test_websocket_service.py tests/unit/test_websocket_service.py 2>/dev/null || true
echo '=== all direct broadcast attribute calls, excluding definition ==='
rg -n -P -C 3 '(?<!def )\bbroadcast\s*\(' src \
-g '*.py' -g '!src/youtube_extension/backend/services/websocket_service.py' || trueLength of output: 10017
Answers to the remaining questions:
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Pull request overview
Concurrently broadcasts WebSocket messages to reduce head-of-line blocking.
Changes:
- Uses
asyncio.gather()for concurrent fan-out. - Adds concurrency, failure-isolation, and empty-state tests.
- Cancellation propagation needs correction.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
websocket_service.py |
Implements concurrent broadcasting. |
test_websocket_service.py |
Tests broadcast fan-out behavior. |
Automated red-team review — result: clean, merge-readyReviewed at head
CI noteThe only red status on this head is Vercel — "Canceled from the Vercel Dashboard", a manual cancellation, not a build failure. The required Terminal state:
|
asyncio.gather(..., return_exceptions=True) captures a child CancelledError
into the results list rather than propagating it. Filtering results with
isinstance(result, Exception) therefore silently swallowed cancellations that
the previous 'except Exception' loop let escape - the opposite of the parity
this change claimed.
Collect BaseException-only results, clean up the ordinary per-peer failures
first so a cancellation cannot leak dead connections, then re-raise.
Adds two tests, both proven to fail without the re-raise
('DID NOT RAISE CancelledError').
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Automated red-team pass — merge-ready, pending human publish gateReviewed the diff adversarially (CodeRabbit is currently No blocking defects found. Notes:
CI status: the gating Disposition: Generated by Claude Code |
Review response —
|
|
Review synthesis (responding to the review request). I independently verified the three findings Copilot and CodeRabbit raised, rather than echoing them — verdicts and scope below. 1.
|
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== current HEAD and changed files ==='
git rev-parse HEAD
git diff --name-status HEAD^ HEAD
echo '=== websocket service outline ==='
ast-grep outline src/youtube_extension/backend/services/websocket_service.py --items all
echo '=== broadcast implementation and lifecycle methods ==='
rg -n -C 12 'async def (broadcast|connect|disconnect)\b|def (connect|disconnect)\b' \
src/youtube_extension/backend/services/websocket_service.py
echo '=== focused source diff ==='
git diff HEAD^ HEAD -- src/youtube_extension/backend/services/websocket_service.py
echo '=== focused test diff ==='
git diff HEAD^ HEAD -- tests/unit/test_websocket_service.py
echo '=== cancellation and fan-out test contexts ==='
rg -n -C 10 'cancel|CancelledError|TestBroadcastFanOut|test_.*broadcast|test_.*slow' \
tests/unit/test_websocket_service.py
echo '=== broadcast call sites outside its definition ==='
rg -n -P -C 4 '(?<!def )\bbroadcast\s*\(' src \
-g '*.py' -g '!src/youtube_extension/backend/services/websocket_service.py'Length of output: 8429 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== locate focused test file ==='
fd -a -i '^test_websocket_service\.py$' . || true
echo '=== full broadcast implementation ==='
sed -n '45,100p' src/youtube_extension/backend/services/websocket_service.py
echo '=== test file tracked path and current blob ==='
git ls-tree -r --name-only HEAD | rg '(^|/)test_websocket_service\.py$' || true
git show HEAD:tests/unit/test_websocket_service.py | sed -n '520,680p' || true
echo '=== async test configuration ==='
rg -n -C 3 '(asyncio_mode|pytest\.mark\.asyncio|pytest-asyncio)' \
pyproject.toml pytest.ini setup.cfg tox.ini conftest.py tests 2>/dev/null || true
echo '=== complete PR diff ==='
git diff HEAD~1 HEAD -- src/youtube_extension/backend/services/websocket_service.py tests/unit/test_websocket_service.pyLength of output: 13371
One test robustness issue remains:
Change the test to assert that both fast peers complete before the slow peer, without asserting an order between No other blocking issue found in the cancellation change. 🐇 ✅ ✏️ Learnings added
If you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! ✅ Action performedReview finished.
|
test_slow_peer_does_not_delay_other_peers asserted an exact completion order of ["fast-1", "fast-2", "slow"], but both fast peers use sleep(0) so their order relative to each other is scheduler-dependent and can flake in CI even when fan-out is correct (flagged by CodeRabbit and Copilot). Assert only the meaningful invariant: both fast peers complete before the slow head-of-list peer, without ordering the two fast peers. Still fails against a sequential broadcast (slow first -> slow completes first). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KmYe7a6hFvFC5uA19YGTQ7
|
Review triage on the three points raised (CodeRabbit + Copilot):
At head Generated by Claude Code |
CI triage on
|
|
CI triage on
None of these block on code in this PR. Substantively the PR is ready: review findings resolved (#1 in Generated by Claude Code |
Canonical issue
Closes #1185
Outcome
WebSocketConnectionManager.broadcast()now fans out to all connected peers concurrentlyinstead of awaiting each
send_text()in sequence.Previously each client waited for every client ahead of it in
active_connections, so a singleslow or backpressured peer delayed delivery to the entire fleet and broadcast latency for the
last client was the sum of all preceding sends. Now the sends are issued together and the
broadcast completes in roughly the time of the slowest peer rather than the total of all
peers.
send_text()(5 peers)slow, fast-1, fast-2fast-1, fast-2, slowWhat this does not change: the number of frames sent, per-peer send cost, message ordering
within a single connection, or the removal semantics for dead peers.
Scope
One function plus tests.
src/youtube_extension/backend/services/websocket_service.py—broadcast()(+import asyncio)tests/unit/test_websocket_service.py— newTestBroadcastFanOut(5 tests)No API, signature, or behavioural contract changes. The three pre-existing
broadcasttests(
test_broadcast_sends_to_all,test_broadcast_removes_failed_connections,test_broadcast_empty_connections_no_error) are untouched and still pass.Design notes
Why
gatherhere, when #1153 and #1169 deliberately used a bounded worker pool?Those changes fanned out over a shared, finite resource — a Redis connection pool and a
Firestore client — where unbounded concurrency causes pool exhaustion and quota pressure, so a
semaphore was load-bearing. A WebSocket broadcast is the opposite shape: each peer owns an
independent send buffer, there is no shared pool to exhaust, and the task count is already
bounded by the number of connected clients (which the server admits and tracks). Adding a
semaphore here would reintroduce exactly the serialisation skew this change removes, so it would
be a pessimisation rather than a safeguard.
Failure isolation.
return_exceptions=Trueensures one dead peer cannot abort delivery tothe rest — under
gather's default, the first raise would cancel the remaining sends.Exception parity (corrected at
69f0b9848). The original version of this note claimed thatisinstance(result, Exception)alone preserved the previousexcept Exception:semantics becauseCancelledErrorderives fromBaseException. That was wrong, and@Copilotcaught it:gather(..., return_exceptions=True)captures a childCancelledErrorintoresultsinstead ofpropagating it, so filtering on
Exceptionsilently swallowed a cancellation that the old loop letescape.
broadcast()now collectsBaseException-but-not-Exceptionresults, cleans up theordinary per-peer failures first (so a cancellation cannot leak dead connections), and then
re-raises. Two tests pin this, both proven to fail without the re-raise
(
Failed: DID NOT RAISE CancelledError).Snapshot.
connections = list(self.active_connections)is taken up front becausedisconnect()mutates that list during cleanup;zip(..., strict=True)then guarantees resultsstay aligned with their connections.
Empty case. An early
returnavoids constructing an emptygather.Risk
Low. Single function, no signature change, fully covered by pre-existing plus new tests.
The one behavioural difference worth naming: sends now start in parallel, so peers no longer
receive the message in strict list order. Broadcast has no cross-peer ordering contract (ordering
within a connection is unchanged and guaranteed by the transport), and the previous ordering was
an artefact of the serial loop rather than a designed property.
Verification
At head
97e1d4f01837259b5800822ca62934a36533bcbc:Non-vacuity — the new tests fail against the previous sequential implementation. Reverting
only the source (
git stash push <file>) and re-running the new tests:The remaining 3 tests are invariants (all peers receive, failures isolated, empty no-op) that hold
under both implementations by design.
Production evidence
websocket_serviceis reachable from the containerised production entrypointyoutube_extension.main:app(rootDockerfile:93):Also registered as a DI singleton at
backend/containers/service_container.py:115andconstructed at
:286-288. Verified present in a transitive-import closure computed over alldeployed entrypoints (
[project.scripts], rootDockerfileCMD,docker-compose.full.yml) —this check is why several superficially-attractive candidates in this series were dropped as
unreachable rather than shipped.
Agent handoff
@coderabbitai(see comment below for specific challenge questions).intelligent_cachetag fan-out).