flake: TestPubsubCluster/ClusterAuthRequired

[flake-investigator]

CI failure

• CI run: https://github.com/coder/coder/actions/runs/30976975074
• Failing job: https://github.com/coder/coder/actions/runs/30976975074/job/92212913393 (test-go-pg (windows-2022))
• Failure date: 2026-08-05 05:15 UTC; job completed at 05:21 UTC, before the 05:29 Slack alert
• Run attempt: 1
• Commit: 7a4ae2649eb592b961c142caf43dbe064a15a308
• Commit author: Jake Howell

The complete 4,462-line Windows job log was available and reviewed. The macOS matrix job was canceled only after the Windows failure; this is not a rerun/cancellation artifact.

Failure evidence

=== FAIL: coderd/x/nats TestPubsubCluster/ClusterAuthRequired (0.04s)
    pubsub_internal_test.go:519:
        Error: Target error should be in err chain:
               expected: "nats: authorization violation"
               in chain: "read tcp 127.0.0.1:59696->127.0.0.1:59631: wsarecv: An existing connection was forcibly closed by the remote host."
                         "wsarecv: An existing connection was forcibly closed by the remote host."
                         "An existing connection was forcibly closed by the remote host."
        Messages: route dial with wrong token must be rejected

The embedded NATS server started successfully and shut down cleanly. The wrong-token route connection was rejected as intended, but Windows surfaced the rejection as WSAECONNRESET instead of nats.ErrAuthorization.

Root cause assessment

Classification: flaky/OS-specific test assertion.

ClusterAuthRequired dials the NATS cluster route listener with a normal nats.go client and requires the returned error to wrap natsgo.ErrAuthorization. That assumes the route listener always writes a client-protocol -ERR 'Authorization Violation' that nats.go parses before closing the socket.

On Windows, the server can close/reset the rejected route connection before the client reads a parseable protocol error. Winsock then reports wsarecv: An existing connection was forcibly closed by the remote host. The security behavior is correct—the wrong token was rejected—but the exact error-class assertion is not portable/stable for this route-protocol misuse.

Possible fixes:

• treat a Windows connection reset as an accepted rejection outcome in this subtest;
• assert that no route forms / the connection closes rather than requiring one exact nats.go client error;
• or test route authentication through a route-protocol-aware mechanism/server callback instead of connecting a normal NATS client to the cluster port.

Race, panic, OOM, and resource checks

The full job log contains no Go race warning, panic, OOM, killed process, allocation failure, disk exhaustion, or file-descriptor exhaustion indicator. This is unrelated to the NATS shutdown panic tracked in #1631.

Assignment analysis

Ownership/history targets:

git blame -L 500,530 coderd/x/nats/pubsub_internal_test.go
git log --oneline -10 --follow coderd/x/nats/pubsub_internal_test.go

The exact ClusterAuthRequired subtest and its require.ErrorIs(..., natsgo.ErrAuthorization) assertions were introduced together in 167ac7b8 (feat: add nats experiment, PR coder/coder#25703) by sreya. Later changes modified other NATS tests or only increased cluster TLS timeout; they did not alter this assertion.

Assigning sreya based on exact test-line ownership, not the failing CI commit author.

Duplicate search

Searched open and closed coder/internal issues, including the last 30 days, for:

• TestPubsubCluster/ClusterAuthRequired and TestPubsubCluster;
• nats: authorization violation, wrong-token route auth, and pubsub_internal_test.go;
• An existing connection was forcibly closed by the remote host / wsarecv;
• NATS Windows failures, process crashes, panic/OOM/unknown failures, and race signatures.

No issue describes this test and failure mode.

Related but not duplicate:

• flake: Test process crash - NATS peer refresh after shutdown #1631 — NATS peer-refresh shutdown panic (send on closed channel), different code path/root cause.
• flake: TestSubscribeError/Flush #1617 — TestSubscribeError/Flush stale cleanup, different subtest/root cause.
• flake: TestClientAndConnectionError (windows error string mismatch) #1497 — same generic Winsock wording in an AI bridge test, but a different component and assertion.

Reproduction

go test ./coderd/x/nats -run '^TestPubsubCluster/ClusterAuthRequired$' -count=100

Run on Windows; full-suite socket churn may increase occurrence frequency.

[linear-code]

ENG-3134