fix: SSE streaming stall on MAUI Android (SDK-2755) - #124
Merged
Conversation
Adds a small HTTP service that wraps LaunchDarkly.EventSource and exposes it to the sse-contract-tests harness. The .NET SSE library was the only launchdarkly/*-eventsource repo without a contract-tests service; the Go, JavaScript, Java (okhttp), and C++ SSE libraries each have one. Adding one here lets us participate in cross-language SSE conformance testing and gives us a target to point the harness at when validating fixes. Structure and endpoints mirror the Java service in okhttp-eventsource's contract-tests. The main structural difference: .NET's EventSource is push-based (fires event handlers on its own internal thread) rather than pull-based via a blocking iterator, so StreamEntity subscribes handlers directly instead of spinning up a background reader thread. The service is not shipped as part of the LaunchDarkly.EventSource NuGet package -- it lives in contract-tests/ as internal tooling only.
…g stall When LaunchDarkly.EventSource runs on MAUI Android, the response body Stream returned by HttpClient is silently wrapped in a System.IO.BufferedStream by Xamarin.Android.Net.AndroidMessageHandler (see dotnet/android's AndroidMessageHandler.GetContent). BufferedStream's default internal buffer is 4096 bytes, and its Read implementation takes a different code path when the caller's requested count is smaller than that internal buffer -- it attempts to fill its own 4096-byte buffer by issuing a 4096-byte read against the underlying Java InputStream. That aggressive prefetch drains the underlying stream on the first read. When the caller comes back for more bytes, BufferedStream must issue another read, which then blocks on Java's InputStream.read waiting for the next byte from the socket. On a LaunchDarkly SSE stream, the next byte doesn't arrive until the next server-side keepalive (~60 seconds), stalling the entire client for that duration. The bug reproduces for payload sizes in a specific window between the caller's buffer size and BufferedStream's internal size -- large enough to force a second read but small enough that the whole payload arrives in one TCP burst before the socket goes idle. Empirically that window was 1 KiB through ~4 KiB with the default 1024-byte StreamReader buffer. Payloads above ~4 KiB span enough TCP frames that the underlying stream never goes idle between reads and the bug doesn't manifest. Setting our read buffer to 8192 bytes takes the caller's Read count to a value >= BufferedStream._bufferSize (4096), which routes into BufferedStream's passthrough branch instead of its prefetching branch -- each caller Read maps 1:1 to a single Java read, with no over-drain. 8192 matches Okio's Segment.SIZE (which Android's OkHttp uses internally) and gives one doubling of headroom in case dotnet/runtime raises BufferedStream's default in a future release. Both the StreamReader (default text-mode path) and the ByteArrayLineScanner (PreferDataAsUtf8Bytes path) needed to be updated, since both were using sub-4096 buffers (1024 and 1000 respectively). Fixes SDK-2755.
Systematically ran the sse-contract-tests harness against the .NET test
service and trimmed / retained capabilities based on actual pass/fail
rather than a copy-paste from other language SDKs.
Removed:
* "comments" -- CommentReceived fires with the raw line including the
leading colon (see EventParser.cs). Harness expects the colon stripped.
Real .NET EventSource behavior gap; not fixed in this PR.
* Corresponding CommentReceived subscription in StreamEntity so we don't
leak comment callbacks into tests where the capability is not declared.
Kept (each verified end-to-end against the harness):
bom, headers, last-event-id, payload-size-stress-testable, post,
read-timeout, report, restart.
Also documented in a comment block on the Capabilities array the two
other harness capabilities intentionally omitted:
* "event-type-listeners" -- N/A, MessageReceived fires for all types
* "server-directed-shutdown-request" -- .NET retries on 204 instead
of halting; see EventSource.cs while-loop condition on Shutdown.
Post-audit vs Java (okhttp-eventsource): .NET declares 8 capabilities
including "bom" which Java does not; .NET is missing "comments" due to
the real behavioral gap noted above. Net coverage is broader than Java,
not narrower.
tanderson-ld
marked this pull request as ready for review
August 6, 2026 14:25
Wires the contract-tests service added earlier in this PR into CI so we actually validate LaunchDarkly.EventSource's SSE behavior against the sse-contract-tests harness on every push. Without this step the test service was inert -- the code existed but nothing ran the harness against it. Follows the pattern used by launchdarkly/eventsource (Go) and launchdarkly/okhttp-eventsource (Java): a Makefile at repo root with build-contract-tests / start-contract-test-service-bg / contract-tests targets, plus a GitHub Actions job that uses the shared launchdarkly/gh-actions/actions/contract-tests action to download and run a released harness (VERSION=v2, currently resolves to v2.31.0). Once the payload-size-stress-testable capability (defined in sse-contract-tests PR #42) lands in a released v2.x, this job will automatically pick up the sweep on its next run -- no additional CI changes needed. The extra_params skips four pre-existing dotnet-eventsource behavioral gaps unrelated to SDK-2755: null-byte ID handling, CR-only line terminator parsing, CRLF-split-across-chunks handling, and partial-message ID carry-over on reconnect. Filed a follow-up ticket to survey and fix these; once fixed, remove the corresponding --skip.
The Content-Type key was being skipped from the request-header loop with a case-insensitive OrdinalIgnoreCase match, but retrieved for body construction with a case-sensitive Dictionary lookup. Practical impact is nil because the sse-contract-tests spec guarantees lowercase header keys, but the inconsistency is a code smell and would silently fall back to text/plain if the harness ever sent Content-Type with any capitals. Replace the case-sensitive TryGetValue with the same case-insensitive iteration pattern used for skipping, so both operations agree on what counts as the Content-Type key.
jsonbailey
reviewed
Aug 6, 2026
jsonbailey
reviewed
Aug 6, 2026
jsonbailey
reviewed
Aug 6, 2026
jsonbailey
reviewed
Aug 6, 2026
jsonbailey
approved these changes
Aug 6, 2026
jsonbailey
left a comment
Contributor
There was a problem hiding this comment.
Mainly nits on comments. I didn't mark all of them but all the "this is analogous..." comments can likely be removed.
Moves Program.Main into its own file so the shared Webapp class (HTTP routing, capability declaration, StreamEntity management) can be hosted by both the existing desktop test service and the forthcoming Android test service. Behavior is unchanged; Webapp itself is untouched apart from a doc-comment refresh. The Android target added in the follow-up commit references TestService.cs, Representations.cs, and StreamEntity.cs by <Compile Include Link>, so having the desktop entry point in a separate Program.cs prevents the Android target from picking up a conflicting Main.
Adds a .NET for Android build of the SSE contract-tests service and a
new GitHub Actions job that runs the sse-contract-tests harness against
it on an emulator. Fills a validation gap the desktop Contract Tests
job cannot cover: the SDK-2755 fix defends against a bug specific to
Xamarin.Android.Net.AndroidMessageHandler's BufferedStream wrap, which
sits below the response-body Stream only on Android; desktop uses
SocketsHttpHandler and cannot reproduce that code path.
Structure follows launchdarkly/android-client-sdk's Android CI pattern:
- Fragile adb glue (install / launch / port-forward / wait) lives in
scripts/start-android-test-service.sh so the workflow's script:
block stays a two-line call to make.
- Emulator-runner options (google_apis target, KVM-friendly emulator
flags, disable-animations, SHA-pinned action) copied from that repo.
- runs-on: ubuntu-latest because GH Linux runners have /dev/kvm
available for hardware acceleration; macOS mixed Intel/Apple-Silicon
fleets would run x86_64 emulator images under slower nested
virtualization.
Emulator is booted with -memory 8192; the sweep exercises payloads up
to 128 MiB which peak at ~500 MB of managed heap through UTF-16
decoding and JSON serialization, exceeding the 2 GB default emulator
RAM. android:largeHeap=true in the manifest raises the per-app dalvik
heap ceiling from ~192 MB to ~512 MB to accommodate this.
Android-only test skips live in contract-tests-android/testharness-suppressions.txt,
read by the Makefile via a plain while-loop into inline -skip arguments
(the harness does not yet support -skip-from). The file records two
categories of skip: the pre-existing LaunchDarkly.EventSource behavior
gaps also skipped in the desktop job, and two Android-specific ones --
REPORT method (Java HttpURLConnection whitelist rejects it) and one
reconnection test hitting a JNI ref-counting bug in
Android.Runtime.InputStreamInvoker.Dispose. Both are documented in the
PR description; neither is fixable inside LaunchDarkly.EventSource.
Locally validated end-to-end against my Android emulator: all 80
payload-sweep sizes pass, 6 tests skipped per the suppressions file,
1 auto-skipped for the un-declared "comments" capability, zero
failures.
Co-authored-by: Jason Bailey <accounts@sidewaysgravity.com>
Co-authored-by: Jason Bailey <accounts@sidewaysgravity.com>
Co-authored-by: Jason Bailey <accounts@sidewaysgravity.com>
Trim the 11-line rationale to 4. The essentials are: (a) must be >= 4096, (b) reason is AndroidMessageHandler's BufferedStream wrap, (c) failure mode is ~60 s stalls, (d) SDK-2755 for context. Anyone wanting the full mechanism has the ticket link.
Removes comments that cited other LaunchDarkly repos (android-client-sdk, okhttp-eventsource) as inspiration or precedent. Cross-repo provenance is noise in the code and can rot if the referenced patterns change or disappear upstream. The parallels are still there for anyone who looks for them; the code should stand on its own.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c238772. Configure here.
Two issues from the first CI runs: 1. avdmanager create avd fails on ubuntu-latest with "cannot create /home/runner/.android/avd/test.avd/config.ini: Directory nonexistent" because ~/.android/avd doesn't exist on a fresh runner. Add a mkdir -p step before the emulator-runner. 2. The failure-only 'adb logcat -d' step hangs indefinitely because adb-server has no device to talk to after the emulator-runner tears down its emulator. Move log capture inside the runner's script block (background adb logcat to a file); the file persists on the runner filesystem after emulator teardown and can be uploaded by upload-artifact without touching adb again.
The test sends a 5 MiB + 5 MiB payload and uses the harness's default 5-second RequireEvent timeout (not the size-scaled RequireEventWithin used by the payload-size sweep). At 10 MiB combined, GH Actions Android emulator's adb-tunneled throughput doesn't finish the round-trip within 5s, so a correct implementation times out. The single-chunk 5 MiB variant passes on the same runner, and the payload-size sweep covers up to 128 MiB independently with its own timeout formula. Losing this specific test on Android doesn't meaningfully reduce coverage.
kinyoklion
approved these changes
Aug 7, 2026
tanderson-ld
added a commit
that referenced
this pull request
Aug 10, 2026
🤖 I have created a release *beep* *boop* --- ## [5.3.2](5.3.1...5.3.2) (2026-08-10) ### Bug Fixes * increase SSE read buffer to 8192 bytes to avoid Android streaming stall ([cdd255a](cdd255a)) * SSE streaming stall on MAUI Android (SDK-2755) ([#124](#124)) ([3989eca](3989eca)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Release 5.3.2** via Release Please: bumps the package version from **5.3.1** to **5.3.2** in `.release-please-manifest.json`, `LaunchDarkly.EventSource.csproj`, and `PROVENANCE.md`, and adds the **5.3.2** section to `CHANGELOG.md`. > > The changelog records the shipped fix for **SSE streaming stalls on MAUI Android** (SDK-2755): raising the SSE read buffer to **8192** bytes so reads avoid Xamarin Android’s small-buffer path that could block for ~60s between keepalives. This PR’s diff is version and release docs only; the buffer change lives in the commits referenced by the changelog. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 195abcf. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
tanderson-ld
added a commit
to launchdarkly/dotnet-core
that referenced
this pull request
Aug 10, 2026
…all (SDK-2755) (#328) **Related issues** - Fix PR in dotnet-eventsource: launchdarkly/dotnet-eventsource#124 - 5.3.2 release notes: https://github.com/launchdarkly/dotnet-eventsource/releases/tag/5.3.2 **Describe the solution you've provided** Bumps `LaunchDarkly.EventSource` from 5.3.1 to 5.3.2 in `LaunchDarkly.ClientSdk`. The 5.3.2 release increases the SSE read buffer from 1024 to 8192 bytes to bypass a stall in `Xamarin.Android.Net.AndroidMessageHandler`'s BufferedStream small-count code path, which caused MAUI Android consumers' SSE reads to block for ~60 s waiting for the next server keepalive. The fix was validated end-to-end with a new sse-contract-tests payload-size sweep run against a .NET-for-Android build of the SSE test service in an Android emulator; the sweep flags every regression to a sub-4096-byte buffer. **Describe alternatives you've considered** - Working around the stall in-repo (e.g., swapping out `AndroidMessageHandler` for a different HTTP handler) was rejected as riskier than adjusting the buffer size at the SSE layer. - The `ServerSdk` consumer of `LaunchDarkly.EventSource` is out of scope for this PR: the stall is Android-only, so bumping `ServerSdk` would be pure hygiene rather than a fix. **Additional context** - No API changes; single-line `PackageReference` version bump. - Companion to previously merged #323, which addressed the network-connectivity side of the same customer-reported symptom. - Verified locally that `LaunchDarkly.ClientSdk` builds cleanly against 5.3.2 for both `netstandard2.0` and `net8.0-android`. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Bumps** `LaunchDarkly.EventSource` from **5.3.1** to **5.3.2** in `LaunchDarkly.ClientSdk` only (no API or source changes). > > The newer EventSource release increases the SSE read buffer (1024 → 8192 bytes) so MAUI Android clients avoid long (~60s) stalls when `AndroidMessageHandler`’s buffered stream blocks on small reads until the next keepalive. `LaunchDarkly.ServerSdk` is intentionally left on 5.3.1 because the issue is Android-specific. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit b73204d. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Fixes a 60–90 second streaming stall observed by a LaunchDarkly customer running the .NET client SDK on MAUI Android. Root cause is a buffer-size interaction between LaunchDarkly.EventSource's
StreamReader(default 1024-byte buffer) and Xamarin.Android.Net'sAndroidMessageHandler, which silently wraps HTTP response bodies in aSystem.IO.BufferedStreamwith a 4096-byte default. Payload sizes in the ~1 KiB – ~4 KiB range trigger a code path whereBufferedStreamover-drains the underlying JavaInputStream, causing the subsequent read to block waiting for the next server-side SSE keepalive (~60 s later).Also adds the missing
contract-tests/service so LaunchDarkly.EventSource can participate in thesse-contract-testsharness alongside the Go, JavaScript, Java, and C++ SSE libraries.Commits
chore: add SSE contract-tests service— Addscontract-tests/(TestService.csproj, TestService.cs, StreamEntity.cs, Representations.cs, README.md). Mirrors the structure ofokhttp-eventsource's Java contract-tests service. UsesLaunchDarkly.TestHelpers.HttpTest(idiomatic .NET version of Java'stest-helpers). Not shipped in the NuGet package. Declares thepayload-size-stress-testablecapability so this repo participates in the new sweep from feat: add payload-size-stress-testable capability and test sweep sse-contract-tests#42.fix: increase SSE read buffer to 8192 bytes...— BumpsEventSourceService'sStreamReaderbuffer andByteArrayLineScannerbuffer from their previous 1024 / 1000 to a shared 8192. 8192 matches Okio'sSegment.SIZEand gives one doubling of headroom overBufferedStream's current 4096-byte default. Both text-mode andPreferDataAsUtf8Bytespaths are covered.Root cause detail
Full mechanism:
BufferedStream.Read(dotnet/runtime) withcount < _bufferSizetakes a code path that fills its internal 4096-byte buffer via a single_stream.Read(_buffer, 0, _bufferSize)call. On MAUI Android this call goes intoAndroidMessageHandler'sInputStreamInvoker, which is a thin passthrough to JavaHttpURLConnection.getInputStream().read(...). When the whole first SSE HTTP chunk arrives in one TCP burst and fits inside 4096 bytes,BufferedStreamdrains it entirely on the first user Read. The next user Read triggers a second Javaread(), which blocks on the socket. Since LaunchDarkly's SSE server only sends the next chunk (a:keepalive) at ~60-second intervals, that second read stalls the whole client for 60–90 seconds.Setting the SSE reader's buffer to
>=BufferedStream._bufferSizeroutes intoBufferedStream's passthrough branch instead, so each user Read maps 1:1 to a single Java read with no over-drain. Empirically confirmed with a reproduction repo that swept payload sizes 510 B – 33 KiB across buffer configurations of 1 KiB, 4 KiB, 8 KiB, and 32 KiB, plus regression testing showing the 1 KiB reader stalls at 4/8 sub-4 KiB sizes and the 8 KiB reader passes all sizes.Test plan
LaunchDarkly.EventSource.Testsall pass (131 tests, 0 failures)dotnet build)GET /status requestsse-contract-testsharnessAndroidMessageHandleron Android emulator (deferred — requires the automated Android CI runner, tracked separately)Related
payload-size-stress-testablecapability + sweep test suite that this PR opts into)LaunchDarkly.ClientSdkwill get the fix via a dependency version bump indotnet-core(separate PR against SDK-2755)Note
Medium Risk
Core streaming read path changes affect all platforms; mitigated by contract tests and payload-size sweep, but Android-specific behavior is the main regression surface.
Overview
Fixes MAUI Android SSE stalls (~60–90s) by raising
EventSourceServiceread buffers from ~1 KiB to 8192 bytes on both theStreamReaderandByteArrayLineScannerpaths, avoidingAndroidMessageHandler’s 4096-byteBufferedStreamover-read behavior (SDK-2755).Adds SSE contract-test infrastructure: a desktop
contract-tests/HTTP wrapper aroundLaunchDarkly.EventSourcefor the sharedsse-contract-testsharness, plus an Android variant (contract-tests-android/,MainActivity, sharedTestServicelogic) with Makefile targets andscripts/start-android-test-service.sh. CI gains desktop contract tests (with documented skips for known gaps) and an emulator job that runs the harness against the Android build to catch regressions desktop cannot.Reviewed by Cursor Bugbot for commit 6f57f61. Bugbot is set up for automated code reviews on this repo. Configure here.