Skip to content

⚡️ Fix FormData.readAsBytes excessive memory usage with large payloads#2544

Merged
AlexV525 merged 3 commits into
mainfrom
alexv525-fix-form-data-read-as-bytes-memory
Jun 29, 2026
Merged

⚡️ Fix FormData.readAsBytes excessive memory usage with large payloads#2544
AlexV525 merged 3 commits into
mainfrom
alexv525-fix-form-data-read-as-bytes-memory

Conversation

@AlexV525

Copy link
Copy Markdown
Member

The previous implementation used reduce((a, b) => Uint8List.fromList([...a, ...b])), which re-allocates and copies all accumulated bytes on every chunk — O(n²) memory in total.

Since FormData.length is already computed before finalization, we can pre-allocate the exact result buffer and write each chunk in-place with setRange, bringing memory usage down to O(n) with a single allocation.

Related: #2506

Replace the O(n²) `reduce`+spread approach with a single pre-allocated
`Uint8List` and `setRange` writes. Since `FormData.length` is already
known before finalization, we allocate the exact result buffer upfront and
stream each chunk directly into it — no intermediate copies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@AlexV525 AlexV525 requested a review from a team as a code owner June 27, 2026 02:17
AlexV525 and others added 2 commits June 27, 2026 11:26
@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage Report: Only Changed Files listed

Package Base Coverage New Coverage Difference
Overall Coverage 🟢 85.54% 🟢 85.55% 🟢 0.01%

Minimum allowed coverage is 0%, this run produced 85.55%

@CaiJingLong

Copy link
Copy Markdown
Contributor

PR Review: Fix FormData.readAsBytes excessive memory usage with large payloads

Risk: LOW

Changes Summary

Correctness

The core fix is correct. The previous reduce((a, b) => Uint8List.fromList([...a, ...b])) re-allocated and copied all accumulated bytes on every chunk, giving O(n²) total copies for n chunks. The new implementation pre-allocates the result buffer using FormData.length and writes each chunk in place via setRange, giving O(n) total copies with a single allocation.

Behavioral equivalence is preserved:

  • finalize() is still invoked synchronously before the first await, so isFinalized semantics are unchanged — the existing expect(fm.readAsBytes(), throwsA(StateError)) tests still pass.
  • Stream error propagation is unchanged (await for rethrows stream errors).
  • The Future.sync(() => ...)async conversion preserves the semantics that synchronous exceptions (e.g. already-finalized StateError) become a rejected Future rather than a thrown error.

Notes

1. Behavior when length is inconsistent (non-blocking)

The new implementation relies on FormData.length matching the actual byte count emitted by finalize(). length is computed from MultipartFile.length, which for fromStream is caller-supplied. If the supplied length is wrong:

Case Old behavior New behavior
Stream > length Silently oversized buffer setRange throws RangeError (fail-fast)
Stream < length Silently undersized buffer Trailing bytes left as zero

The "stream > length" case is a fail-fast improvement — the old code silently produced corrupted data. The "stream < length" case still fails silently (zero padding). Optional suggestion: add assert(offset == result.length) before return to catch length mismatches in debug mode. Not required, but worth considering.

2. Test helper _indexOfSubList

Hand-rolled sublist search — Dart's stdlib has no direct equivalent, so this is acceptable. Since headers/boundaries are ASCII, utf8.decode + String.indexOf would also work, but the current approach is more robust for mixed binary content. Fine as-is.

3. Test coverage

The new test precisely targets the O(n²) scenario — a chunked stream upload. Assertions are thorough: total length, header location, payload integrity, and trailing boundary. The expectedPayload uses (i ~/ chunkSize) % 256 to produce distinguishable per-chunk content, which would catch chunk reordering.

Missing Coverage

  • No readAsBytes test for an empty FormData (boundary-only) — existing tests cover field-only scenarios, low risk.
  • No readAsBytes test for stream error propagation — pre-existing gap, can be addressed separately.

Recommendation

APPROVE. The fix is correct, well-tested, and low-risk. Optional nit: add assert(offset == result.length) to surface length mismatches in debug builds.


⚠️ This review was authored with AI assistance. The reviewer (a human) directed the analysis and validated the findings.

Agent: Devin (Cognition) — interactive command-line agent
Model: GLM-5.2 High

@AlexV525 AlexV525 merged commit 0940678 into main Jun 29, 2026
5 checks passed
@AlexV525 AlexV525 deleted the alexv525-fix-form-data-read-as-bytes-memory branch June 29, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants