[ENG-43320] Fix InputStream leak in PresignedUrlFileUploader + error-path tests - #192
Merged
Merged
Conversation
…path tests The small-file branch of PresignedUrlFileUploader#getRequest called IOUtils.toByteArray(fileStreamData.getInputStream()) without closing the stream. On both the happy path and the IOException catch, the InputStream leaked file descriptors / underlying HTTP connections (the large-file streaming branch already closed it correctly via try-with-resources). Fix: wrap the small-file read in try-with-resources so the InputStream closes on success and on IOException. Tests added to PresignedUrlFileUploaderTest: - close-verification on small-file happy path (Mockito spy) - close-verification when the read throws IOException - acceptable 4xx response (403) does not trigger retry Refactored @beforeeach to move the streamFileAsync stubbing into a per-test stubStreamFile() helper so error-path tests can supply their own InputStream without breaking strict-stubbing in the existing tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
niks002
approved these changes
Jun 15, 2026
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
The small-file branch of
PresignedUrlFileUploader#getRequestcalledIOUtils.toByteArray(fileStreamData.getInputStream())without closing the stream, leaking file descriptors / underlying HTTP connections from S3/GCS/Azure SDKs on every upload. The large-file streaming branch already does it right via try-with-resources — this PR aligns the small-file branch.Then adds the missing error-path tests (the existing class only covered happy, 5xx-failure, and large-file paths — 3 tests / 177 LOC).
Code change (
PresignedUrlFileUploader.java)Before:
After:
Tests added (
PresignedUrlFileUploaderTest.java)testUploadFileToPresignedUrl_smallFileClosesInputStreamOnSuccess— wraps the testInputStreamin a Mockito spy and assertsverify(inputStream).close()after a successful upload.testUploadFileToPresignedUrl_smallFileClosesInputStreamOnReadFailure— uses anInputStreamwhoseread(...)throwsIOException; asserts the wrappedFileUploadExceptionsurfaces with the originalIOExceptioncause AND thatclose()was still called.testUploadFileToPresignedUrl_acceptable4xxNotRetried—MockWebServerreturns 403 (acceptable failure code perACCEPTABLE_HTTP_FAILURE_STATUS_CODES); assertsgetRequestCount() == 1to prove retry was skipped even withmaxRetries=3, AND that the failure metric is emitted.Refactored
@BeforeEachto move thestreamFileAsyncstub into astubStreamFile(InputStream, long)helper so the new error-path tests can supply their ownInputStreamwithout tripping strict-stubbing on the existing 3 tests.Why this matters
Too many open filesif the leak compounds against the S3/GCS/Azure SDK's internal HTTP connection pool.ACCEPTABLE_HTTP_FAILURE_STATUS_CODESwould silently inflate API spend (3× requests for every expired/invalid pre-signed URL).Regression risk
Zero. The only behavior change in the production code is that the
InputStreamfrom the small-file path now closes after the byte buffer is materialised. Nothing else uses theInputStreamaftergetRequestreturns —IOUtils.toByteArrayhas already drained it into abyte[]that backs theRequestBody. The streaming large-file path is unchanged.Test plan
./gradlew :lakeview:test --tests PresignedUrlFileUploaderTest— all 6 tests green (3 existing + 3 new) locally../gradlew :lakeview:test --tests "ai.onehouse.storage.*"— full storage package green../gradlew :lakeview:spotlessCheck— green.Background code-flow doc for the metadata-extractor data-plane Push-Model job lives at
10XEngineer/docs/lakeview-metadata-extractor-flow.md(just merged).