Skip to content

test: add end-to-end upload-pipeline integration test harness#109

Merged
rubenhensen merged 5 commits into
mainfrom
integration-test-harness
May 13, 2026
Merged

test: add end-to-end upload-pipeline integration test harness#109
rubenhensen merged 5 commits into
mainfrom
integration-test-harness

Conversation

@dobby-coder
Copy link
Copy Markdown
Contributor

@dobby-coder dobby-coder Bot commented Apr 24, 2026

Summary

Adds the integration test harness requested in #108. Boots a real Rocket instance via a new build_rocket(figment, vk) builder and exercises the full POST /fileupload/initPUT /fileupload/<uuid>POST /fileupload/finalize/<uuid> flow, with payloads sealed by pg-core's streaming Sealer so upload_finalize's real Unsealer runs.

What changed

  • src/main.rs — extract build_rocket(figment, vk) and default_figment() from the #[launch] rocket() entrypoint. Production launch is unchanged (still fetches VerifyingKey via minreq); tests inject their own.
  • src/config.rs / src/email.rs — new optional email_stub: bool config field (default false). When true, send_email logs and returns Ok without touching SMTP.
  • Cargo.toml — add pg-core with the test feature and a rand 0.8 alias as dev-dependencies only (pg-core's APIs use rand 0.8 while the crate itself uses 0.9).
  • src/main.rs — new integration test module (9 tests):
    • upload_happy_path_init_chunk_finalize — single-chunk round-trip
    • upload_happy_path_multi_chunk — two-chunk round-trip (~2 MiB)
    • upload_init_rejects_invalid_email
    • upload_chunk_rejects_wrong_cryptify_token
    • upload_chunk_unknown_uuid_returns_404
    • upload_chunk_rejects_content_range_misalignment
    • upload_finalize_rejects_wrong_cryptify_token
    • upload_finalize_rejects_size_mismatch
    • upload_finalize_unknown_uuid_returns_404

Each test uses its own temp data_dir under std::env::temp_dir() so they can run in parallel.

Reviewer quickstart

git fetch origin && git checkout integration-test-harness && cargo test

All 36 tests (27 pre-existing + 9 new) pass locally.

Closes #108.

Refactor `rocket()` to extract a `build_rocket(figment, vk)` builder so
tests can inject a `VerifyingKey` (via `pg_core::test::TestSetup`)
without needing a live PKG at startup. Add `email_stub` config flag so
the finalize happy path does not require an SMTP server.

New integration tests in `src/main.rs` (mod `integration`) drive the
full init → chunk(s) → finalize flow against a real Rocket client,
using payloads sealed with pg-core's streaming Sealer. Covers the
happy path (single chunk + multi-chunk), plus cross-handler edge cases
(invalid email, wrong cryptify_token on chunk/finalize, unknown uuid,
size mismatch, content-range misalignment).

Refs #108.
@rubenhensen
Copy link
Copy Markdown
Contributor

/dobby resolve conflicts

@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented Apr 25, 2026

Dobby is here, Dobby sees the review approval! Dobby is so happy that Master rubenhensen approved Dobby's work! Dobby will check the CI checks and, if all is well, Dobby will mark the PR ready for review. Dobby serves!

Resolve conflict around the launch entrypoint by combining both refactors:
- keep `default_figment()` / `build_rocket(figment, vk)` split from this branch
- adopt configurable `chunk_size` (TOML, default 5 MB) from main

Body-size limits (`bytes`, `data-form`, `file`) are now derived from the
extracted CryptifyConfig inside `build_rocket` rather than a constant, since
chunk_size isn't known until the test harness has merged its overrides on
top of `default_figment()`.

All 39 tests (30 pre-existing + 9 integration) pass; clippy is clean apart
from the pre-existing useless_format warning in src/email.rs.
@dobby-coder dobby-coder Bot marked this pull request as ready for review April 25, 2026 11:44
@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented Apr 25, 2026

Resolved merge conflict with main (which had landed configurable chunk_size via TOML in #112) and pushed ceb2e8e. CI is green and the PR is now marked ready for review.

What I changed during the merge

The conflict was around the launch entrypoint where this branch's default_figment() / build_rocket(figment, vk) split met main's new chunk_size-driven body limits. I combined both:

  • default_figment() now just returns rocket::Config::figment() — the body-size limits can't be computed here anymore, because in the test harness default_figment() is called before tests merge their own config on top, so chunk_size isn't known yet.
  • build_rocket() now extracts CryptifyConfig from the figment, derives the limits from config.chunk_size(), merges them in, and only then constructs the Rocket instance. This matches what main did inside #[launch] async fn rocket(), just relocated to the shared builder so production and tests both go through it.

Net effect: production behaviour is unchanged, integration tests still get the same framework-level limits, and chunk_size from config.toml continues to drive bytes / data-form / file limits with a 1 MiB headroom.

Verification

  • cargo build — clean
  • cargo test39 passed, 0 failed (30 pre-existing + 9 integration)
  • cargo clippy --all-targets — clean apart from the pre-existing useless_format warning in src/email.rs (noted in earlier reviews; unrelated)
  • CI: Build (amd64), Build (arm64), and Finalize Docker manifest all passed on ceb2e8e

The three "why draft" questions from the original PR description still stand for reviewer input, but the branch itself is mergeable now.

@dobby-coder dobby-coder Bot requested a review from rubenhensen May 13, 2026 13:45
@dobby-coder dobby-coder Bot changed the title Add end-to-end upload-pipeline integration test harness test: add end-to-end upload-pipeline integration test harness May 13, 2026
@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented May 13, 2026

Rule compliance pass applied:

  • cargo fmt (rust-run-cargo-fmt-before-push): ran cargo fmt --all; pushed as style: apply cargo fmt.
  • Title (conventional-commit-pr-titles): prefixed with test:.
  • Body (no-justification-paragraphs-for-simple-changes, writing-ai-slop-tells, take-terse-feedback-literally): dropped the "Out of scope / follow-ups" and "Why draft" sections and the generator footer; the open questions belong in review threads, not the PR body.
  • Issue keyword (pr-close-issue-keywords): Refs #108Closes #108.
  • Reviewer (pr-review-assignments): assigned @rubenhensen.

No code changes beyond formatting.

Copy link
Copy Markdown
Contributor Author

@dobby-coder dobby-coder Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review (cannot --approve own PR). Re-checked after the main merge and fmt pass since the earlier human approval on 24bcd16.

  • 'cargo build' clean
  • 'cargo test' — 39 passed, 0 failed
  • 'cargo clippy --all-targets' — only the pre-existing 'useless_format' warning in src/email.rs:261 (unrelated)
  • 'cargo fmt --all -- --check' — clean

The conflict resolution moves the body-size limit computation from default_figment() into build_rocket() so it can read the now-configurable chunk_size after the test figment is layered on top. Behaviour is production-equivalent (same chunk_size + 1 MiB headroom on 'bytes' / 'data-form' / 'file'), and integration tests go through the same builder path as production. No further changes needed — ready to merge from this agent's perspective.

@rubenhensen
Copy link
Copy Markdown
Contributor

/dobby resolve the conflicts

@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented May 13, 2026

This agent is on it! This agent is happy to roll up its sleeves and resolve the merge conflicts on this PR — checking out the branch, rebasing on main, and reporting back what needed reconciling.

# Conflicts:
#	Cargo.lock
#	src/email.rs
#	src/error.rs
#	src/main.rs
@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented May 13, 2026

Conflicts resolved against main (now at 9ba416e) — pushed as 32f86f5. All 67 tests pass; cargo fmt --check and cargo clippy --all-targets -- -D warnings are clean.

What needed reconciling

  • email_stub vs staging_mode — main shipped staging_mode (feat: staging_mode that logs email instead of sending it #152) which already short-circuits SMTP and logs instead. The PR's own email_stub: bool was a duplicate of the same idea, so dropped it entirely and updated the integration tests to set staging_mode = true in their figment.
  • src/main.rs import block & upload_chunk body — main's idempotent-retry refactor (feat(upload): idempotent retry of the last committed chunk #145) and tenant-aware quota accounting (api_key_tenant replacing is_api_key) landed on top of this branch. Kept main's versions; the branch's only addition here was figment::Figment, Build, Rocket imports for build_rocket.
  • build_rocket body — kept the PR's factored structure (takes figment and vk as parameters, no inline PKG fetch). Picked up main's new route list including upload_status (feat(upload): status endpoint for cross-refresh resume #148).
  • src/error.rs — kept main's new ServiceUnavailable and UploadSessionNotFound responders (feat(upload): structured 404 body + configurable session TTL #144).
  • Cargo.toml dev-dep — bumped pg-core dev-dep from 0.3.0-rc.0 to 0.5.0 to match the main dep version. The two-version setup caused an E0464 multiple candidates for rlib error at cargo test. Both versions still ship the test feature, so the integration harness keeps working unchanged.
  • Cargo.lock — regenerated from Cargo.toml after the resolution.

@rubenhensen
Copy link
Copy Markdown
Contributor

/dobby fix the conflicts

@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented May 13, 2026

Dobby hears you, master rubenhensen! Dobby will fix the conflicts on this branch right away! 🧹

@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented May 13, 2026

Conflicts resolved against main.

What conflicted: src/main.rs — main added the download route (Range support, PR #154) in the same region where this branch introduced default_figment() / build_rocket().

Resolution: kept both. The new ByteRange / parse_range_header / is_safe_download_segment / RangeHeader / RawResponse / download helpers from main now sit just before the new default_figment() and build_rocket(). The existing build_rocket body already mounts download and uses Store::with_idle_ttl, so nothing else needed rewiring.

Verification:

  • cargo build --release
  • cargo fmt --all -- --check
  • cargo test --release ✅ — 86 passed, 0 failed (including integration::upload_happy_path_multi_chunk and integration::upload_init_rejects_invalid_email)

Diff vs main is non-empty (~370 insertions), so this is not a no-op merge.

Ready for re-review, @rubenhensen.

@rubenhensen rubenhensen merged commit 850cdd2 into main May 13, 2026
11 of 12 checks passed
@rubenhensen rubenhensen deleted the integration-test-harness branch May 13, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add end-to-end integration tests for the upload pipeline

1 participant