Skip to content

ci(build): split allTests into parallel per-target jobs to end OOM flakes#854

Merged
michalharakal merged 1 commit into
developfrom
feature/split-alltests-parallel-jobs
Jul 21, 2026
Merged

ci(build): split allTests into parallel per-target jobs to end OOM flakes#854
michalharakal merged 1 commit into
developfrom
feature/split-alltests-parallel-jobs

Conversation

@michalharakal

@michalharakal michalharakal commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

The single build-job ran assemble allTests in one ubuntu-latest job. On a 7 GB runner the combined peak memory (Gradle JVM + Kotlin daemon + per-target test forks + Node for JS/Wasm + native toolchain, all at once) intermittently exceeded physical RAM. The runner was OOM-killed and GitHub reported "The operation was canceled" after ~15 min with no BUILD FAILED — the flaky ~20-minute red builds (seen on PRs #850, #852).

Changes:

  • Split the test run into a matrix of three legs (jvm / js-wasm / native), each running one target family so no single job holds the whole footprint. Legs run in parallel, so wall-clock is the slowest leg instead of the sum.
  • Run assemble as its own memory-light job, preserving the build/package guarantee for all targets.
  • Add Gradle dependency/wrapper caching to each job (same pattern as docs.yml) so parallel legs don't cold-download the dependency graph.
  • Capture the memory snapshot on cancelled() as well as failure() — an OOM-killed runner reports cancelled, not failed, so the existing diagnostic never fired on the runs that needed it.
  • Keep an aggregate build-job gate with the same name so the existing required status check keeps working without re-pointing branch protection.

No production code changed; allTests passes locally.

…akes

The single build-job ran `assemble allTests` in one ubuntu-latest job.
On a 7 GB runner the combined peak memory (Gradle JVM + Kotlin daemon +
per-target test forks + Node for JS/Wasm + native toolchain, all at
once) intermittently exceeded physical RAM. The runner was OOM-killed
and GitHub reported "The operation was canceled" after ~15 min with no
BUILD FAILED — the flaky ~20-minute red builds (seen on PRs #850, #852).

Changes:
- Split the test run into a matrix of three legs (jvm / js-wasm /
  native), each running one target family so no single job holds the
  whole footprint. Legs run in parallel, so wall-clock is the slowest
  leg instead of the sum.
- Run `assemble` as its own memory-light job, preserving the
  build/package guarantee for all targets.
- Add Gradle dependency/wrapper caching to each job (same pattern as
  docs.yml) so parallel legs don't cold-download the dependency graph.
- Capture the memory snapshot on `cancelled()` as well as `failure()`
  — an OOM-killed runner reports cancelled, not failed, so the existing
  diagnostic never fired on the runs that needed it.
- Keep an aggregate `build-job` gate with the same name so the existing
  required status check keeps working without re-pointing branch
  protection.

No production code changed; allTests passes locally.
@michalharakal michalharakal changed the title ci(build): split allTests into parallel per-target jobs to end OOM fl… ci(build): split allTests into parallel per-target jobs to end OOM flakes Jul 21, 2026
@michalharakal
michalharakal requested review from MacOS and aharakal July 21, 2026 16:30
@michalharakal
michalharakal removed the request for review from MacOS July 21, 2026 17:27
@michalharakal
michalharakal merged commit b943b63 into develop Jul 21, 2026
21 checks passed
@michalharakal
michalharakal deleted the feature/split-alltests-parallel-jobs branch July 21, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants