Skip to content

fix(release): link the three binaries one at a time in binaries.yml - #15

Merged
vyncint merged 1 commit into
mainfrom
fix/release-binaries-link-serially
Sep 7, 2026
Merged

fix(release): link the three binaries one at a time in binaries.yml#15
vyncint merged 1 commit into
mainfrom
fix/release-binaries-link-serially

Conversation

@vyncint

@vyncint vyncint commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Today

Every Linux binary job of the last two releases died at the same point: linking oxidelake-runtime after all 444 dependencies had compiled, with The runner has received a shutdown signal, exit 143.

Release Job Died after Last crate in flight
v0.1.0 aarch64-unknown-linux-musl 9m16s oxidelake-runtime
v0.1.0 x86_64-unknown-linux-musl 13m46s oxidelake-runtime
v0.1.2 aarch64-unknown-linux-musl 8m55s oxidelake-runtime

v0.1.0 shipped with macOS archives only; its macOS jobs took 4h11m and 4h37m.

Why

The release profile is lto = "fat", codegen-units = 1. Each binary's link is a whole-program LLVM pass over Arrow, DataFusion and Ballista, and Cargo schedules the three binaries' links in parallel. Three of those exceed a 16 GB Linux runner with no swap, so the VM is torn down. macOS swaps instead, slowly. cargo install from crates.io passes because a crate built outside the workspace does not carry the workspace release profile.

Change

  • Build with one cargo build --release --locked -p oxidelake-runtime --bin <name> --target <t> per binary. Dependencies compile in parallel during the first call; the other two only link, so one LTO pass is in memory at a time.
  • A re-run against an existing tag checks the release first and skips targets whose archive is already attached. docs/RELEASING.md already documents gh workflow run binaries.yml --ref main -f tag=vX.Y.Z for this; it now costs nothing for targets that are done.
  • CHANGELOG under [Unreleased].

Verified

  • make zizmor (pedantic): no findings. YAML parses.
  • The skip step dry-run locally with the real gh against v0.1.2/aarch64 musl (absent → build), v0.1.0/aarch64 darwin (present → skip), and a nonexistent tag (build).

Not done

  • No automated test covers the workflow shell. The link change is verified by dispatching binaries.yml at main for v0.1.2 after this merges; the Linux archives are the evidence.
  • The release profile itself is unchanged, per the audit's note not to alter production optimisation for CI's sake. The macOS runners will still swap through a single LTO link, just one at a time.

Every Linux binary job of the last two releases died at the same point,
linking oxidelake-runtime after all 444 dependencies had compiled, with
"The runner has received a shutdown signal" and exit code 143: v0.1.0
(both musl targets) and v0.1.2 (aarch64 musl). The release profile is
fat LTO with a single codegen unit, so each binary's link is a
whole-program LLVM pass over Arrow, DataFusion and Ballista, and Cargo
schedules the three binaries' links in parallel. Three of those exceed
a 16 GB Linux runner with no swap and the VM is torn down. macOS
survives by swapping, which is why its jobs took four hours on v0.1.0.
`cargo install` from crates.io is unaffected because a crate built
outside the workspace does not carry the workspace release profile,
which is why the install job passes while the binaries job dies.

Build the binaries with one `cargo build --bin` per binary: the
dependencies still compile in parallel during the first invocation, and
the second and third only link, so one LTO pass is in memory at a time.

A re-run against an existing tag now checks the release first and skips
targets whose archive is already attached instead of rebuilding them,
so `gh workflow run binaries.yml --ref main -f tag=vX.Y.Z` fills in the
missing Linux archives for v0.1.2 without repeating the macOS builds.

Not done: no automated test covers the workflow shell; the skip step
was dry-run locally against v0.1.0, v0.1.2 and a nonexistent tag, and
the link change is verified by the dispatched run for v0.1.2.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
@vyncint
vyncint merged commit 46f2e69 into main Sep 7, 2026
16 checks passed
@vyncint
vyncint deleted the fix/release-binaries-link-serially branch September 7, 2026 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant