From f826b4c37da0bf3b715dabff1852581dad2503e7 Mon Sep 17 00:00:00 2001 From: HaRoLd <303926+HarryR@users.noreply.github.com> Date: Fri, 10 Jul 2026 06:40:27 +0000 Subject: [PATCH] docs: add REPRODUCIBLE-BUILDS.md; refresh stale README/toolchain notes Document how the chain's builds are made byte-for-byte reproducible (and how CI and local are kept identical): pinned+aligned toolchain, SOURCE_DATE_EPOCH, locked deps, per-target static-linking rustflags, and the two path-embedding traps that broke it -- CARGO_HOME (/tmp vs /src) and rust-src sysroot paths -- both fixed with --remap-path-prefix. Includes the toolchain-bump procedure (update the /rustc/ in the rust-src remap) and the CI==local verify steps. Also refresh notes that went stale when the global RUSTFLAGS moved to per-target config and stage1's toolchain was aligned to 1.91.1: the README image table and toolchain/target policy, and this file's own comment (dropped the "lockboot (1.91.0)" reference; stage0/stage1 are both 1.91.1 now). Co-Authored-By: Claude Opus 4.8 --- README.md | 21 ++++---- REPRODUCIBLE-BUILDS.md | 115 +++++++++++++++++++++++++++++++++++++++++ rust-toolchain.toml | 6 +-- 3 files changed, 130 insertions(+), 12 deletions(-) create mode 100644 REPRODUCIBLE-BUILDS.md diff --git a/README.md b/README.md index 2cbf2f8..9f172a4 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ own `target/` (we share the registry, not build outputs). | Image | Purpose | Where it's defined | Builds Rust? | |-------|---------|--------------------|:---:| -| `lockboot:build` | Reproducible Rust builds: pinned toolchain, identical `/src/.cargo`+`/src/.rustup` paths, static-musl RUSTFLAGS. | `Dockerfile.build`, kept **byte-identical in each project** (canonical copy in `stage0`) so CI can build each repo standalone. | ✅ | +| `lockboot:build` | Reproducible Rust builds: pinned toolchain, committed lockfiles, per-target static-musl rustflags + path remapping (see [`REPRODUCIBLE-BUILDS.md`](REPRODUCIBLE-BUILDS.md)). | `Dockerfile.build`, kept **byte-identical in each project** (canonical copy in `stage0`) so CI can build each repo standalone. | ✅ | | `lockboot:harness` | Lean QEMU runner for stage0's boot tests (boots a disk, mocks EC2 metadata). Entrypoint-baked. | `stage0/Dockerfile.harness` | ❌ | | **workspace devcontainer** | The environment you **edit in and drive builds from**: bind `/src`, host network, docker-outside-of-docker, `/dev/kvm`. | `.devcontainer/Dockerfile` (this repo, lean) | ❌ | @@ -83,14 +83,17 @@ from `stage0` (the canonical/reference project). ## Toolchain & target policy - `rust-toolchain.toml` here pins a **default** (`1.91.1`); each sub-repo's own - `rust-toolchain.toml` wins by nearest-file (e.g. stage1 `1.91.0`, stage0 `1.91.1`). The - risc0 guest in `vaportpm-zk` uses its own `rzup`-managed toolchain. All install once into the - shared `.rustup`. -- `.cargo/config.toml` sets `SOURCE_DATE_EPOCH=0` and static-musl link flags that **only bind - when a musl target is opted into**. It deliberately sets **no** global `[build] target` and - **no** gnu-host linker override, so the gnu-host repos (`vaportpm`, the risc0 host crate) and - the Python site (`wavebend.org`) are never forced into musl/`rust-lld`. Repos that want - musl-by-default (stage1, os402) set it in their own config. + `rust-toolchain.toml` wins by nearest-file. stage0 and stage1 both pin `1.91.1` and are kept in + **lockstep** (they must produce byte-compatible builds). The risc0 guest in `vaportpm-zk` uses its + own `rzup`-managed toolchain. All install once into the shared `.rustup`. +- The shared `.cargo/config.toml` carries only workspace-wide `[env]` (`SOURCE_DATE_EPOCH=0`). Each + repo owns its per-target rustflags (static-musl linking, path remapping) in its **own** + `.cargo/config.toml` -- CI builds each repo standalone without this workspace file, so anything a + build needs must live in the repo. This file sets **no** global `[build] target` and **no** gnu-host + linker override, so the gnu-host repos (`vaportpm`, the risc0 host crate) and the Python site + (`wavebend.org`) are never forced into musl/`rust-lld`. +- See [`REPRODUCIBLE-BUILDS.md`](REPRODUCIBLE-BUILDS.md) for how CI and local builds are kept + byte-identical, and what to update on a toolchain bump. ## Known follow-ups diff --git a/REPRODUCIBLE-BUILDS.md b/REPRODUCIBLE-BUILDS.md new file mode 100644 index 0000000..24ad26d --- /dev/null +++ b/REPRODUCIBLE-BUILDS.md @@ -0,0 +1,115 @@ +# Reproducible builds + +Lock.Boot is a **measured** boot chain: stage0 and the stage1 UKI are hashed into TPM PCRs, and an +attestation is checked against expected values. That only means anything if the same source produces +the same bytes: a verifier (or an auditor rebuilding from source) must be able to reproduce the exact +`stage0.efi` and `linux.efi` that CI published, and get the same PCRs. This file explains what makes +the builds bit-for-bit reproducible, the two non-obvious traps that broke it, and how to verify it. + +Scope: the chain repos `stage0` and `stage1`. Both build inside the shared `lockboot:build` image +(`stage0/Dockerfile.build`, identical in stage1) via their Makefiles. + +## What guarantees reproducibility + +1. **Pinned, aligned toolchain.** Every repo's `rust-toolchain.toml` pins the *same* channel + (currently `1.91.1`), so `rustc`/`std` are identical everywhere. rustup installs the pinned + toolchain regardless of what the base image ships. Keep stage0 and stage1 on the same channel -- + they must produce byte-compatible signature framing, and `std`/sysroot paths are + toolchain-version-specific. +2. **`SOURCE_DATE_EPOCH=0`.** Set in both `Dockerfile.build`s and in `stage1/tools/build-uki/build.sh`, + so any timestamp baked into an artifact is fixed. FAT timestamps in the boot disk are separately + normalized (`stage0/tools/normalize-fat-timestamps.py`); the UKI is assembled deterministically by + `mkuki`. +3. **Locked dependencies.** `Cargo.lock` is committed in every crate, and the build recipes pass + `--locked` so a build can never silently resolve newer transitive deps. +4. **Deterministic linking.** Static musl / UEFI builds use `rust-lld` + `+crt-static` via per-target + `rustflags` in each repo's `.cargo/config.toml` (not a global `RUSTFLAGS`, which would also hit the + host and break proc-macros). The shared workspace `/.cargo/config.toml` carries only `[env]`; each + repo owns its rustflags (CI checks out each repo alone, so it must be self-sufficient). +5. **Path canonicalization** -- the subtle part, below. + +## The two path traps + +rustc embeds absolute **source paths** into binaries (panic `Location` strings, some debuginfo). If +those paths depend on the build environment, two environments building identical source produce +different bytes -- and a different measured `linux.efi` / PCR. Two such paths leaked between CI and a +local dev build; both are neutralized with `--remap-path-prefix` in the per-target `rustflags`. + +### 1. `CARGO_HOME` (dependency registry paths) + +CI runs with `CARGO_HOME=/tmp/.cargo` (the Makefile's `CACHE_ENV` redirect keeps caches ephemeral and +out of the bind-mounted tree under `gh act`), while a local build uses `/src/.cargo`. So dependency +sources embed as `/tmp/.cargo/registry/...` vs `/src/.cargo/registry/...`. Fix: + +```toml +"--remap-path-prefix=/src/.cargo=/cargo", +"--remap-path-prefix=/tmp/.cargo=/cargo", +``` + +Both collapse to `/cargo`, so the build no longer depends on where `CARGO_HOME` is. + +### 2. `rust-src` (std sysroot paths) + +If the `rust-src` component is installed, rustc emits `std` panic-location paths from the *sysroot* +(`$RUSTUP_HOME/toolchains//lib/rustlib/src/rust/library/core/...`) instead of the `/rustc/` +that a clean toolchain bakes in. **rust-analyzer installs `rust-src` into the shared local +`/src/.rustup`**, but CI's fresh `/tmp/.rustup` has only the pinned `rustfmt`+`clippy` -- so a local +build emitted the sysroot path where CI emitted `/rustc/`. Fix: remap the sysroot rust-src path +to that same `/rustc/`: + +```toml +"--remap-path-prefix=/src/.rustup/toolchains/1.91.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust=/rustc/ed61e7d7e242494fb7057f2657300d9e77bb4fcb", +``` + +This is a **no-op in CI** (no rust-src there), so it only pulls a local build onto CI's existing +output. The host triple in the FROM path is always `x86_64-unknown-linux-gnu` (the build host), so one +remap covers every target arch. + +> `trim-paths` (the Cargo profile option) would replace all of this cleanly, but it is still +> nightly-only as of Rust 1.91, so we use the stable `--remap-path-prefix`. + +## Bumping the toolchain (READ THIS) + +The rust-src remap hardcodes **the toolchain version** (FROM path) and **the rustc commit-hash** (TO). +A stale value silently becomes a no-op and reproducibility quietly regresses. On any toolchain bump: + +1. Update `channel` in **every** `rust-toolchain.toml` (stage0, stage1, workspace) to the same value. +2. Get the new commit-hash: `rustc -Vv | grep commit-hash` (or grep a freshly built binary for + `/rustc/`). +3. Update the version + hash in the rust-src `--remap-path-prefix` in **both** `stage0/.cargo/config.toml` + and `stage1/.cargo/config.toml`. +4. Re-run the CI==local verification below -- it catches a stale hash. + +## Verifying CI == local + +CI uploads build artifacts on every push to `main` and on PRs (`stage0/.github/workflows/build.yml`, +`stage1/.github/workflows/build.yml`), so no merge or tag is needed. + +```sh +# 1. Build locally (same repo, same commit) and note the hashes: +# stage1: make x86_64 stage2-x86_64 -> tools/build-uki/x86_64/{linux.efi,stage2} +# stage0: make build/x86_64/stage0.efi +# 2. Grab the CI artifacts from the run for that commit: +gh run download -R lockboot/stage1 -n uki-x86_64 -D ci-s1 +gh run download -R lockboot/stage0 -n stage0-x86_64 -D ci-s0 +# 3. Compare. +``` + +- **stage1** uploads the unsigned `linux.efi` and `stage2`: compare `sha256` directly -- they must be + identical. (`linux.efi` matching is the acceptance test: it is the measured UKI.) +- **stage0** uploads the **db-signed** `BOOTX64.EFI` (and `boot.disk`). The Secure Boot db/PK/KEK keys + are ephemeral snakeoil regenerated per build, so the raw bytes differ between any two builds. + Compare the **Authenticode** hash (which excludes the signature) against a local `stage0.efi`, or -- + lacking Authenticode tooling -- grep the binary and confirm it is path-canonical (only `/cargo` and + `/rustc/`, no `/tmp` or `/src`). + +A mismatch means an environment-dependent input leaked (most likely a stale rust-src hash after a +toolchain bump, or a new dependency embedding a path the remaps don't cover). + +## Known limitations + +- The rust-src remap is toolchain-hash-specific (see the bump procedure). +- A definitive stage0 raw-hash comparison needs Authenticode tooling (`osslsigncode` / `pesign` / + `sbverify`), which is not required for the path-canonical check. +- Reproducibility is relative to the pinned toolchain + the `lockboot:build` image; reproduce in that + environment, not on an arbitrary host. diff --git a/rust-toolchain.toml b/rust-toolchain.toml index d8aec22..a1ca69b 100644 --- a/rust-toolchain.toml +++ b/rust-toolchain.toml @@ -2,9 +2,9 @@ # # This is the fallback used when running rustup/cargo from the workspace root. # Each sub-repo pins its own toolchain via its own rust-toolchain.toml (nearest -# file wins), so this does not override e.g. lockboot (1.91.0) or stage0 -# (1.91.1) when working inside them. All resolved toolchains are installed once -# into the shared RUSTUP_HOME ($WORKSPACE/.rustup). +# file wins); stage0 and stage1 both pin 1.91.1 (kept in lockstep -- see +# REPRODUCIBLE-BUILDS.md). All resolved toolchains are installed once into the +# shared RUSTUP_HOME ($WORKSPACE/.rustup). [toolchain] channel = "1.91.1" components = ["rustfmt", "clippy"]