refactor(packaging): single source of truth for lambda shared libraries (uv workspace)

[Austin-s-h]

Description

Lambdas consume two first-party shared libraries — quilt-shared (py-shared) and t4-lambda-shared (lambdas/shared) — but each one pins them independently via opaque archive/<sha>.zip URLs, with nothing keeping those pins coherent. This PR gives the shared libraries a single source of truth so a shared-code change lands consistently across consumers, while keeping per-lambda independence a deliberate, opt-in choice rather than an accident of drift.

The problem this solves

Per-consumer SHA pins drift, in two opposite directions, on master today:

• quilt-shared fragments — it's pinned to several different commits at once across its consumers, so they're not on a common version of the same library. This is functional, not cosmetic: CRC64NVME checksum support (shared: Add CRC64NVME checksum type support #4623) ships only in the consumers that happened to re-pin, and is silently absent from the rest.
• t4-lambda-shared freezes — every consumer is stuck on a single Dec-2024 commit (update lambdas to Python 3.11 #4241) while the library itself has moved on repeatedly (BSQIK/Search: Indexing #4422, enable UP0{15,30,33,34,39} in ruff #4606, Add QuiltSync and Benchling Webhook documentation #4640, shared: migrate to uv and Python 3.13 #4652, build(deps): bump pyarrow from 18.1.0 to 23.0.1 in /lambdas/shared #4951). The shared lib's own uv/Python-3.13 migration (shared: migrate to uv and Python 3.13 #4652) has reached zero consumers.
• There's no single place to edit: one consumer pins inline as a PEP-508 URL while the rest use a [tool.uv.sources] table, and the dependency key is spelled both t4_lambda_shared and t4-lambda-shared.

Propagating one shared change today means hand-editing the SHA in up to ~9 consumers; the #4930/#4931 pair last week is the mechanism in miniature (edit py-shared, then a separate follow-up to re-pin a single consumer).

The approach: an opt-in shared space

• uv workspace: adds a root pyproject.toml + consolidated root uv.lock; members resolve the shared libs from one coherent in-tree source — drift becomes structurally impossible for anything in the shared space, and a shared change is one edit instead of N.
• Independence preserved at the boundary: lambdas that need to move on their own stay standalone with their own lock (via path = "../shared"). Joining the shared space is a choice, not a mandate.
• quilt3 stays out, on purpose: it keeps its own uv.lock and >=3.10 floor and is referenced only as a path dependency — preserving the quilt3-vs-lambdas cadence split.
• Workspace-aware build pipeline: lambda packaging/deploy flows use a uses-workspace helper that reads the members list from the root pyproject.toml at runtime, so the zip and ECR paths branch generically (no per-lambda special-case).
• Python support: quilt3 drops Python 3.9 and continues to support 3.10+ (requires-python = ">=3.10"). Workspace-rooted lambda/shared packaging is on the higher baseline (generally >=3.13, with lambdas/shared at >=3.12).

Open design question

This PR sets the dependency-management model the rest of the series inherits, so it's worth settling deliberately. The core team has reasonably noted that per-component isolation is a deliberate choice for differing change cadences — see the discussion thread for the tradeoff and the alternatives I'm open to (narrowing workspace membership, or keeping per-consumer pins but replacing the opaque SHAs with enforced git + tag pins). Feedback on scope very welcome.

Validation

• uv lock --check across the root workspace and standalone members
• uses-workspace classification verified for every deploy-matrix entry (member vs standalone), both zip and ECR paths
• CI/workflow updates validated for workspace-aware packaging flows

TODO

• Unit tests
• Security: Confirm that this change meets security best practices and does not violate the security model
• Open and Embed: Confirm that this change doesn't break Open variant and Embed widget
• Documentation
  • run optipng on any new PNGs
  • Python: Run build.py for new docstrings
  • JavaScript: basic explanation and screenshot of new features
  • Markdown somewhere in docs/**/*.md that explains the feature to end users (said .md files should be linked from SUMMARY.md so they appear on https://docs.quilt.bio)
  • Markdown docs for developers
• Changelog entry (skip if change is not significant to end users, e.g. docs only)

Greptile Summary

This PR replaces per-consumer SHA-pinned archive URLs for quilt-shared and t4-lambda-shared with a uv workspace at the repo root, eliminating version drift across ~9 lambda consumers. Eight packages join the workspace (py-shared, lambdas/shared, and six lambdas); standalone lambdas that need independence retain their own lock files with path = \"../shared\" local sources instead of opaque ZIP URLs.

• Root workspace + guardrails script: pyproject.toml defines the workspace; .github/scripts/python_packaging.py enforces the dependency contract at CI time and drives workspace-aware uv export / install-target selection in both the zip and ECR build paths.
• Workspace-aware CI and deploy: py-ci.yml and build_zip.sh branch on uses-workspace to choose the correct uv export invocation; the ECR workflow selects build context (repo root vs lambda dir) using the same helper.
• tabular_preview and thumbnail have a build-breaking gap: both are non-workspace-member (so the ECR workflow uses ./lambdas/<name> as the Docker build context), but their pyproject.toml/uv.lock now declare t4-lambda-shared = { path = \"../shared\" } — a directory that is outside the build context. Both uv sync calls in the Dockerfile will fail to resolve this path, breaking ECR deploys for those two lambdas.

Confidence Score: 4/5

Safe to merge for zip-deployed lambdas and the indexer; the tabular_preview and thumbnail ECR builds will fail on the next push to master that touches the lambdas/ tree.

The workspace setup, guardrails script, zip build path, and indexer Dockerfile are all internally consistent. The gap is that tabular_preview and thumbnail switched from URL sources to path = ../shared without joining the workspace, so the ECR workflow keeps building them from their own directory — a context that does not contain lambdas/shared. Both uv sync stages in each Dockerfile will fail to resolve the path dependency.

lambdas/tabular_preview/Dockerfile and lambdas/thumbnail/Dockerfile — both need either promotion to workspace members (so the build context becomes the repo root) or explicit COPY directives for lambdas/shared.

Important Files Changed

Filename	Overview
.github/scripts/python_packaging.py	New packaging helper script: provides guardrails, install-targets, package-name, and uses-workspace subcommands; logic is self-consistent and the guardrail contract enforcement is well-designed
.github/workflows/deploy-lambdas.yaml	Zip deploy correctly mounts the whole repo; ECR deploy correctly selects context per uses-workspace, but thumbnail and tabular_preview are not workspace members while now having path deps outside their build context — Docker builds will fail
lambdas/tabular_preview/Dockerfile	Adds --no-default-groups to both uv sync calls; now depends on t4-lambda-shared = { path = ../shared } which is outside the standalone build context — Docker build will fail
lambdas/thumbnail/Dockerfile	Same issue as tabular_preview — adds --no-default-groups, has t4-lambda-shared = { path = ../shared } path dep outside the standalone build context
lambdas/indexer/Dockerfile	Correctly updated to Python 3.13 base image and restructured to COPY workspace sources (py-shared, lambdas/shared, indexer) before uv sync — workspace-aware pattern works correctly
lambdas/scripts/build_zip.sh	Workspace-aware zip build: branches on uses-workspace for the correct uv export invocation, assembles install_targets correctly for both workspace members and standalone path-dep lambdas; libscipy_openblas strip exclusion is a good addition
pyproject.toml	New root workspace definition with 8 members; uv version pin (>=0.11,<0.12) and workspace sources for quilt-shared / t4-lambda-shared are correctly set
lambdas/shared/pyproject.toml	Lowered requires-python to >=3.12 (enables both workspace and standalone/local-source consumers), relaxed version pins to ranges, cleaned up redundant self-extras in preview optional-deps
.github/workflows/py-ci.yml	Adds guardrails step and workspace-aware test flow; enables the previously-commented-out format check; both workspace-member and standalone export paths are correctly handled

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    ROOT["Root pyproject.toml\n(uv workspace)"]

    subgraph WS["Workspace Members"]
        PS["py-shared (quilt-shared)"]
        LS["lambdas/shared (t4-lambda-shared)"]
        ESI["lambdas/es_ingest"]
        ICE["lambdas/iceberg"]
        IDX["lambdas/indexer (ECR)"]
        MI["lambdas/manifest_indexer"]
        PKP["lambdas/pkgpush + quilt3 path dep"]
        S3H["lambdas/s3hash + quilt3 path dep"]
    end

    subgraph SA["Standalone (path dep)"]
        AC["lambdas/access_counts"]
        PKE["lambdas/pkgevents"]
        PRV["lambdas/preview"]
        TP["lambdas/tabular_preview (ECR)"]
        TH["lambdas/thumbnail (ECR)"]
        TR["lambdas/transcode"]
    end

    ROOT --> WS
    LS --> SA

    subgraph CI["CI / Deploy"]
        ZIP["build_zip.sh"]
        ECR["docker buildx"]
    end

    WS --> ZIP
    WS --> ECR
    SA --> ZIP
    SA --> ECR

    ECR -. "context outside build context" .-> TP
    ECR -. "context outside build context" .-> TH

Loading

_{Reviews (3): Last reviewed commit: "perf: lint" | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 99.70326% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 46.50%. Comparing base (8eec5f4) to head (c868d1c).

Files with missing lines	Patch %	Lines
lambdas/indexer/test/test_index.py	97.36%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4933   +/-   ##
=======================================
  Coverage   46.50%   46.50%           
=======================================
  Files         832      832           
  Lines       34090    34092    +2     
  Branches     5833     5833           
=======================================
+ Hits        15854    15856    +2     
  Misses      16237    16237           
  Partials     1999     1999           

Flag	Coverage Δ
api-python	93.14% <100.00%> (+<0.01%)	⬆️
catalog	21.55% <ø> (ø)
lambda	96.63% <99.11%> (+<0.01%)	⬆️
py-shared	98.02% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Austin-s-h]

@greptile please re-review

[Austin-s-h]

@copilot Please thoroughly review this PR, including updating the language about Python support. 3.9 has been dropped but 3.10 is still supported.

[Copilot]

@copilot Please thoroughly review this PR, including updating the language about Python support. 3.9 has been dropped but 3.10 is still supported.

Addressed in 1f975a8. I reviewed the PR and updated the changelog language to clarify that only Python 3.9 was dropped and quilt3 still supports Python 3.10+.

[Austin-s-h]

@drernie @sir-sigurd This should be ready to review!

[sir-sigurd]

(AI-assisted analysis; conclusions are my own — sanity-check the specifics.)

The Python-floor handling looks right now (quilt3 staying >=3.10), and the indexer Dockerfile root-COPY is sorted. My one remaining question is the central design choice: the uv workspace trades away independent per-lambda versioning, and I'm not sure that's what we want for independently-deployed lambdas.

Today each lambda pins its own version of quilt-shared/t4-lambda-shared and resolves its own third-party deps in its own lockfile. For services that deploy independently, that independence seems like the property we want: each can bump or hold a shared-lib/dependency version on its own schedule — security fixes, divergent needs, isolated blast radius — and staged migration of the shared libs is possible (one consumer moves, others don't have to).

A uv workspace gives that up. A single shared lock means one version of every dependency across all members (no divergence without evicting a member from the workspace), and all members track the in-tree shared libs rather than a pinned version (so no per-lambda version, no staged migration). That's not a side detail — it's the opposite of how these deploy today.

And I'm not sure there's a problem here that needs it. The current archive/<sha>.zip pins work and are reproducible; their only real cost is co-development friction (you must push the shared lib before a consumer can pin a new version) — but that's the inherent, accepted price of versioned shared code, and it's mitigable (point at a local path while iterating, re-pin the tag to commit).

So the question: what concrete problem does the workspace solve that justifies giving up independent per-lambda versioning? If one-version lockstep across these lambdas is a deliberate goal, that's legitimate — but it should be stated and argued, because it inverts today's model. If it isn't a goal, the current versioned approach already matches what we want; the most I'd change is tidying the drifting archive/<sha>.zip pins into deliberate git/tag pins (stability + legibility), keeping per-lambda version control.

Not blocking — just that this PR sets the dependency-management model the rest of the series inherits, so it's worth settling deliberately rather than defaulting to the workspace.

[nl0]

yeah my thoughts exactly: not sure what are we trying to solve here.
we've revisited most of the packaging pipelines fairly recently, migrating individual pieces to uv,
so the current status quo feels quite workable for the core team.
change cadence is quite different per component, especially for lambdas/py-shared vs quilt3,
so the per-component isolation and explicit cross-component dependencies were deliberate choices directly rooted in our workflows.

[Austin-s-h]

cc @drernie

Appreciate the pushback — it's the right thing to settle here, and it pushed me to quantify what this is actually for. "chore: modernize packaging" undersells it, and I think that framing is why the intent isn't landing. Let me reframe around the problem and back it with numbers from our own history.

The problem: shared-space changes land inconsistently across lambdas

Shared code is pulled in per-consumer via opaque archive/<sha>.zip pins, with nothing keeping those pins coherent. The result on master today:

• quilt-shared (py-shared) is pinned to three different SHAs at once across its six consumers: df53c9ce (es_ingest, indexer, manifest_indexer, py-shared: add utilities for ES ingest #4433), a4b31030 (pkgpush, s3hash, shared: Add CRC64NVME checksum type support #4623), 12bb4a33 (iceberg, py-shared: retarget QueryMaker to per-bucket Iceberg tables #4930) — a spread of ~11 months (Jul 2025 → Jun 2026). These lambdas are not on a common version of the same library.
• That's functional, not cosmetic: CRC64NVME checksum support landed in py-shared at a4b31030 (shared: Add CRC64NVME checksum type support #4623) and therefore ships only in pkgpush and s3hash — the other four consumers silently don't have it.
• t4-lambda-shared (lambdas/shared) is the opposite failure mode — all nine consumers are frozen on one SHA, d496dffb (update lambdas to Python 3.11 #4241, Dec 2024), while the lib itself has moved on at least five times since (BSQIK/Search: Indexing #4422, enable UP0{15,30,33,34,39} in ruff #4606, Add QuiltSync and Benchling Webhook documentation #4640, shared: migrate to uv and Python 3.13 #4652, build(deps): bump pyarrow from 18.1.0 to 23.0.1 in /lambdas/shared #4951). The shared lib's own uv/Python-3.13 migration (shared: migrate to uv and Python 3.13 #4652) ships in zero of its nine consumers.
• There isn't even a single place to edit: thumbnail pins inline as a PEP-508 URL while the other eight use a [tool.uv.sources] table, and the key is spelled both t4_lambda_shared and t4-lambda-shared.

The #4930/#4931 pair from last week is the mechanism in miniature: editing py-shared (#4930) needed a separate same-day follow-up (#4931) to re-pin just the iceberg consumer — and the other five were left on older SHAs, which is exactly why three pins coexist.

What the modernization accomplishes

The workspace isn't "merge everything into lockstep" — it's an opt-in shared space with a consistent git state, and it directly removes the churn above:

• Inherit the shared space, or don't. A lambda co-developing with the shared libs joins the workspace and resolves them from one coherent in-tree source; a lambda that needs to move on its own stays standalone with its own lock (the standalone lambdas already do this via path = "../shared"). Independence stays a deliberate choice at the component boundary.
• Consistent git state by construction. For anything in the shared space, the 3-SHA split and the year-behind freeze become structurally impossible — one source, one resolution, legible at a glance.
• It scales. Propagating a shared change today means up to 9 hand-edited SHA bumps for t4-lambda-shared and 6 for quilt-shared (PR Use master hash for lambdas shared #4251 hand-edited 7 lambdas / 14 files for a single shared bump). In the shared space that collapses to one edit. Adding a lambda is "join or stay standalone," and the build path already adapts automatically — uses-workspace reads the members list from the root pyproject.toml at runtime, so both the zip and ECR paths branch generically (this also closes the hard-coded indexer special-case Greptile flagged).
• quilt3 stays out on purpose. It keeps its own lock and >=3.10 floor and is referenced only as a path dependency — so the lambdas/py-shared-vs-quilt3 cadence split you specifically called out, @nl0, is intentionally preserved.

The tradeoff I'm not hiding

Workspace members do share one third-party resolution, and a few deps differ across them today (e.g. indexer on jsonschema 3.2.0 vs 4.25.x elsewhere, a boto3 range). For deps that should genuinely diverge, the answer is to keep that lambda standalone rather than force it into the shared space — the model supports both. And in fairness, the core team treats per-component independence as a feature, not a bug, and @sir-sigurd reasonably frames the push-then-re-pin step as the accepted price of versioned shared code. So the real question is whether the drift above (three live quilt-shared SHAs, all nine t4-lambda-shared consumers a year-plus behind, the shared lib's own migration never propagated) is worth trading some of that independence for a single source of truth — and I think for the genuinely co-developed cluster it is.

Worth noting the same per-component fan-out shows up beyond shared code: the Dec-2025 uv/Python-3.13 migration shipped as nine separate PRs (#4618, #4647, #4649–#4655), and routine security bumps fan out across the per-lambda lockfiles (idna→3.15 = 14 PRs #4904–#4927, urllib3→2.7.0 = 11 PRs, aiohttp→3.14 = 5 PRs). A single shared lock coalesces a lot of that.

What I'm genuinely open to

I care more about killing the drift than about the workspace specifically, so these are real questions, not rhetorical ones — happy to be steered on any of them:

1. Membership scope. The biggest lever is which lambdas belong in the shared space. If the current set sweeps in lambdas you'd rather keep independent, I'm glad to narrow it to just py-shared + lambdas/shared and the consumers that are genuinely co-developed with them, and leave everyone else standalone. Where would you draw that line?

2. Workspace vs. enforced tag-pins. If the team would rather keep per-consumer pins entirely, I think we can still get the "single source of truth, no silent drift" win without a workspace: replace the opaque archive/<sha>.zip URLs with legible git + tag pins (the pattern we already use for pydoc-markdown in gendocs) and add a guardrail that asserts every consumer points at the same shared tag. That keeps full per-lambda independence and just makes drift a CI failure instead of a surprise. I'm happy to go this route instead if it's a better fit for how you all work — @sir-sigurd, this is close to what you suggested.

3. Sequencing. Regardless of which model we land on, the build-path generalization and the drift cleanup stand on their own. If it's easier to review, I can split those out into a smaller first PR and let the workspace-vs-tag-pin decision land separately, so we're not coupling a mechanical fix to the design call.

4. Python floor. The root currently sits at >=3.13 with lambdas/shared held at >=3.12. If you'd rather align those (or keep more spread for a specific component), I'm flexible — it's a knob, not a constraint.

Does scoping it down one of these ways address the independence concern, or do you read the per-component drift differently than I'm seeing it? Happy to rework in whatever direction gets this to "obviously right" for the core team.

[Austin-s-h]

@greptile re-review

[Austin-s-h]

@sir-sigurd @nl0 @drernie Curious if you have had a chance to discuss this yet? I don't want to base future work off of it unless it is an accepted path. I do think it would largely address #4869 type of behavior on the python side.

[sir-sigurd]

(AI-assisted: verification and research by Claude; conclusions are mine — sanity-check the specifics.)

Sorry for the slow reply — I wanted to verify rather than opine, and it took a few rounds. Your numbers all check out (3 quilt-shared SHAs / 6 consumers; 9 t4-lambda-shared consumers on d496dffb at the time; idna = 14 PRs, urllib3 = 11; #4251 = 7 lambdas / 14 files). But after digging into what each number means, I land differently on most of them.

The quilt-shared examples don't show harm. CRC64NVME lives in quilt_shared/crc64.py/pkgpush.py; the four "lacking" consumers import only es/const/log/athena/iceberg_queries — none touch checksum code — and the two that do (pkgpush, s3hash) are exactly the two on the #4623 SHA. Of the five t4-lambda-shared changes since its pin: one test-only, one lint-only, the lib's own packaging migration (consumers don't need it — the old zip still installs), a deletion of already-relocated constants, and the pyarrow constraint bump — a dependency-floor change consumers pick up whenever they next re-pin (preview already has, via #5029). So the drift you measured is real divergence, but benign in each instance you cited — the pins track what each consumer needs. I'll grant the meta-point though: I could only establish that by reading imports by hand; nothing in the repo would tell us, and that missing visibility is a fair concern.

On #4869 — I don't think this addresses it. #4869 is about the ~26 Dependabot PRs that are almost all /catalog (npm), and its own "what would help" is auto-merge + grouping config, explicitly not repo reorg. The Python fan-out you hit (idna/urllib3) is real toil, but fixable with a setting: Dependabot grouped security updates raise "a single pull request to update as many vulnerable dependencies as possible," across directories per ecosystem — and it's not enabled here (dependabot.yml covers only github-actions). The fan-out argues for turning that on, on both the JS and Python sides, before it argues for a workspace.

Which narrows the disagreement to one shared lock vs. N locks — and I lean N. uv's own docs say workspaces "are not suited for cases in which members… desire a separate virtual environment for each member; path dependencies are often preferable." Fair reply that the members don't conflict today — but they co-resolve partly because this PR reconciled them (e.g. indexer's jsonschema), per-component cadence is the workflow @nl0 described, and the classic one-version rationale (diamond dependencies) doesn't apply to closed lambda bundles that each resolve independently. Given we treat per-component independence as a feature, a single shared lock trades away the thing we want to keep.

Two things I checked expecting to recommend, and now won't:

• git+https@<sha> pins instead of the zips: worse for this repo. uv does no shallow/partial fetch for git sources — it pulls the commit's full reachable history (~184 MiB of quilt history vs ~45 MB per snapshot zip), the CI export→pip path would multiply that per matrix job, and the lambda Docker images don't ship git. The archive-URL form is actually the right one here.
• Tag-based pins with a same-tag CI gate (your option 2 as written): tags add release ceremony for an in-repo lib (SHAs exist for free on every commit), and a hard same-tag gate is one-version lockstep by other means — it re-creates the property under dispute rather than resolving it.

What I'd actually change is small, and some of it is done: the pin-shape normalization you rightly flagged (three coexisting variants, two spellings — no single place to edit) is merged as #5091, so a shared-lib bump is now a single mechanical find-and-replace. Next: re-pin the consumers still on d496dffb where their runtimes allow it (indexer's comes with its Python 3.13 migration, already in flight), and enable grouped security updates. Beyond that I'd let machinery be pulled by demonstrated recurring toil rather than built up front: if shared-lib propagation keeps biting, a bump-automation step — one bot PR re-pinning all consumers and relocking — gets the same one-step propagation the workspace buys, without giving up per-lambda locks.

One thing your direction gets right: a shared-lib change is never CI-tested against consumers until each re-pins. If that bites, a CI job building the co-developed consumers against the in-tree libs (path deps — uv's own recommendation) gets that testing without unifying resolution.

Your questions:

1. Direct answer, since you asked whether this is an accepted path: I don't think we should use a workspace — we're keeping per-component pins and independent locks. I'll keep the workspace idea in mind for the future though; if sustained co-development of some cluster makes lockstep genuinely attractive, we can revisit it then.
2. Covered above — pins stay, so the tag variant is moot too.
3. Yes to the split, with one precision: the drift cleanup is genuinely model-independent — #5091 covered the pin shapes, and the d496dffb re-pins follow as runtimes allow. The build-path generalization is mostly workspace enablement though — the uses-workspace branching has no purpose without one — so from that piece I'd take only whatever proves model-independent — and on checking, even the build_zip.sh openblas-stripping exclusion is inert today (scipy is only in tabular_preview, which is Docker-deployed), so it belongs with whatever PR first puts scipy into a zip-built lambda.
4. No objection to the lambda floors as long as quilt3 stays >=3.10.

One process ask, looking at how this series has gone: for anything carrying a design decision, could we start with a focused issue for the problem (or a pointer to an existing one) and have the discussion before implementation? Your strongest contribution in this thread — the drift quantification — is exactly issue material, and it would have been far cheaper for everyone (you included) to settle "is this drift worth a workspace?" there, before 33 commits. This matters double for outside contributions: maintainer attention is the practical bottleneck, an issue is the cheapest thing for us to engage with, and a PR that implements something pre-agreed gets reviewed as "does it do what we agreed" rather than stalling on "do we want this" — which is where both #4932 and this PR lost their time. And since we're all visibly running AI-assisted workflows, problem-first is also dramatically more token-efficient on both sides than litigating design inside a 10k-line diff.

Net: with the workspace off the table, I don't think there's a PR left here — everything salvageable lands better as small focused PRs, and that's already happening (#5091; the d496dffb re-pins). The parts of this branch your follow-up stack genuinely needs (dependency bumps for the imaging work) belong in those PRs directly. So I'd close this one — with real appreciation for the pressure-testing it forced: the drift you surfaced was real, and it's getting fixed, just by smaller tools. And the workspace idea stays on file for when co-development patterns might actually call for it.