feat(pipeline): add U-socket arc-length policy

[ElmoPA]

feat(pipeline): add U-socket arc-length policy

test(pipeline): make rotvec expectation explicit

feat(sim): add PushShapes simulator embodiments

fix(sim): make parallel-gripper grasps rigid and collision-safe

feat(sim): add guarded chain-gripper data pipeline

Add the four-link equal-angle chain gripper, resumable collected-demo generation, and control-gap matrix collection. Include complete articulated collision geometry and harden the UMI finger guard and physical grasp detector.

feat(sim): add chain gripper point control mode

Merge commit '6bb7b41f743a1889273304ae8da00146bd52a683' into elmo/pipeline-chain-points-20260826

refactor(sim): share chain geometry constants

feat(pipeline): add chain-gripper point policies

refactor(pipeline): consolidate chain point transforms

fix(pipeline): preserve bf16 U-socket arc rollout

config(flow-transfer): add full latent training matrix

feat(eval): support oriented PushShapes pipeline rollout

Port the tracked canonical eval core from a4d1839, then preserve U-Socket and ChainGripper state/action widths and route Pipeline forward_rollout through a BF16-safe native-action queue.

fix(eval): expose explicit seeds and rollout watchdog

feat(pipeline): add genuine multidomain diffusion baseline

fix(pipeline): make diffusion smoke validation-complete

fix(eval): honor every explicit rollout seed

test(pipeline): add exact Flow Transfer GPU smoke recipes

refactor(pipeline): make stage contracts mode-aware

test(pipeline): retain offline validation metrics in smokes

docs(flow-transfer): record implemented DP and exact dataset counts

config(flow-transfer): group DP in flow-transfer project

test(flow-transfer): pin DP wandb project

feat(pipeline): restore mode-aware config graph lint

fix(pipeline): derive graph seeds after transforms

docs(flow-transfer): record final graph and eval gates

test(smoke): verify W&B offline stream metrics

test(flow-transfer): smoke large latent recipes

Add level-specific ChainGripper obstacle collection

Refine diagonal corner initialization policies

Refine ChainGripper obstacle collection levels

Sample gate passages for obstacle collection

style(sim-v2): clean obstacle collection lint

feat(sim-v2): add validated obstacle MimicGen expansion

fix(sim-v2): isolate per-level generation provenance

fix(sim-v2): anneal obstacle MimicGen retries

fix(sim-v2): exclude nonreplayable obstacle sources

fix(sim-v2): redistribute failed source variants

feat(flow-transfer): add direct dense medium obstacle matrix

style(flow-transfer): sort config test imports

fix(flow-transfer): resolve obstacle DP data base

fix(flow-transfer): merge obstacle DP data in place

style(flow-transfer): format direct dense changes

feat(flow-transfer): add provenance-gated L40S launcher

ops(flow-transfer): submit L40S matrix after controller recovery

fix(flow-transfer): create per-arm smoke parent

fix(flow-transfer): verify visible L40S devices and full requests

fix(flow-transfer): inventory symlinked U-Socket episodes

fix(flow-transfer): validate cotrain domains order-independently

fix(flow-transfer): launch obstacle-only cotrain fairly

fix(flow-transfer): launch fair no-clip obstacle3k matrix

[ElmoPA]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat(pipeline): add U-socket arc-length policy #589 : 2 dependent PRs (#590 , #592 ) 👈 (View in Graphite)
• feat(pipeline): port arc-length action tokens #585 : 1 other dependent PR (#586 )
• feat(pipeline): consolidate Fold, PushT, overlays, and MoE #579 : 2 other dependent PRs (#580 , #583 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

Claude Code Review

Review of PR #589

Summary

This PR is enormous, mixes many unrelated concerns (simulator versioning, obstacle levels, arc-length policy, diffusion baseline, eval fixes, launcher ops, W&B smoke tests), and adds a top-level Tsimulation/ package with a sim_v1//sim_v2/ split plus environment-variable-based version selection. The commit list alone spans ~40 distinct changes across pipeline, sim, eval, flow-transfer, and ops.

Key concerns

1. PR scope is unreviewable

The description alone lists ~40 commits touching arc-length policy, chain gripper, PushShapes v2, MimicGen expansion, diffusion baseline, eval seeds, W&B smoke, L40S launchers, symlinked episode inventory, cotrain fairness, etc. This must be split. No reviewer can responsibly approve this as a single unit, and bisecting regressions later will be painful. Please break into at least: (a) sim_v1/sim_v2 restructure, (b) arc-length + chain policies, (c) eval/rollout changes, (d) flow-transfer configs/launchers, (e) diffusion baseline.

2. TSIM_VERSION env-var version selection is fragile

Tsimulation/__init__.py aliases submodules of the "active" simulator to the top-level package based on TSIM_VERSION. This is exactly the "silent version pickup" the docstring claims to avoid:

• DDP workers, subprocess launchers, and Hydra multi-run children may not all inherit the same env — a rank could import a different sim than rank 0.
• The alias is set at first import; anything that imports Tsimulation before TSIM_VERSION is set gets the default silently.
• The recommended assert env.SIM_VERSION == 2 is opt-in, not enforced.

Prefer explicit imports (from Tsimulation.sim_v2.pushshapes import ...) at call sites, or record the resolved version into every zarr episode's task_description and assert it in the loader.

3. Zarr writing bypasses conventions

Tsimulation/sim_v1/collect/zarr_writer.py uses ZarrWriter.create_and_write (good), but:

• Episode filename is episode_{obj}_{pusher}_obs{N}_{idx:06d}.zarr, not the codebase's UTC-timestamp episode hash convention (YYYY-MM-DD-HH-MM-SS-ffffff). This will break any downstream tool that assumes the hash format for dedup, SQL indexing, or R2 upload keys.
• No mention of R2 upload; if these demos are ever pushed to storage, they need to go to Cloudflare R2 with the standard naming.
• embodiment="pushshapes_sim" — confirm this exact string matches the enum (PUSHSHAPES_SIM = 15 per the notes); the SCHEMA_NOTES mentions adding it but the diff for embodiment.py isn't shown.

4. sim_v1 shipped as new code

sim_v1/ is marked DEPRECATED.md and FROZEN.md yet is being added in this PR. If it's frozen, it should already exist at a prior commit; if it's new, calling it "frozen" is misleading. Either:

• Add only sim_v2/ and reference the archive path in DEPRECATED.md, or
• Land sim_v1/ in a prior no-op commit that literally copies the pre-consolidation tree, so git blame shows the freeze point.

5. Coordinate frame / state semantics

observations.state = [agent_x, agent_y, obj_x, obj_y, obj_theta] in 512-px world coords. This is a 2D toy sim so head-frame reprojection isn't relevant, but confirm nothing in the training path assumes SLAM-world→head-frame conversion on this embodiment (there's a hint the pushshapes path bypasses standard transforms — good, but verify).

6. Test coverage claims are unverifiable from the diff

The diff was truncated at 80k chars. The description lists many test(...) commits (Flow Transfer GPU smoke, W&B offline stream, DP wandb project pin, offline validation metrics) but I can't see the test bodies. Concrete asks:

• Does test_smoke.py cover all {T,U,Z} × {circle,stick} × {0,1,2,3}? README claims so.
• Are the W&B smoke tests marked to skip in CI without credentials?
• Do the smoke tests pin exact dataset counts, and where are those documented?

7. Training regression risk from mode-aware stage contracts + graph seed derivation

"refactor(pipeline): make stage contracts mode-aware" and "fix(pipeline): derive graph seeds after transforms" both sound like they change the training-time graph. These could silently change norm stats or seed sequences for existing configs. Please confirm:

• All existing flow-transfer configs produce byte-identical training data (or explicitly bump a config version).
• Norm stats are recomputed / verified against a known baseline.

8. fix(pipeline): preserve bf16 U-socket arc rollout and BF16-safe queue in eval

BF16 rollout is easy to get subtly wrong (quantization of small pose deltas, action-queue accumulation drift). Are there numerical-parity tests vs fp32?

Suggestions

1. Split this PR. Non-negotiable at this size. Land the sim restructure first as a pure move (no logic changes), then the policies, then eval, then flow-transfer configs.
2. Drop TSIM_VERSION env indirection; use explicit sim_v2 imports and let sim_v1 be reached only by explicit Tsimulation.sim_v1... imports for legacy replay.
3. Switch episode filenames to the UTC-hash convention or document why pushshapes_sim is exemp

---

Reviewed by Claude · Review workflow