BotRunner64 · BotRunner64 · Jun 25, 2026 · Apr 16, 2026 · Apr 16, 2026 · Apr 20, 2026
diff --git a/.gitignore b/.gitignore
@@ -91,7 +91,10 @@ Thumbs.db
 # GMR retargeting assets (downloaded from ModelScope)
 teleopit/retargeting/gmr/assets/
 
-# Legacy training-side robot assets are no longer tracked; FK tooling reuses GMR unitree_g1.
+# Canonical robot XML/meshes (downloaded from ModelScope)
+assets/robots/
+
+# Legacy training-side robot assets are no longer tracked.
 train_mimic/assets/
 
 # ModelScope download cache

diff --git a/.gitmodules b/.gitmodules
@@ -1,3 +1,9 @@
 [submodule "third_party/unitree_sdk2_python"]
 	path = third_party/unitree_sdk2_python
 	url = https://github.com/unitreerobotics/unitree_sdk2_python.git
+[submodule "third_party/linkerhand-python-sdk"]
+	path = third_party/linkerhand-python-sdk
+	url = https://github.com/BotRunner64/linkerhand-python-sdk.git
+[submodule "third_party/somehand"]
+	path = third_party/somehand
+	url = https://github.com/BotRunner64/somehand.git
diff --git a/AGENTS.md b/AGENTS.md
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,12 @@
 # Changelog
 
+## [0.4.0] - 2026-06-25
+
+- 改进 Pico 实时控制：支持 pico-bridge 0.2.1、`ARMS` 模式，以及保留 retargeter warm-start 的模式切换/暂停恢复。
+- 新增可选 LinkerHand L6/O6 sim2real 控制，支持 Pico gripper 输入和低延迟 L6 `vr_hand_pose`。
+- 新增 Pico sim2real 手动 HDF5 录制，以及用于训练数据采集的交互式 Pico motion recorder。
+- 优化训练数据流程：minimal HDF5 shards、显式 precompute、rewind 采样和更新后的 tracking rewards。
+
 ## [0.3.0] - 2026-05-12
 
 - 重构实时输入栈，Pico 4 统一使用 pico-bridge 0.2.0 in-process receiver，并移除旧 ZMQ/onboard Pico 路径。
@@ -13,13 +20,12 @@
 - 新增独立 Standing 控制器、离线播放键盘控制与 Pico sim2sim 模式控制。
 - 优化实时 mocap 缓冲与 catch-up，并将发布模型升级至 30k checkpoint。
 
-## [0.1.1] - 2025-03-28
+## [0.1.1] - 2026-03-28
 
 - 数据集改为 shard-only 输出。
-- 新增 adaptive_bin 采样。
 - 引入外部资源管理并瘦身仓库。
 
-## [0.1.0] - 2025-03-25
+## [0.1.0] - 2026-03-25
 
 - 首个公开版本。
 - 支持 General-Tracking-G1 全身追踪训练与 ONNX sim2sim 推理。

diff --git a/README.md b/README.md
@@ -32,9 +32,13 @@ pip install -e .
 
 ```bash
 pip install modelscope
-python scripts/setup/download_assets.py --only gmr ckpt bvh
+python scripts/setup/download_assets.py --only robots gmr ckpt bvh
 ```
 
+The canonical Unitree G1 robot model is downloaded to
+`assets/robots/unitree_g1/g1_29dof.xml`. Training, sim2sim, retargeting, and FK
+validation all use this same XML.
+
 **3. Run**
 
 ```bash
@@ -54,12 +58,69 @@ python scripts/run/run_sim.py \
     'viewers=[sim2sim,camera]'
 ```
 
+For sim2real, viewers are disabled by default. Add `viewers=retarget` to show
+the retargeted reference in an optional MuJoCo window.
+
+## Pico Motion Recording
+
+Record many Pico clips as training-ready G1 motion NPZ files:
+
+```bash
+pip install -e '.[pico4]'
+python scripts/run/record_pico_motion.py
+```
+
+The recorder starts the Pico receiver and live Retarget viewer before waiting
+for clip names, so preview keeps running while the terminal is idle. Enter a
+semantic clip name, then use `R` to start, `S` to save, `D` to discard, `N` for
+a new name, and `Q` to quit. Saved clips are written to
+`data/pico_motion/clips/` using the semantic label in the filename, with no
+sidecar JSON.
+
+Merge recorded clips into the standard HDF5 shard dataset:
+
+```bash
+python train_mimic/scripts/data/build_dataset.py \
+    --spec data/pico_motion/pico_recorded.yaml --force
+```
+
+## Sim2Real HDF5 Recording
+
+Pico sim2real can also record manual HDF5 episodes from the real G1:
+
+```bash
+pip install -e '.[recording]'
+# If you use RealSense video, install pyrealsense2 manually for your platform.
+# On Arm machines, prefer conda-forge:
+# conda install -c conda-forge pyrealsense2
+python scripts/run/run_sim2real.py --config-name sim2real_record \
+    controller.policy_path=track.onnx \
+    recording.task="walk forward"
+```
+
+Recording uses the terminal controls `R` start, `S` save, `D` discard, and `Q`
+shutdown. `STANDING`, `MOCAP`, `ARMS`, and paused mocap can be recorded. Saved
+episodes are written as `.h5` files under `data/recordings/sim2real_hdf5/episodes/`.
+`sim2real_record.yaml` stores camera frames as compressed MP4 sidecar files under
+`data/recordings/sim2real_hdf5/videos/` and keeps `frame_index` / `timestamp`
+sync metadata in the HDF5 episode. The low-dimensional HDF5 schema records
+`observation.state(68)`, `observation.mode(1)`, `action(36)` as the aligned
+reference qpos sent to the policy path, and `action.hand(12)` as the latest
+LinkerHand left/right 6D pose commands.
+
 ## Documentation
 
 Full docs at **[BotRunner64.github.io/Teleopit](https://BotRunner64.github.io/Teleopit/)**, covering installation profiles, all tutorials, configuration reference, and architecture.
 
 ## Changelog
 
+### v0.4.0 (2026-06-25)
+
+- Improved Pico realtime control with pico-bridge 0.2.1, `ARMS` mode, and retargeter-preserving mode/pause resets.
+- Added optional LinkerHand L6/O6 sim2real control, including Pico gripper input and low-latency L6 `vr_hand_pose`.
+- Added manual Pico sim2real HDF5 recording and an interactive Pico motion recorder for training NPZ clips.
+- Refined the training data path with minimal HDF5 shards, explicit precompute, rewind sampling, and updated tracking rewards.
+
 ### v0.3.0 (2026-05-12)
 
 - Consolidated realtime input around pico-bridge 0.2.0 and removed the old ZMQ/onboard Pico path.
@@ -73,12 +134,12 @@ Full docs at **[BotRunner64.github.io/Teleopit](https://BotRunner64.github.io/Te
 - Added offline playback keyboard controls, Pico sim2sim mode control, and a standalone standing controller.
 - Improved realtime mocap buffering/catch-up and upgraded the released model to the 30k checkpoint.
 
-### v0.1.1 (2025-03-28)
+### v0.1.1 (2026-03-28)
 
-- Dataset shard-only refactor and `adaptive_bin` sampling
+- Dataset shard-only refactor
 - External asset management (ModelScope), repository slimming
 
-### v0.1.0 (2025-03-25)
+### v0.1.0 (2026-03-25)
 
 - Initial public release: General-Tracking-G1 training, ONNX sim2sim inference, Pico 4 VR teleoperation, Unitree G1 hardware deployment
 

diff --git a/docs/docs/configuration/config-reference.md b/docs/docs/configuration/config-reference.md
@@ -15,7 +15,6 @@ Complete reference for all configurable fields.
 | `viewers` | Viewer set: `mocap`, `retarget`, `sim2sim`, `camera`, `all`, `none`. `all` opens `mocap`, `retarget`, and `sim2sim`; add `camera` explicitly. | `sim2sim` |
 | `realtime` | Rate-limit to wall clock | `false` |
 | `num_steps` | Number of steps; `0` = infinite | `0` |
-| `transition_duration` | Smooth transition time (seconds) from current pose to retarget command | - |
 | `keyboard.enabled` | Enable realtime keyboard mode control for sim2sim | `false` |
 | `playback.pause_on_end` | Pause at last frame when offline motion ends | `false` |
 | `playback.keyboard.enabled` | Enable keyboard control for offline playback | `false` |
@@ -63,13 +62,15 @@ Complete reference for all configurable fields.
 | `input.pico4_buffer_size` | Frame buffer size | `60` |
 | `input.pause_button` | Button for pause/resume | `A` |
 | `input.pause_debounce_s` | Debounce time for pause button | `0.25` |
+| `input.arms_button` | Button for Pico `MOCAP` / `ARMS` toggle | `B` |
+| `input.arms_debounce_s` | Debounce time for arms-mode button | `0.25` |
 | `input.bridge_host` | Teleopit host receiver bind host | `0.0.0.0` |
 | `input.bridge_port` | Teleopit host receiver TCP/UDP port | `63901` |
 | `input.bridge_discovery` | Enable pico-bridge discovery advertising | `true` |
 | `input.bridge_advertise_ip` | Optional advertised host IP override | `null` |
 | `input.bridge_start_timeout` | Timeout while starting the bridge | `10.0` |
 | `input.bridge_history_size` | Pico frame history retained by the bridge | `120` |
-| `input.video.enabled` | Stream host camera preview back to Pico through pico-bridge 0.2.0 | `false` |
+| `input.video.enabled` | Stream host camera preview back to Pico through pico-bridge 0.2.1 | `false` |
 | `input.video.source` | Video source: `mujoco`, `realsense`, or `test-pattern` | `null` |
 | `input.video.width` / `height` / `fps` | Video capture/render settings | `1280` / `720` / `30` |
 | `input.video.device` | Optional RealSense serial | `null` |
@@ -84,23 +85,25 @@ Complete reference for all configurable fields.
 | `retarget_buffer_delay_s` | Buffer delay |
 | `reference_steps` | Reference window steps |
 | `realtime_buffer_warmup_steps` | Warmup before playback |
-| `realtime_buffer_low_watermark_steps` | Low watermark |
-| `realtime_buffer_high_watermark_steps` | High watermark |
 | `reference_velocity_smoothing_alpha` | Velocity smoothing |
 | `reference_anchor_velocity_smoothing_alpha` | Anchor velocity smoothing |
 
 ## Sim2Real
 
 Fields used by sim2real configs (`sim2real.yaml`, `pico4_sim2real.yaml`).
 
+Sim2real defaults to `viewers=none`. Set `viewers=retarget` to open an optional
+MuJoCo window showing the retargeted reference; `sim2sim`, `mocap`, `camera`,
+and `all` are simulation-only viewer modes.
+
 ### Safety
 
 | Field | Description | Default |
 |-------|-------------|---------|
-| `startup_ramp_duration` | Seconds to smoothly blend from locked to policy positions | `2.0` |
+| `startup_ramp_duration` | Kp ramp duration after entering `STANDING`; gradually increases PD gains without changing policy targets | `2.0` |
 | `joint_vel_limit` | Joint velocity limit (rad/s); triggers emergency damping if exceeded | `10.0` |
 | `mocap_switch.check_frames` | Consecutive valid frames required before switching to MOCAP | `10` |
-| `mocap_switch.max_position_value` | Position sanity threshold in meters | `5.0` |
+| `arm_mocap.controlled_joint_indices` | G1 joints driven by live retargeting in Pico `ARMS` mode | `[15..28]` |
 
 ### Real Robot
 
@@ -116,22 +119,94 @@ Fields used by sim2real configs (`sim2real.yaml`, `pico4_sim2real.yaml`).
 
 ### Pause/Resume (Pico sim2real)
 
-| Field | Description | Default |
-|-------|-------------|---------|
-| `pause_resume_transition_duration` | Resume blend duration for offline playback; realtime Pico resume uses live tracking re-centering | `1.0` |
-| `pause_resume_warmup_steps` | Realtime mocap frames to collect before tracking resumes | `2` |
-
 Realtime Pico resume re-centers heading and ground-plane position before tracking continues. Operators should keep still and stay as close as practical to the paused pose to reduce sudden reference changes.
 
-### Realtime Catch-up (Pico sim2real)
+### Dexterous Hand (Pico sim2real)
+
+`hands.enabled=true` requires `input.provider=pico4` plus local editable
+installs of `third_party/linkerhand-python-sdk` and `third_party/somehand`.
+When enabled, hand control remains active in all sim2real modes.
+`gripper` supports `linkerhand_l6` and `linkerhand_o6` by interpolating Pico
+trigger input between the configured open and close poses. `vr_hand_pose` is
+L6-only: missing hand pose holds the last command for that side, L6 speed is
+set to the maximum, and Teleopit converts Pico hand state to 21 landmarks before
+calling somehand 0.2.0 through `somehand.api` only.
 
 | Field | Description | Default |
 |-------|-------------|---------|
-| `realtime_catchup_enabled` | Enable catch-up when buffer grows too large | `true` |
-| `realtime_catchup_trigger_steps` | Buffer depth that triggers catch-up | `6` |
-| `realtime_catchup_release_steps` | Buffer depth to release catch-up | `3` |
-| `realtime_catchup_target_delay_s` | Target delay for catch-up | `0.04` |
-| `reference_qpos_smoothing_alpha` | Joint position smoothing (1.0 = no smoothing) | `0.4` |
+| `hands.enabled` | Enable optional hand worker | `false` |
+| `hands.driver` | Hand driver plugin: `linkerhand_l6` or `linkerhand_o6` | `linkerhand_l6` |
+| `hands.mode` | `gripper` or `vr_hand_pose` | `gripper` |
+| `hands.sides` | Controlled sides | `[left, right]` |
+| `hands.rate_hz` | Maximum gripper command rate in Hz | `30.0` |
+| `hands.frame_timeout_s` | Controller or hand-pose staleness threshold | `0.3` |
+| `hands.linkerhand_l6.left_can` / `right_can` | CAN channels for each hand | `can0` / `can1` |
+| `hands.linkerhand_l6.speed` | L6 speed used by `gripper`; `vr_hand_pose` overrides this to maximum speed | see config |
+| `hands.linkerhand_l6.open_pose` / `close_pose` | Six-value L6 open/closed poses | see config |
+| `hands.linkerhand_o6.left_can` / `right_can` | CAN channels for each O6 hand | `can0` / `can1` |
+| `hands.linkerhand_o6.speed` | O6 speed used by `gripper` | see config |
+| `hands.linkerhand_o6.open_pose` / `close_pose` | Six-value O6 open/closed poses | see config |
+| `hands.somehand.config_path` | Official somehand 0.2.0 bi-hand L6 config used by `vr_hand_pose` | see config |
+| `hands.somehand.rate_hz` | Low-latency `vr_hand_pose` command rate in Hz | `60.0` |
+| `hands.somehand.max_iterations` | somehand solver iteration cap for `vr_hand_pose` | `12` |
+| `hands.somehand.temporal_filter_alpha` | somehand input landmark smoothing alpha; `1.0` disables smoothing delay | `1.0` |
+| `hands.somehand.output_alpha` | somehand qpos output smoothing alpha; `1.0` disables smoothing delay | `1.0` |
+
+### HDF5 Recording (Pico sim2real)
+
+`recording.enabled=true` is supported only with `input.provider=pico4`,
+`input.video.enabled=true`, `input.video.source=realsense`, and an interactive
+terminal. The recorder is manual: `R` starts an episode, `S` saves the active
+episode, `D` discards the active episode, and `Q` shuts down. `STANDING`,
+`MOCAP`, `ARMS`, and paused mocap can be recorded.
+
+`sim2real_record.yaml` enables both recording and the required RealSense
+`input.video` path. Recording does not open a second camera; it consumes the
+same frames produced by `pico_input`.
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| `recording.enabled` | Enable manual HDF5 recording | `false` |
+| `recording.output_dir` | Dataset root directory | `data/recordings/sim2real_hdf5` |
+| `recording.task` | Task string stored with frames | `demo` |
+| `recording.fps` | Recording/video clock rate | `30` |
+| `recording.min_episode_seconds` | Discard saved episodes shorter than this duration | `1.0` |
+| `recording.record_modes` | Modes that allow recording start and frame writes | `[standing, mocap, arms, pause]` |
+| `recording.camera.key` | RGB image dataset key | `observation.images.d435i_rgb` |
+| `recording.camera.width` / `height` / `fps` | RealSense RGB capture settings | `640` / `480` / `30` |
+| `recording.camera.device` | Optional RealSense serial | `null` |
+| `recording.video.codec` / `quality` / `pixelformat` | MP4 sidecar encoder settings | `libx264` / `8` / `yuv420p` |
+
+Camera failure behavior is controlled by `input.video.fail_on_error`.
+
+Each saved episode has one `.h5` file under `recording.output_dir/episodes/`
+and one compressed MP4 sidecar under
+`recording.output_dir/videos/<camera_key>/`. The HDF5 episode stores
+`frame_index` and `timestamp` arrays, plus `video_path`, `video_fps`, and
+`video_frames` root attributes for synchronization. Raw RGB image datasets are
+not written.
+
+HDF5 datasets:
+
+```text
+frame_index                    int64[N]
+timestamp                      float64[N]
+observation.state              float32[68]
+observation.mode               float32[1]
+action                         float32[36]
+action.hand                    float32[12]
+```
+
+The root attributes include the Teleopit HDF5 recording format, schema version,
+task, fps, frame count, and video sync metadata.
+
+`observation.state` is ordered as `joint_pos(29)`, `joint_vel(29)`,
+`base_quat_wxyz(4)`, `base_ang_vel(3)`, and `projected_gravity(3)`.
+`observation.mode` is a numeric categorical: `standing=0`, `mocap=1`,
+`arms=2`, and `pause=3`. `action` is the current reference qpos:
+`root_pos(3) + root_quat_wxyz(4) + joint_pos(29)`.
+`action.hand` is the latest LinkerHand command from the hand worker:
+`left_pose(6) + right_pose(6)`, using the SDK's 0-255 pose values.
 
 ## Critical: `default_dof_pos`
 

diff --git a/docs/docs/configuration/faq.md b/docs/docs/configuration/faq.md
@@ -7,8 +7,7 @@ sidebar_position: 3
 ## Why does it fail even though I set `policy_path`?
 
 1. Verify the file exists
-2. Confirm it's not an old 1402D / TWIST2 ONNX model
-3. Confirm the input dimension is `166` with dual inputs (`obs` + `obs_history`)
+2. Confirm the input dimension is `167` with dual inputs (`obs` + `obs_history`)
 
 ## Why must I specify `input.bvh_file` explicitly?
 

diff --git a/docs/docs/contributing.md b/docs/docs/contributing.md
@@ -34,7 +34,7 @@ teleopit/           # Core inference & deployment package
 ├── retargeting/    # GMR motion retargeting
 ├── sim/            # SimulationLoop, reference motion utilities
 ├── sim2real/       # Hardware state machines
-├── recording/      # HDF5Recorder
+├── recording/      # Pico motion recording helpers
 ├── runtime/        # Config parsing, factories, external assets
 ├── bus/            # InProcessBus for inter-component communication
 └── configs/        # Hydra YAML configurations

diff --git a/docs/docs/getting-started/download-assets.md b/docs/docs/getting-started/download-assets.md
@@ -20,25 +20,28 @@ python scripts/setup/download_assets.py
 Download only what you need for inference:
 
 ```bash
-python scripts/setup/download_assets.py --only gmr ckpt bvh
+python scripts/setup/download_assets.py --only robots gmr ckpt bvh
 ```
 
 ## Asset Inventory
 
-| Asset | Size | Purpose |
-|-------|------|---------|
-| `track.onnx` | 4 MB | ONNX inference model |
-| `track.pt` | 27 MB | PyTorch checkpoint (for resume training) |
-| `data/datasets/seed/train/shard_*.npz` | ~25 GB | Training dataset |
-| `data/datasets/seed/val/shard_*.npz` | ~1.4 GB | Validation dataset |
-| `data/sample_bvh/*.bvh` | 5 MB | Sample motion files |
-| `teleopit/retargeting/gmr/assets/` | ~1.2 GB | GMR retargeting robot models |
+Downloaded file sizes change as checkpoints, datasets, and asset bundles are updated. Use the repository paths below as the stable contract.
+
+| Local Path | Purpose |
+|------------|---------|
+| `track.onnx` | ONNX inference model |
+| `track.pt` | PyTorch checkpoint for resume training |
+| `data/datasets/seed/shard_*.h5` | Minimal motion dataset; run precompute before training |
+| `data/sample_bvh/*.bvh` | Sample motion files |
+| `assets/robots/unitree_g1/` | Canonical G1 XML and meshes used by training, sim2sim, retargeting, and FK validation |
+| `teleopit/retargeting/gmr/assets/` | GMR retargeting assets, IK configs, and non-canonical robot descriptions |
 
 ## Asset Groups
 
 | Group | ModelScope Repo | Contents |
 |-------|----------------|----------|
 | `ckpt` | `BingqianWu/Teleopit-models` | `track.onnx`, `track.pt` |
+| `robots` | `BingqianWu/Teleopit-models` | Canonical robot XML/meshes |
 | `gmr` | `BingqianWu/Teleopit-models` | GMR retargeting assets |
 | `bvh` | `BingqianWu/Teleopit-models` | Sample BVH motion files |
 | `data` | `BingqianWu/Teleopit-datasets` | Training/validation shards |