rl_training

IsaacLab manager-based G1 tasks, trained with rl_algos (plain PPO on shaped rewards). No RSL-RL, no AMP.

Four gaits, ported from the Direct-workflow shape_shift_g1 into IsaacLab's manager-based workflow: walk, crouch-walk, knee-walk, crawl.

The split

Package	Role
rl_algos	Pure PyTorch. Algorithms, models, storage, runner. Declares the VecEnv contract; never imports a simulator.
rl_training (here)	The simulator side. Implements that contract against IsaacLab, plus the G1 asset, tasks, and scripts.

rl_algos consumes plain dicts; @configclass lives here and is converted at the boundary by to_runner_dict.

Tasks

Task id	Gait	Height cmd	Contact bodies
RlAlgos-G1-Walk-v0	upright walking	fixed ~0.78	feet (ankles)
RlAlgos-G1-Crouch-v0	walk at a commanded, varying low height	0.47–0.78 m	feet (ankles)
RlAlgos-G1-KneeWalk-v0	walking on the knees/shins	0.38–0.48 m	knees
RlAlgos-G1-Crawl-v0	on hands and knees, torso horizontal	fixed 0.38	hands + knees

Each has a -Play-v0 variant (32 envs, no noise, fixed command) for viewing a trained policy.

Quick start

python scripts/train.py --task RlAlgos-G1-Walk-v0     --num_envs 4096 --headless
python scripts/train.py --task RlAlgos-G1-Crouch-v0   --num_envs 4096 --headless
python scripts/train.py --task RlAlgos-G1-KneeWalk-v0 --num_envs 4096 --headless
python scripts/train.py --task RlAlgos-G1-Crawl-v0    --num_envs 4096 --headless

python scripts/play.py  --task RlAlgos-G1-Crouch-Play-v0 --num_envs 32

How the gaits differ

They share one base env (tasks/g1/base_env_cfg.py): scene, actions, observation groups, domain randomization, and a 4-channel command [vx, vy, yaw_rate, base_height]. Each gait subclasses it and changes only:

• the reward config — the shaped terms that define the gait,
• the start pose — walk/crouch start standing; knee-walk starts kneeling; crawl starts on all fours (assets/g1.py),
• the command's base_height range — the crouch depth / kneeling / crawl height,
• which bodies are illegal to touch — knee-walk forbids torso+arms, crawl forbids torso+head+shoulders+elbows.

The base-height command (mdp/commands.py) is what makes crouch-walk crouch: it extends IsaacLab's velocity command with a sampled target height that the policy observes and track_base_height_exp rewards. A varying command means one crouch policy walks at any depth.

Crawling additionally tracks velocity in the torso's own frame (track_crawl_lin_vel_xy_exp) because the torso is pitched ~90°, and holds the torso horizontal via body_orientation_target_l2 with a sideways target gravity.

Joint / body order — deployment-ready

Everything here shares the same 29-DOF G1 (g1.usd, the same asset shape_shift_g1 uses), so joint order is identical across all four gaits, and the alternate poses are regex-on-joint-name so they transfer cleanly.

[!success] Our joint order + gains == shape_shift == the sim2real deployment rl_training spawns shape_shift's USD with shape_shift's actuator gains (ImplicitActuator, Kp/Kd = deployment kps/kds). Verified in sim: rl_training order == shape_shift == deployment g1.yaml. So an exported model uses shape_shift's joint2motor_idx / kps / kds / default_joint_pos unchanged — no remapping. Full tables (joint order, SDK motor order, gains) in JOINT_ORDER.md; regenerate with python scripts/verify_joint_order.py --headless.

[!warning] Contact rewards and preserve_order Knee-walk and crawl have rewards that index limbs positionally (e.g. [left_hand, right_hand, left_knee, right_knee]). IsaacLab's SceneEntityCfg sorts body ids by default, which would silently pair the wrong limbs. Those configs set preserve_order=True and use single-body names (not .*wrist.*, which matches three links). If you add a positional contact reward, do the same.

Layout

rl_training/
  assets/                   G1 USD (shape_shift's g1.usd); standing / kneeling / crawl poses + gains
  wrappers/vecenv_wrapper   implements rl_algos.env.VecEnv (the bridge)
  wrappers/rl_cfg           @configclass -> plain dict
  mdp/commands              4-channel velocity + base-height command
  mdp/rewards               every shaped reward IsaacLab doesn't ship (~25 terms)
  tasks/g1/
    base_env_cfg            shared scene / obs / actions / command / events
    walk_cfg, crouch_cfg, knee_cfg, crawl_cfg   one gait each
    agents                  plain-PPO runner config
scripts/                    train, play, smoke_test

Testing

pytest tests/                                              # CPU, no simulator
python scripts/smoke_test.py --task RlAlgos-G1-Crawl-v0 --headless   # real sim

The smoke test builds a gait, checks the wrapper contract (obs_format, 4-channel command, 2-D rewards, time-outs), and trains a couple of iterations. All four gaits pass.

Note on the retargeting scripts

scripts/{analyze_skeleton,inspect_motion}.py are standalone mocap-inspection tools left over from the earlier AMP exploration; nothing in the training path uses them. The IK retargeting scripts (retarget_to_g1, render_g1_motion) were removed when the URDF was dropped — pytorch_kinematics needs a URDF and can't parse the USD, so they could no longer run. If you ever want mocap retargeting again you'd re-add a G1 URDF for FK.

Not implemented yet

Rough terrain / height scanning (shape_shift's *RoughEnvCfg variants), recurrent policies, camera input, sim2real export.