mujoco-ant

Minimal RL project for training and visualizing a MuJoCo Ant agent with SAC (Stable-Baselines3).

A minimal Gymnasium MuJoCo Ant + Stable-Baselines3 SAC setup.

Default task is AntGoal-v4: each episode samples a red target on the map (4–10 m away). The ant sees the vector to that target and is rewarded for closing the distance. Use --env Ant-v4 for the original run-forward task.

Install (Python 3.12 required)

This project requires Python 3.12.

If python3.12 is not installed yet (Ubuntu):

sudo apt update
sudo apt install -y python3.12 python3.12-venv

Create and activate the virtual environment with Python 3.12:

python3.12 -m venv myenv
source myenv/bin/activate
python -m pip install --upgrade pip
pip install "gymnasium[mujoco]" stable-baselines3 torch tensorboard

Train (saved model + stats)

python3 train.py \
  --env AntGoal-v4 \
  --timesteps 1000000 \
  --n-envs 8 \
  --vec-env subproc \
  --device cuda \
  --mujoco-gl glfw \
  --model-path models/ant \
  --vecnorm-path models/ant_vecnormalize.pkl

Notes:

• Defaults: --env AntGoal-v4, --device auto (uses CUDA if available), --mujoco-gl glfw.
• Original forward-locomotion: --env Ant-v4.
• Parallel rollout collection: --n-envs + --vec-env subproc.
• Outputs: <model-path>.zip and <vecnorm-path>.
• Training UI: clean one-line progress bar (use --no-progress to disable).
• Fast default: --timesteps 10000 (~100x less than 1,000,000).
• Training ends when --timesteps is reached, then model/stats are saved.
• Post-train eval defaults to --eval-episodes 1 (--eval-episodes 0 skips eval).

Render (load saved artifacts)

python3 render.py --env AntGoal-v4 --model-path models/ant --vecnorm-path models/ant_vecnormalize.pkl --episodes 3
python3 save_walk.py

Notes:

• render.py defaults to --render-mode auto:
  • Uses human when DISPLAY is available.
  • Uses rgb_array in headless environments.
• If artifacts are missing, run train.py first (or pass existing paths).