MimicKit-G1-LAFAN

5 motion-tracking policies for Unitree G1 (29-DoF) trained with MimicKit DeepMimic-style PPO on LAFAN1 retargeted slices: fight, run, dance, jumps (15 s each), plus a longer 30 s dance (dance1_subject2) that warm-starts from the 15 s dance and holds full-horizon tracking. Single 4090 24G, ~1 h per 15 s motion, 4096 envs.

Project repo (源码 / 复现入口):

• 🧪 vitorcen/isaaclab-experience — MimicKit training scripts, eval chain, USD material fix, design docs

---

TL;DR

3/4 of the 15 s motions reach ship quality at iter 1500. run plateaus at ~63 %, likely needing ADD-style residual or curriculum sequencing — kept as a baseline. The 30 s dance doubles the horizon (900 frames @ 30 fps): warm-started from the 15 s dance ckpt and run to 2500 iters, its converged Test_Return (244) actually exceeds the 15 s dance (227) — the discounted return saturates near the same ceiling regardless of clip length, so matching/exceeding it means full-horizon coverage held.

---

Demos (eval, 4 envs, with restored per-link material)

30 s dance (longer-horizon, dance1_subject2) — DeepMimic PPO holds the full 30 s; an AMP baseline on the same clip could not keep rhythm (see notes below).

Student renders use the per-link material fix (assets/g1_textured.usd) — the shipped MimicKit g1.usd collapses every MJCF <geom rgba> into a single white DefaultMaterial. We re-parse g1.xml, build one UsdPreviewSurface per unique color, and rebind 36 mesh prims via UsdShade.MaterialBindingAPI. Details in the repo's scripts/g1_usd_recolor.py.

---

Dataset chain

Slicing rule: pick the most representative central segment of each *1 clip (skip start/end idle), 450 frames @ 30 fps = 15 s.

---

Files

MimicKit-G1-LAFAN/
├── README.md
├── videos/
│   ├── dance1s2.mp4 (0.7 MB, 30 s dance, first 8 s)
│   ├── fight.mp4   (6.9 MB)
│   ├── run.mp4     (2.6 MB)
│   ├── dance.mp4   (1.9 MB)
│   └── jumps.mp4   (4.7 MB)
├── fight_15s/
│   ├── model_0000001500.pt          # 11 MB
│   ├── env.yaml                      # train_lafan_fight_15s_env.yaml
│   └── motion.pkl                    # 450-frame LAFAN slice
├── run_15s/   …
├── dance_15s/ …
├── jumps_15s/ …
├── dance_30s/                        # longer-horizon dance (dance1_subject2)
│   ├── model.pt                      # 11 MB, final (2500 iters, warm-started)
│   ├── env.yaml                      # 900-frame / 30 s slice
│   └── motion.pkl
└── assets/
    └── g1_textured.usd               # 26 MB, per-link material restored

---

Training recipe (all 4 identical)

See <motion>/env.yaml for the full Isaac Lab env spec and data/agents/deepmimic_g1_ppo_agent.yaml upstream for PPO config.

---

Reproduce

git clone https://github.com/vitorcen/isaaclab-experience
cd isaaclab-experience && git submodule update --init dependencies/MimicKit

# 1. Pull LAFAN G1 retargeting npz (from ember-lab-berkeley)
huggingface-cli download ember-lab-berkeley/LAFAN-G1 --repo-type dataset \
  --local-dir dependencies/MimicKit/data/motions/g1_extra/ember_lab

# 2. Convert + slice (4 × 450-frame center clips)
python scripts/lafan_g1_npz_to_mimickit.py  # see script for slice ranges

# 3. (optional) Restore per-link material on the student USD
python scripts/g1_usd_recolor.py
export MIMICKIT_G1_USD=$PWD/dependencies/MimicKit/data/assets/g1/g1_textured.usd

# 4. Train all 4 motions sequentially (~4 h total on a 4090)
scripts/mimickit_train_queue.sh

# 5. Eval chain (sequential GUI windows)
scripts/mimickit_eval_chain.sh

---

Known limitations

• run plateau at 63 %: the LAFAN run clip has fast contact + slip; vanilla DeepMimic reward + fixed action std saturates here. Likely fixes: ADD residual, motion-curriculum from walk, or larger action std at start.
• Single-clip overfit: each policy tracks one clip; no multi-motion conditioning. For composition, see ProtoMotions / OmniH2O.
• AMP vs phase-tracking on dance: an AMP (adversarial motion prior) baseline on the full 131 s dance1_subject2 failed — the discriminator plateaued at ~0.98 agent-accuracy and the policy could not keep the choreography's rhythm. Phase-conditioned DeepMimic tracking (used here) is the right tool for high-fidelity dance; AMP fits continuous/loopable skills that don't require exact timing. The 30 s dance is the DeepMimic answer to "longer dance."
• No sim-to-real transfer attempted: trained in Isaac Lab with raw observations, no domain randomization, no actuator delay model.

---

License

• Code & policy weights: Apache-2.0
• LAFAN1 motion data: CC-BY-NC 4.0 (Ubisoft Montreal) — non-commercial only
• G1 USD assets: Unitree

Citation

Upstream papers (cite these instead — this repo is just trained weights, not a publication):

@inproceedings{peng2018deepmimic,
  title     = {DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills},
  author    = {Peng, Xue Bin and Abbeel, Pieter and Levine, Sergey and van de Panne, Michiel},
  booktitle = {ACM Trans. Graph. (SIGGRAPH)},
  year      = {2018}
}

@article{harvey2020robust,
  title   = {Robust Motion In-betweening},
  author  = {Harvey, F{\'e}lix G. and Yurick, Mike and Nowrouzezahrai, Derek and Pal, Christopher},
  journal = {ACM Trans. Graph. (SIGGRAPH)},
  year    = {2020}
}