README.md

---
license: apache-2.0
library_name: mimickit
tags:
  - motion-tracking
  - deepmimic
  - humanoid
  - unitree-g1
  - lafan
  - ppo
  - reinforcement-learning
  - isaac-lab
datasets:
  - lvhaidong/LAFAN1_Retargeting_Dataset
  - ember-lab-berkeley/LAFAN-G1
base_model: []
pipeline_tag: reinforcement-learning
---

# MimicKit-G1-LAFAN

5 motion-tracking policies for Unitree G1 (29-DoF) trained with [MimicKit](https://github.com/xbpeng/MimicKit) DeepMimic-style PPO on LAFAN1 retargeted slices: **fight, run, dance, jumps** (15 s each), plus a longer **30 s dance** (`dance1_subject2`) that warm-starts from the 15 s dance and holds full-horizon tracking. Single 4090 24G, ~1 h per 15 s motion, 4096 envs.

**Project repo (源码 / 复现入口):**
- 🧪 [vitorcen/isaaclab-experience](https://github.com/vitorcen/isaaclab-experience) — MimicKit training scripts, eval chain, USD material fix, design docs

---

## TL;DR

| Motion | Episode-Length (终态 / max=15s) | Ship quality | Frames slice |
|---|---|---|---|
| **fight** | 14.85 s / **99.0 %** | 🟢 触顶 | LAFAN `fight1_subject2` [600:1050] |
| **dance** | 14.70 s / **98.0 %** | 🟢 触顶 | LAFAN `dance1_subject1` [1746:2196] |
| **jumps** | 14.70 s / **98.0 %** | 🟢 触顶 | LAFAN `jumps1_subject1` [3441:3891] |
| **run**   | 9.45 s / **63.0 %**  | 🟡 plateau | LAFAN `run1_subject2` [3341:3791] |
| **dance (30 s)** | full 30 s, `Test_Return` **244** (>15 s base 227) | 🟢 longer-horizon | LAFAN `dance1_subject2` [1521:2421] |

3/4 of the 15 s motions reach ship quality at iter 1500. `run` plateaus at ~63 %, likely needing ADD-style residual or curriculum sequencing — kept as a baseline. The **30 s dance** doubles the horizon (900 frames @ 30 fps): warm-started from the 15 s dance ckpt and run to 2500 iters, its converged `Test_Return` (244) actually exceeds the 15 s dance (227) — the discounted return saturates near the same ceiling regardless of clip length, so matching/exceeding it means full-horizon coverage held.

---

## Demos (eval, 4 envs, with restored per-link material)

**30 s dance (longer-horizon, `dance1_subject2`)** — DeepMimic PPO holds the full 30 s; an AMP baseline on the same clip could not keep rhythm (see notes below).

<video controls width="640" src="https://huggingface.co/wsagi/MimicKit-G1-LAFAN/resolve/main/videos/dance1s2.mp4"></video>

<video controls width="640" src="https://huggingface.co/wsagi/MimicKit-G1-LAFAN/resolve/main/videos/fight.mp4"></video>

<video controls width="640" src="https://huggingface.co/wsagi/MimicKit-G1-LAFAN/resolve/main/videos/dance.mp4"></video>

<video controls width="640" src="https://huggingface.co/wsagi/MimicKit-G1-LAFAN/resolve/main/videos/jumps.mp4"></video>

<video controls width="640" src="https://huggingface.co/wsagi/MimicKit-G1-LAFAN/resolve/main/videos/run.mp4"></video>

> Student renders use the **per-link material fix** (`assets/g1_textured.usd`) — the shipped MimicKit `g1.usd` collapses every MJCF `<geom rgba>` into a single white `DefaultMaterial`. We re-parse `g1.xml`, build one `UsdPreviewSurface` per unique color, and rebind 36 mesh prims via `UsdShade.MaterialBindingAPI`. Details in the repo's `scripts/g1_usd_recolor.py`.

---

## Dataset chain

| Layer | Source | What it provides |
|---|---|---|
| Original mocap | [ubisoft-la-forge/lafan1](https://github.com/ubisoft/ubisoft-laforge-animation-dataset) (CC-BY-NC 4.0) | Raw Ubisoft Montreal BVH |
| G1 retargeting | [lvhaidong/LAFAN1_Retargeting_Dataset](https://huggingface.co/datasets/lvhaidong/LAFAN1_Retargeting_Dataset) | Unitree-style G1/H1/H1_2 retarget (community-mirrored — original unitreerobotics repo is currently offline) |
| IsaacLab AMP repack | [ember-lab-berkeley/LAFAN-G1](https://huggingface.co/datasets/ember-lab-berkeley/LAFAN-G1) | Berkeley npz repackaging (CC-BY 4.0) |
| **MimicKit slices (this repo)** | `scripts/lafan_g1_npz_to_mimickit.py` | 30 fps → MimicKit pkl, 450-frame center slices |

Slicing rule: pick the most representative central segment of each `*1` clip (skip start/end idle), 450 frames @ 30 fps = 15 s.

---

## Files

```
MimicKit-G1-LAFAN/
├── README.md
├── videos/
│   ├── dance1s2.mp4 (0.7 MB, 30 s dance, first 8 s)
│   ├── fight.mp4   (6.9 MB)
│   ├── run.mp4     (2.6 MB)
│   ├── dance.mp4   (1.9 MB)
│   └── jumps.mp4   (4.7 MB)
├── fight_15s/
│   ├── model_0000001500.pt          # 11 MB
│   ├── env.yaml                      # train_lafan_fight_15s_env.yaml
│   └── motion.pkl                    # 450-frame LAFAN slice
├── run_15s/   …
├── dance_15s/ …
├── jumps_15s/ …
├── dance_30s/                        # longer-horizon dance (dance1_subject2)
│   ├── model.pt                      # 11 MB, final (2500 iters, warm-started)
│   ├── env.yaml                      # 900-frame / 30 s slice
│   └── motion.pkl
└── assets/
    └── g1_textured.usd               # 26 MB, per-link material restored
```

---

## Training recipe (all 4 identical)

| Hyperparam | Value |
|---|---|
| Algorithm | DeepMimic PPO (MimicKit default) |
| Optimizer | SGD, lr=1e-4, momentum=0.9 |
| Num envs | 4096 (Isaac Lab, RTX 4090 24 G) |
| Iters | 1500 |
| Action std (Gaussian policy) | 0.05 |
| Reward | DeepMimic style: pose + vel + ee + root + COM, no AMP discriminator |
| Termination | head fall, knee contact, large pose error |
| Wallclock | ~1 h / motion |

See `<motion>/env.yaml` for the full Isaac Lab env spec and `data/agents/deepmimic_g1_ppo_agent.yaml` upstream for PPO config.

---

## Reproduce

```bash
git clone https://github.com/vitorcen/isaaclab-experience
cd isaaclab-experience && git submodule update --init dependencies/MimicKit

# 1. Pull LAFAN G1 retargeting npz (from ember-lab-berkeley)
huggingface-cli download ember-lab-berkeley/LAFAN-G1 --repo-type dataset \
  --local-dir dependencies/MimicKit/data/motions/g1_extra/ember_lab

# 2. Convert + slice (4 × 450-frame center clips)
python scripts/lafan_g1_npz_to_mimickit.py  # see script for slice ranges

# 3. (optional) Restore per-link material on the student USD
python scripts/g1_usd_recolor.py
export MIMICKIT_G1_USD=$PWD/dependencies/MimicKit/data/assets/g1/g1_textured.usd

# 4. Train all 4 motions sequentially (~4 h total on a 4090)
scripts/mimickit_train_queue.sh

# 5. Eval chain (sequential GUI windows)
scripts/mimickit_eval_chain.sh
```

---

## Known limitations

- **`run` plateau at 63 %**: the LAFAN run clip has fast contact + slip; vanilla DeepMimic reward + fixed action std saturates here. Likely fixes: ADD residual, motion-curriculum from walk, or larger action std at start.
- **Single-clip overfit**: each policy tracks one clip; no multi-motion conditioning. For composition, see ProtoMotions / OmniH2O.
- **AMP vs phase-tracking on dance**: an AMP (adversarial motion prior) baseline on the full 131 s `dance1_subject2` failed — the discriminator plateaued at ~0.98 agent-accuracy and the policy could not keep the choreography's rhythm. Phase-conditioned DeepMimic tracking (used here) is the right tool for high-fidelity dance; AMP fits continuous/loopable skills that don't require exact timing. The 30 s dance is the DeepMimic answer to "longer dance."
- **No sim-to-real transfer attempted**: trained in Isaac Lab with raw observations, no domain randomization, no actuator delay model.

---

## License

- Code & policy weights: Apache-2.0
- LAFAN1 motion data: CC-BY-NC 4.0 (Ubisoft Montreal) — non-commercial only
- G1 USD assets: Unitree

## Citation

Upstream papers (cite these instead — this repo is just trained weights, not a publication):

```bibtex
@inproceedings{peng2018deepmimic,
  title     = {DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills},
  author    = {Peng, Xue Bin and Abbeel, Pieter and Levine, Sergey and van de Panne, Michiel},
  booktitle = {ACM Trans. Graph. (SIGGRAPH)},
  year      = {2018}
}

@article{harvey2020robust,
  title   = {Robust Motion In-betweening},
  author  = {Harvey, F{\'e}lix G. and Yurick, Mike and Nowrouzezahrai, Derek and Pal, Christopher},
  journal = {ACM Trans. Graph. (SIGGRAPH)},
  year    = {2020}
}
```