README.md

---
library_name: tdmpc2
tags:
  - reinforcement-learning
  - humanoid
  - mujoco
  - humanoid-bench
  - locomotion
  - unitree-h1
  - unitree-g1
  - model-based-rl
  - mpc
datasets:
  - carlosferrazza/humanoid-bench
license: mit
---

# HumanoidBench-TD-MPC2 · 自训通关 checkpoints

_Self-trained TD-MPC2 checkpoints on HumanoidBench locomotion tasks._

> 🛠 **训练源码 / Training source**: <https://github.com/vitorcen/humanoid-training>
> 完整训练脚本、patches、eval harness、分析文档全在 GitHub 配套仓库。
> _Full training scripts, patches, eval harness, and analysis docs in the companion GitHub repo._

TD-MPC2 是 model-based RL 算法，结合 world model + sample-based MPC planning。
本仓库收录在 [HumanoidBench](https://github.com/carlosferrazza/humanoid-bench) 上**从零自训**的 checkpoints。

_TD-MPC2 is a model-based RL algorithm combining a world model with sample-based MPC planning. This repo hosts checkpoints **trained from scratch** on HumanoidBench tasks._

---

## 📊 性能 / Performance

| Task | success_rate | mean_return | N | mean_steps | 备注 |
|---|---|---|---|---|---|
| **`h1-walk-v0`** | **100%** | **816.7** | 3 | 1000/1000 | 训练全程稳定，从 step 800k 起 success=100% |
| **`g1-walk-v0`** | **50%** | **601.7 ± 271.1** | 6 | 755/1000 | 高方差，1/6 集 early fall |

`success_bar = 700` (HumanoidBench locomotion threshold). _Success = episode return ≥ success_bar._

---

## 🎬 演示 / Demos

### H1-walk-v0 (Unitree H1, 19 DoF)

<video controls width="720" src="https://huggingface.co/wsagi/HumanoidBench-TD-MPC2/resolve/main/assets/tdmpc2-h1-walk.mp4"></video>

完整 walking cycle，1000 步不倒。_Full walking cycle, runs through 1000 steps without falling._

### G1-walk-v0 (Unitree G1, 23 DoF with PD + BlockedHands wrappers)

<video controls width="720" src="https://huggingface.co/wsagi/HumanoidBench-TD-MPC2/resolve/main/assets/tdmpc2-g1-walk.mp4"></video>

37D action 屏蔽 14D 手指剩 23D，配 PD 位置控制；含偶发踉跄但 50% 集 ≥ success_bar。
_With 14D fingers masked (37→23 act dim) + PD position control; occasional stumbles but 50% of eps clear the success bar._

### 对比同任务 DR.Q 自训 ([wsagi/HumanoidBench-DR.Q](https://huggingface.co/wsagi/HumanoidBench-DR.Q))

| Task | Algo | Final step | mean_return | success_rate |
|---|---|---|---|---|
| h1-walk-v0 | DR.Q | 500k | 801 | 90% (N=10) |
| h1-walk-v0 | **TD-MPC2** (this) | 950k | **817** | **100%** (N=3) ⭐ |
| g1-walk-v0 | DR.Q PDBH | 500k | 711 | 70% (N=10) |
| g1-walk-v0 | **TD-MPC2** PDBH (this) | 950k | **602** | **50%** (N=6) |

**结论**：TD-MPC2 在 **h1-walk** 上略胜 DR.Q（同 step 范围，更稳定）；在更难的 **g1-walk** (37D + PDBH wrappers) 上落后于 DR.Q，但仍满足 ≥30% 通关阈值。

_TD-MPC2 slightly outperforms DR.Q on h1-walk (more stable); falls behind DR.Q on the harder g1-walk task but still passes the 30% threshold._

---

## 🔧 训练配置 / Training config

| Task | Robot | act_dim | Wrappers | Steps | Hardware | Wall time |
|---|---|---|---|---|---|---|
| `h1-walk-v0` | Unitree H1 | 19 | none | 1M | 4090 24GB | ~24h |
| `g1-walk-v0` | Unitree G1 | 23 | PD + BlockedHands | 1M | AutoDL 4080S 32GB | ~22h (3-seed parallel) |

- **Algorithm**: TD-MPC2 `model_size=5` (small, ~16M params)
- **Seed**: 0 for h1-walk; 0 for g1-walk (best of 3 seeds 0/10/20, multi-seed parallel on same GPU)
- **Multi-seed parallel pattern**: see [feedback_tdmpc2_multiseed.md](https://github.com/vitorcen/humanoid-training/blob/main/.claude/memory/feedback_tdmpc2_multiseed.md) — 3 seeds time-slice one GPU, util 15% → 98%, total throughput 2.7×

### Patches applied to upstream submodules

Both **required** for G1-walk — torque-only G1 will not learn to walk ([memory record](https://github.com/vitorcen/humanoid-training/blob/main/.claude/memory/project_benchmark_validation.md)):

- `patches/g1-pos-control.patch` — replaces torque actuators with PD position actuators
- `patches/humanoid-bench-g1-and-lazy.patch` — BlockedHands wrapper to freeze 14 finger DoFs (irrelevant noise for walk task)
- `patches/tdmpc2-save-agent.patch` — fixes upstream TD-MPC2 to actually save weights every eval (the only patch required for h1-walk)

Apply with `bash patches/apply.sh` from the [training repo](https://github.com/vitorcen/humanoid-training).

---

## 🚀 推理 / Inference

完整 deterministic eval + GUI viewer 脚本：

- `scripts/tdmpc2_eval.py` — N-ep JSONL eval (headless)
- `scripts/tdmpc2_viewer.py` — GUI viewer (GLFW)

均在 [配套 GitHub 仓库](https://github.com/vitorcen/humanoid-training/tree/main/scripts)。

```bash
# headless N=10 eval
DISPLAY=:0 conda run -n humanoidbench python scripts/tdmpc2_eval.py \
    --task humanoid_g1-walk-v0 \
    --ckpt TDMPC2+HBench-g1-walk-v0+0/step_950000.pt \
    --seed 0 --eval 10 --out g1_eval.jsonl

# GUI replay
DISPLAY=:0 conda run -n humanoidbench python scripts/tdmpc2_viewer.py \
    --task humanoid_g1-walk-v0 \
    --ckpt TDMPC2+HBench-g1-walk-v0+0/step_950000.pt \
    --seed 0 --fps 50
```

---

## 📁 仓库结构 / Repo layout

```
HumanoidBench-TD-MPC2/
├── README.md                                  (this file)
├── assets/
│   ├── tdmpc2-h1-walk.mp4                     (515 KB — H1-walk GUI recording)
│   └── tdmpc2-g1-walk.mp4                     (257 KB — G1-walk GUI recording)
├── TDMPC2+HBench-h1-walk-v0+0/
│   ├── step_950000.pt                         (32 MB — agent + world model + critic)
│   ├── train.log                              (~370 KB — full training log)
│   └── ckpt_eval.csv                          (auto-eval per ckpt, N=3 quick)
└── TDMPC2+HBench-g1-walk-v0+0/
    ├── step_950000.pt                         (32 MB)
    └── train.log                              (~700 KB)
```

`+0` 表示 seed=0。后续如果发其他 seed 会按 `+10` / `+20` 命名。

---

## 📜 License & Attribution

- **Code**: MIT (consistent with [TD-MPC2](https://github.com/nicklashansen/tdmpc2) and [HumanoidBench](https://github.com/carlosferrazza/humanoid-bench) upstream)
- **Algorithm**: [TD-MPC2 (Hansen et al., 2024)](https://www.tdmpc2.com/)
- **Benchmark**: [HumanoidBench (Sferrazza et al., 2024)](https://arxiv.org/abs/2403.10506)
- **Trained by**: <https://github.com/vitorcen> on AutoDL infrastructure