| --- |
| license: other |
| license_name: nvidia-open-model-license |
| license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ |
| tags: [robotics, humanoid, unitree-g1, whole-body-control, loco-manipulation] |
| --- |
| # H2R box loco-manipulation policies for Unitree G1 (R15, P23) |
|
|
| Residual policies on top of frozen **SONIC** (NVIDIA GR00T-WholeBodyControl) for walking up to a box, picking it |
| up, carrying and placing it on a Unitree G1 with rubber hands (R15) or Dex3 hands (P23). Trained with PPO in Isaac Lab |
| on retargeted OMOMO `largebox` motions (paired clean / corrupted references) only. |
|
|
| | folder | policy | box input | notes | |
| |---|---|---|---| |
| | `r15/` | output residual over frozen SONIC, rubber hands | **box estimator** (deployable: virtual camera + odometry, no simulator truth) | step 15000 | |
| | `p23/` | output residual over frozen SONIC, Dex3 hands | simulator box pose (privileged) | step 15000 | |
|
|
| Each folder: `model_step_015000.pt` (policy, value and optimizer state; includes the frozen SONIC weights) and the |
| `config.yaml` it was trained with. Load with the GR00T-WholeBodyControl fork used for training |
| (`scripts_sonic_render/eval/run_eval.py --checkpoint <folder>/model_step_015000.pt ...`, which reads the |
| `config.yaml` next to the checkpoint). |
|
|
| ## Reference results (no steering) |
| HOI2ROI feasible43 (43 out-of-distribution pick-carry-place cases × 8 episodes), full-size largebox: |
|
|
| | policy | pickup | success (pickup and box ≤ 30 cm from goal) | |
| |---|---|---| |
| | R15, box estimator (emulator) | 29.4% | 20.1% | |
| | R15, real camera pipeline (sim) | 29.7% | 20.9% | |
| | P23 | 20.3% | 11.3% | |
|
|
| Caveat: the HOI2ROI motions were generated for a box scaled ×0.76 (~25 cm); these numbers use the full-size box. |
| R15 is box-size specific (trained on one box only). |
|
|
| ## degen_v2 step 19000 (`degen_v2_19k/`) - best HOI2ROI checkpoint so far |
| Same architecture as R15 (rubber hands, box estimator inputs), trained on OMOMO `largebox` motions at box scale 0.7601 |
| (box-mix place-A corpus + degenerate-grasp reference families), no point cloud. Evaluated as a **hybrid**: base SONIC |
| until a planned wrist is within 0.55 m of the box, the residual after that, plus pace token steering with the |
| base-SONIC basis from [`daibfc/h2r-steering`](https://huggingface.co/daibfc/h2r-steering) (`bases/A_sonic_omomo_iso64.npz`). |
|
|
| | HOI2ROI feasible43, box scale 0.7601, 344 episodes | pickup | success | |
| |---|---|---| |
| | degen_v2 19k hybrid | 58.7% | 40.7% | |
| |
| Reproduce: `scripts_sonic_render/benchmark_hybrid.sh` and `docs/DEGEN_V2_HYBRID.md` in the GR00T-WholeBodyControl fork. |
|
|
| ## License |
| Model weights are derived from NVIDIA SONIC (GR00T-WholeBodyControl) and are distributed under the |
| [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
| Training motions derive from the OMOMO dataset; no motion data is included here. |
|
|
