| --- |
| license: apache-2.0 |
| base_model: nvidia/GR00T-N1.6-3B |
| pipeline_tag: robotics |
| tags: [robotics, vla, gr00t, gr00t-n1.6, manipulation, maniguard, franka] |
| --- |
| |
| # GR00T-N1.6 - Sim2Real Clutter (task_0048) (joint, 2-cam) |
| |
| NVIDIA Isaac **GR00T-N1.6-3B** fine-tuned on the ManiGuard **clutter-task0048** base task (sim Franka Panda). |
| Part of the ManiGuard VLA benchmark - GR00T vs pi0.5 on the same task families with |
| identical data, cameras, and controller. |
| |
| ## Model |
| - **Base:** [nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B) - Eagle (nvidia/Eagle-Block2A-2B-v2) VLM + flow-matching DiT action head |
| - **Embodiment:** NEW_EMBODIMENT - Franka Panda, **8-D joint** state/action (7 arm joints + 1 gripper) |
| - **Cameras (2):** image_left (overview) + wrist (256x256) |
| - **Action:** arm = state-relative chunks (reconstructed to absolute at inference), gripper = absolute; 16-step horizon at 30 fps (0.53 s); NON_EEF (joint space) |
| - **Tuning:** GR00T-N1.6 default - VLM (LLM + visual) **frozen**, train projector + diffusion action head (**no LoRA**) |
|
|
| ## Training |
| - single-card config, bf16, global batch 4, 50000 steps (~8 epochs over 23,904 frames), cosine LR (peak 2.5e-5, sqrt-scaled), warmup 0.05 |
| - Data: [IDEAS-Lab-Northwestern/sim2real-clutter-task0048-sim-joint-5cam](https://huggingface.co/datasets/IDEAS-Lab-Northwestern/sim2real-clutter-task0048-sim-joint-5cam); videos decoded as H.264 for GR00T's torchcodec loader |
|
|
| ## Usage |
| Load with `Gr00tPolicy` from [Isaac-GR00T (n1d6)](https://github.com/NVIDIA/Isaac-GR00T/tree/n1d6), `--embodiment-tag NEW_EMBODIMENT`. The included `processor/` carries the normalization stats + modality config. |
|
|
| > WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets, NON_EEF) + 2 cameras (image_left overview + wrist). A mismatched controller or camera set silently feeds an out-of-distribution input. |
|
|
