File size: 2,940 Bytes
c6a5d52 7adbe1b c6a5d52 726c193 c6a5d52 726c193 c6a5d52 726c193 c6a5d52 726c193 c6a5d52 7adbe1b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
license: other
license_name: nvidia-oneway-noncommercial-license
license_link: https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf
base_model: nvidia/GR00T-N1.6-3B
pipeline_tag: robotics
tags: [robotics, vla, gr00t, gr00t-n1.6, manipulation, maniguard, franka]
---
# GR00T-N1.6 - Sim2Real Clutter (task_0048) (joint, 2-cam)
NVIDIA Isaac **GR00T-N1.6-3B** fine-tuned on the ManiGuard **clutter-task0048** base task (sim Franka Panda).
Part of the ManiGuard VLA benchmark - GR00T vs pi0.5 on the same task families with
identical data, cameras, and controller.
## Model
- **Base:** [nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B) - Eagle (nvidia/Eagle-Block2A-2B-v2) VLM + flow-matching DiT action head
- **Embodiment:** NEW_EMBODIMENT - Franka Panda, **8-D joint** state/action (7 arm joints + 1 gripper)
- **Cameras (2):** image_left (overview) + wrist (256x256)
- **Action:** arm = state-relative chunks (reconstructed to absolute at inference), gripper = absolute; 16-step horizon at 30 fps (0.53 s); NON_EEF (joint space)
- **Tuning:** GR00T-N1.6 default - VLM (LLM + visual) **frozen**, train projector + diffusion action head (**no LoRA**)
## Training
- single-card config, bf16, global batch 4, 50000 steps (~8 epochs over 23,904 frames), cosine LR (peak 2.5e-5, sqrt-scaled), warmup 0.05
- Data: [IDEAS-Lab-Northwestern/sim2real-clutter-task0048-sim-joint-5cam](https://huggingface.co/datasets/IDEAS-Lab-Northwestern/sim2real-clutter-task0048-sim-joint-5cam); videos decoded as H.264 for GR00T's torchcodec loader
## Usage
Load with `Gr00tPolicy` from [Isaac-GR00T (n1d6)](https://github.com/NVIDIA/Isaac-GR00T/tree/n1d6), `--embodiment-tag NEW_EMBODIMENT`. The included `processor/` carries the normalization stats + modality config.
> WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets, NON_EEF) + 2 cameras (image_left overview + wrist). A mismatched controller or camera set silently feeds an out-of-distribution input.
## Paper & Citation
Part of **ManiGuard**: [paper (arXiv:2608.17386)](https://arxiv.org/abs/2608.17386) ·
[code](https://github.com/NU-IDEAS-Lab/ManiGuard) ·
[docs](https://nu-ideas-lab.github.io/ManiGuard/)
```bibtex
@misc{peng2026maniguard,
title = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
Safety Evaluation and Improvement of Robotic Manipulation},
author = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
and Zhang, Ruohan and Zhu, Qi},
year = {2026},
eprint = {2608.17386},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2608.17386},
}
```
|