IDEAS-Lab-Northwestern/gr00t-n16-real-clutter-60-refined

GR00T-N1.6 - Real Clutter (joint, 2-cam)

NVIDIA Isaac GR00T-N1.6-3B fine-tuned on the ManiGuard real-clutter base task (real Franka Panda, DROID-schema teleop). Part of the ManiGuard VLA benchmark - GR00T vs pi0.5 on the same task families with identical data, cameras, and controller.

Model

• Base: nvidia/GR00T-N1.6-3B - Eagle (nvidia/Eagle-Block2A-2B-v2) VLM + flow-matching DiT action head
• Embodiment: NEW_EMBODIMENT - Franka Panda, 8-D joint state/action (7 arm joints + 1 gripper)
• Cameras (2): image_left (overview) + wrist (180x320, 16:9 centre-cropped at conversion time)
• Action: arm = joint VELOCITY (rad/s, ABSOLUTE representation - no state delta is applied), gripper = next-frame target; 16-step horizon at 15 fps (1.07 s); NON_EEF (joint space)
• Tuning: GR00T-N1.6 default - VLM (LLM + visual) frozen, train projector + diffusion action head (no LoRA)

Training

• single-card config, bf16, global batch 4, 50000 steps (~15 epochs over 13,171 frames), cosine LR (peak 2.5e-5, sqrt-scaled), warmup 0.05
• Data: IDEAS-Lab-Northwestern/real-clutter-60-droid-refined; videos decoded as H.264 for GR00T's torchcodec loader

Usage

Load with Gr00tPolicy from Isaac-GR00T (n1d6), --embodiment-tag NEW_EMBODIMENT. The included processor/ carries the normalization stats + modality config.

WARNING - Convention (must match at eval): the policy emits joint VELOCITY in rad/s - the client must apply delta = action / 15 with NO clip, and must send 16:9 centre-cropped frames (the crop is baked into the training data), from 2 cameras (exterior_image_1_left + wrist_image_left). A mismatched controller or camera set silently feeds an out-of-distribution input.