Upload GR00T N1.5 fine-tuned policy at step 110000

e4e6c77 verified 2 months ago

4.46 kB

library_name: lerobot
pipeline_tag: robotics
base_model: nvidia/GR00T-N1.5-3B
base_model_relation: finetune
datasets:
  - CoRL2026-CSI/Isaaclab-so101_11task_baseCaP_3300epi
license: other
license_name: nvidia-license
license_link: >-
  https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf
tags:
  - robotics
  - lerobot
  - groot
  - gr00t-n1.5
  - so101
  - imitation-learning
  - flow-matching
  - safetensors

CoRL2026-CSI/IsaacLab-so101-multi-gr00t

This is a LeRobot GR00T N1.5 policy fine-tuned from nvidia/GR00T-N1.5-3B on CoRL2026-CSI/Isaaclab-so101_11task_baseCaP_3300epi.

The model is intended for SO-101 style manipulation experiments using RGB observations, robot proprioception, language instructions, and continuous action chunks. It is uploaded as a LeRobot policy checkpoint and should be loaded through the matching LeRobot GR00T implementation used for training.

Model Details

Policy type: GR00T N1.5
Base model: nvidia/GR00T-N1.5-3B
Tokenizer assets: lerobot/eagle2hg-processor-groot-n1p5
Embodiment tag: new_embodiment
Observation steps: 1
Action chunk size: 16
Action steps: 16
Max state/action dims: 64 / 32

Fine-Tuning Setup

Training steps: 110000
Approx. epochs: 7.99
Final training samples: 28160000
Final training loss: 0.010196
Runtime: 76.72 hours
Micro batch size: 64
Gradient accumulation steps: 2
Effective batch size: 256
Optimizer: adamw, lr 0.0001, weight decay 1e-05
Scheduler: cosine_decay_with_warmup
Mixed precision: bf16 policy weights/config enabled: true
VLM/LLM backbone fine-tuned: false
Vision tower fine-tuned: false
Action projector fine-tuned: true
Action DiT fine-tuned: true

The GR00T vision-language backbone was frozen for this run. The action head projector and flow-matching diffusion transformer were fine-tuned.

Inputs

observation.ee_pos.robot_xyzrpy: STATE, shape [6]
observation.gripper_binary: STATE, shape [1]
observation.images.left_wrist: VISUAL, shape [3, 480, 640]
observation.images.top: VISUAL, shape [3, 480, 640]
observation.state: STATE, shape [6]
observation.state.radian_urdf0: STATE, shape [6]

Outputs

action: ACTION, shape [6]
action.radian_urdf0: ACTION, shape [6]

Usage

Install and use the same LeRobot checkout/environment that contains the GR00T policy implementation, then point policy.path to this Hub repo.

lerobot-record \
  --robot.type=<your_robot> \
  --dataset.repo_id=<your_eval_dataset_repo> \
  --policy.path=CoRL2026-CSI/IsaacLab-so101-multi-gr00t \
  --episodes=10

For local Python usage, load the policy with LeRobot's policy factory or GR00T policy loader from the training checkout.

Evaluation

This upload records the offline training run metrics only. No rollout success rate is claimed here unless a separate real/sim evaluation is added later.

Final logged training metrics:

loss: 0.010196
grad norm: 0.193945
update time: 1.2521 s/step
dataloading time: 0.0046 s/step

Limitations and Safety

This model is a robot control policy and can produce unsafe actions if deployed on hardware without appropriate validation, workspace limits, emergency stop handling, and task-specific safety checks. Test in simulation or a constrained setup before any physical deployment.

The model is specialized to the training dataset and embodiment configuration. It may not transfer reliably to different cameras, calibration, action spaces, robot hardware, or tasks without further validation or fine-tuning.

License and Terms

This model is a fine-tune of nvidia/GR00T-N1.5-3B; users are responsible for complying with the NVIDIA model license and any dataset/license constraints. See the base model card and NVIDIA license terms linked in the metadata.

Files

model.safetensors: fine-tuned policy weights
config.json: LeRobot GR00T policy config
train_config.json: training configuration
policy_preprocessor.json and policy_postprocessor.json: LeRobot processor pipelines
policy_*_step_*.safetensors: normalization/statistics state used by processors