| --- |
| license: other |
| tags: [gr00t, isaac-gr00t, vla, isaac-lab, simulation, ffw-sg2, robotics] |
| base_model: nvidia/GR00T-N1.7-3B |
| --- |
| # groot-blockpick-dr-defaultaug |
|
|
| GR00T N1.7 (3B) fine-tuned on the **domain-randomized** 500-demo top-down block-pick set (`pick-cube-topdown-25cm-dr-500-3cam`: ROBOTIS FFW SG2 in Isaac Lab, left-arm top-down pick of a lime-green block, 3 cameras at 240x240, 84,111 frames, fixed start pose, block uniform over x 0.23-0.48 / y 0.065-0.315 m; per episode random backdrop, table look, lights, camera mounts, block colour, and varied demo timing). |
| Checkpoints live in `step-<n>/` folders (this upload: `step-20000`, `step-40000`, `step-60000`, `step-80000`, `step-100000`); each folder loads with the ROBOTIS fork of Isaac-GR00T |
| (`omkarpatil18/Isaac-GR00T-n1.7` at e81d02b) via `Gr00tPolicy(embodiment_tag="new_embodiment", model_path=<folder>)` after |
| registering the modality config `modality_config/ffw_bg2_pick_cube_3cam_config.py` (state = arm_left 8 + arm_right 8, action = the |
| same 16 absolute joint targets, chunk 16, language "pick up the lime green block with the left arm"). |
|
|
| ## Fine-tune recipe |
| | | | |
| |---|---| |
| | base model | nvidia/GR00T-N1.7-3B (Cosmos-Reason2-2B backbone frozen; projector + diffusion action head tuned, 1.62 B trainable) | |
| | steps / batch | 100,000 planned, global batch 32, checkpoint every 20k | |
| | optimizer | lr 1e-4 cosine, warmup 5 %, weight decay 1e-5, AdamW, fp32 weights | |
| | image aug | the repo's default colour jitter brightness 0.3 / contrast 0.4 / saturation 0.5 / hue 0.08 + random crop | |
| | state | no state noise; state dropout 0.2 (repo default) | |
| | data sampling | shard 1024, episode sampling rate 0.1, 8 loader workers | |
| | hardware | ASU Sol A100 80 GB, ~0.68 s/step | |
|
|
| Files per step folder: model shards (fp32, 12.6 GB), config.json, processor_config.json, statistics.json (normalization), |
| embodiment_id.json, experiment_cfg/, trainer_state.json. Optimizer, scheduler and RNG states are not uploaded. |
|
|
| ## Evaluation |
| Sim rollouts (50 picks, same protocol as the diffusion policies in `padiredd/dp-blockpick-*`) are run with |
| `data_collection/rollout_policy.py --policy groot` against `groot_server.py`; results are added here as they come in. |
|
|
