--- license: apache-2.0 base_model: nvidia/GR00T-N1.7-3B library_name: gr00t pipeline_tag: robotics tags: [robotics, vla, gr00t, isaac-gr00t, decent-vla, cube-stacking, so-101, federated] --- # GR00T N1.7 CubeStack — Centralized (3-client union) Centralized ceiling: pooled union of all 6 tasks, **37500 steps** (= 3 clients x 50 rounds x 250 local steps, equal compute to the federation). Trained on Isambard-AI (GH200) with [decent-vla](https://github.com/kevinDuan1/decent-vla). **Scope: action-head-only** — the Cosmos-Reason2 backbone is frozen (`tune_llm`/`tune_visual` false); the ~1.62B DiT action head plus the embodiment projector are trained. That is the standard NVIDIA new-embodiment recipe: the SO-101 projector slot ships at random init, so `tune_projector` must stay on. **Embodiment:** SO-101 (`NEW_EMBODIMENT`), state/action 6-D `single_arm`+`gripper`, cameras `front`/`wrist`. The `DecentVLA/Cube_*` datasets are LeRobot v3.0, which the GR00T loader cannot read, so they were converted to v2.1 and the video pre-scaled to shortest edge 256 (GR00T resizes to exactly that internally; it cut decode ~3.9x). **Partition:** the SAME colour-pair non-IID split as the pi0.5 and SmolVLA CubeStack 3-client studies — c0 harry Green/Orange, c1 zhekai Green/Blue, c2 kevin Orange/Blue, each client blind to the third colour. **Caveat worth knowing:** the two kevin prompts are worded differently from the other four ("Put orange cube on the blue cube." vs "Put the orange cube on top of the blue cube."). Kept verbatim so this arm trains on exactly the strings the pi0.5 and SmolVLA arms saw. Both checkpoints hold the SAME parameter set: **537 tensors, 1.6205 B params, all `action_head.*`** (verified by reading the tensors) — i.e. the frozen-backbone scope. Checkpoint wrapper differs by arm, because they came off different code paths — load with decent-vla `--resume`, not `from_pretrained`: * FL `server_round_0050.pt` — `{"state": {...}, "round": 49}` (server aggregate; `round` is 0-indexed). * Centralized `round_0075.pt` — `{"state_dict", "include", "extra", "embodiment", "backend"}` (the LocalRunner end-of-run save, which also records embodiment + backend metadata). Note there is no `server_round_0075.pt`: periodic saves ran every 10 rounds, and 75 is not a multiple of 10, so round 75 exists only as this end-of-run file. ## Running these weights (experiment_cfg/) The checkpoint holds ONLY `action_head.*` (537 tensors, 1.6205 B). Three things GR00T derives from the training dataset are not in it and not in the public N1.7 base repo, so they are published here under `experiment_cfg/`: | file | what it is | |---|---| | `embodiment_id.json` | the projector-slot mapping used in this run | | `processor_config.json` | modality config (state/action index ranges, video keys) + image/state processor settings | | `statistics.json` | state/action normalization statistics | **Embodiment slot = 10.** `new_embodiment` is absent from the base repo's `embodiment_id.json`; the value comes from `EMBODIMENT_TAG_TO_PROJECTOR_INDEX` in `gr00t/model/gr00t_n1d7/processing_gr00t_n1d7.py`. Verified against the weights, not just that table: diffing this checkpoint's 14 embodiment-indexed tensors against the base head puts **100.0% of the drift in slot 10 and exactly 0.0 in all 31 other slots** (see `experiment_cfg/slot_drift.json`). Slot 0 is untrained — falling back to it runs without error and produces meaningless actions. Modality (from `processor_config.json`): video keys `front`, `wrist`; state and action `single_arm` (0-5) + `gripper` (5-6); action horizon 16; `shortest_image_edge` 256. The training config for this run is included as `experiment_cfg/train_config.yaml`.