add README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: nvidia/GR00T-N1.7-3B
|
| 4 |
+
library_name: gr00t
|
| 5 |
+
pipeline_tag: robotics
|
| 6 |
+
tags: [robotics, vla, gr00t, isaac-gr00t, decent-vla, cube-stacking, so-101, federated]
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# GR00T N1.7 CubeStack — Centralized (3-client union)
|
| 10 |
+
|
| 11 |
+
Centralized ceiling: pooled union of all 6 tasks, **37500 steps** (= 3 clients x 50 rounds x 250 local steps, equal compute to the federation).
|
| 12 |
+
|
| 13 |
+
Trained on Isambard-AI (GH200) with [decent-vla](https://github.com/kevinDuan1/decent-vla).
|
| 14 |
+
|
| 15 |
+
**Scope: action-head-only** — the Cosmos-Reason2 backbone is frozen
|
| 16 |
+
(`tune_llm`/`tune_visual` false); the ~1.62B DiT action head plus the embodiment
|
| 17 |
+
projector are trained. That is the standard NVIDIA new-embodiment recipe: the SO-101
|
| 18 |
+
projector slot ships at random init, so `tune_projector` must stay on.
|
| 19 |
+
|
| 20 |
+
**Embodiment:** SO-101 (`NEW_EMBODIMENT`), state/action 6-D `single_arm`+`gripper`,
|
| 21 |
+
cameras `front`/`wrist`. The `DecentVLA/Cube_*` datasets are LeRobot v3.0, which the
|
| 22 |
+
GR00T loader cannot read, so they were converted to v2.1 and the video pre-scaled to
|
| 23 |
+
shortest edge 256 (GR00T resizes to exactly that internally; it cut decode ~3.9x).
|
| 24 |
+
|
| 25 |
+
**Partition:** the SAME colour-pair non-IID split as the pi0.5 and SmolVLA CubeStack
|
| 26 |
+
3-client studies — c0 harry Green/Orange, c1 zhekai Green/Blue, c2 kevin Orange/Blue,
|
| 27 |
+
each client blind to the third colour.
|
| 28 |
+
|
| 29 |
+
**Caveat worth knowing:** the two kevin prompts are worded differently from the other
|
| 30 |
+
four ("Put orange cube on the blue cube." vs "Put the orange cube on top of the blue
|
| 31 |
+
cube."). Kept verbatim so this arm trains on exactly the strings the pi0.5 and SmolVLA
|
| 32 |
+
arms saw.
|
| 33 |
+
|
| 34 |
+
Both checkpoints hold the SAME parameter set: **537 tensors, 1.6205 B params, all
|
| 35 |
+
`action_head.*`** (verified by reading the tensors) — i.e. the frozen-backbone scope.
|
| 36 |
+
|
| 37 |
+
Checkpoint wrapper differs by arm, because they came off different code paths — load with
|
| 38 |
+
decent-vla `--resume`, not `from_pretrained`:
|
| 39 |
+
* FL `server_round_0050.pt` — `{"state": {...}, "round": 49}` (server aggregate; `round` is 0-indexed).
|
| 40 |
+
* Centralized `round_0075.pt` — `{"state_dict", "include", "extra", "embodiment", "backend"}`
|
| 41 |
+
(the LocalRunner end-of-run save, which also records embodiment + backend metadata).
|
| 42 |
+
Note there is no `server_round_0075.pt`: periodic saves ran every 10 rounds, and 75 is
|
| 43 |
+
not a multiple of 10, so round 75 exists only as this end-of-run file.
|