Update model card (task / checkpoint table)
Browse files
README.md
CHANGED
|
@@ -6,24 +6,39 @@ tags:
|
|
| 6 |
---
|
| 7 |
# ACT — Humanoid Benchmark
|
| 8 |
|
| 9 |
-
ACT (Action Chunking Transformer, ResNet18,
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
| `
|
| 18 |
-
| `
|
| 19 |
-
| `
|
| 20 |
-
| `
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
---
|
| 7 |
# ACT — Humanoid Benchmark
|
| 8 |
|
| 9 |
+
ACT (Action Chunking Transformer, ResNet18 backbone, CVAE; Psi0 `baselines/act`) checkpoints trained on Humanoid Benchmark tasks.
|
| 10 |
+
|
| 11 |
+
## Checkpoints
|
| 12 |
+
|
| 13 |
+
100 demonstrations per task. Every row is one training run; the epoch folders are its published checkpoints.
|
| 14 |
+
|
| 15 |
+
| Folder | Task | Robot | Cameras (model input) | Global batch | Proprio augmentation | Checkpoints |
|
| 16 |
+
|---|---|---|---|---|---|---|
|
| 17 |
+
| `b6-pedal-bin/` | B6 — pick up a chip barrel, step on the pedal bin, drop it in | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-020` |
|
| 18 |
+
| `a3-fridge-open/` | A3 — open the fridge door | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 19 |
+
| `a7-cabinet-close/` | A7 — close the cabinet door | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 20 |
+
| `a10-trolley-push/` | A10 — push a trolley | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 21 |
+
| `a20-basket-transfer/` | A20 — transfer a basket | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 22 |
+
| `b7-black-clothes-hamper/` | B7 — take the black garment off the hanger and drop it into the hamper | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 23 |
+
| `a18-pour-water/` | A18 — pour water from a pitcher into a glass | H2 (Sharpa hands) | head, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 24 |
+
| `a17-unscrew-bottle/` | A17 — unscrew the cap of a bottle | H2 (Sharpa hands) | head, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
|
| 25 |
+
|
| 26 |
+
## What an epoch folder contains
|
| 27 |
+
|
| 28 |
+
Every `epoch-XXX/` folder is self-contained for evaluation and deployment: weights, model configuration, normalization
|
| 29 |
+
statistics and `protocol.json` (task, data, input/output dimensions, cameras, batch, steps, augmentation, file list).
|
| 30 |
+
Simple training records are included for reproducibility (training plan, training log, configs, metrics); optimizer /
|
| 31 |
+
scheduler / RNG / DeepSpeed resume state is not. `code/` holds the adapter code used for training and inference
|
| 32 |
+
(`code/VERSIONS.json`: upstream commits and package versions).
|
| 33 |
+
|
| 34 |
+
## Inputs and augmentation
|
| 35 |
+
|
| 36 |
+
Model inputs are identical across all benchmark models: G1 state 44-D (29 body joint angles + 6 + 6 measured hand values
|
| 37 |
+
+ 3 projected gravity), H2 state 78-D (75 joint angles + 3 projected gravity), no joint velocities. Cameras: G1 ego view +
|
| 38 |
+
chest; H2 head + chest (A14: head + chest + wrist). The third-person camera recorded in the datasets is for data review
|
| 39 |
+
only and is never a model input. Each camera is a separate image input, except for Delta Flow and DiT4DiT, which
|
| 40 |
+
place the views side by side in one frame.
|
| 41 |
+
Training uses the Humanoid Benchmark online data augmentation spec v2 (15 photometric / geometric / sensor / lighting
|
| 42 |
+
effects, one of them boosted to 1.5x per sample) with the model's own image augmentation turned off, plus joint-angle
|
| 43 |
+
bias and noise on the proprioception (column "Proprio augmentation"; the earlier recipe also delayed the
|
| 44 |
+
proprioception and dropped action-history rows).
|