BIue-Jay commited on
Commit
bd52195
·
verified ·
1 Parent(s): d32ecc1

Update model card (task / checkpoint table)

Browse files
Files changed (1) hide show
  1. README.md +36 -21
README.md CHANGED
@@ -6,24 +6,39 @@ tags:
6
  ---
7
  # ACT — Humanoid Benchmark
8
 
9
- ACT (Action Chunking Transformer, ResNet18, VAE; Psi0 baselines/act) checkpoints trained on Humanoid Benchmark tasks (100 demonstrations per task). Global batch 696 = 87 per GPU x 8 GPUs, one node per run, 20 epochs, saved at epoch 10 and 20.
10
-
11
- | Folder | Task | Robot |
12
- |---|---|---|
13
- | `b6-pedal-bin/` | Pick up a chip barrel, step on the pedal bin, drop it in | Unitree G1 |
14
- | `a3-fridge-open/` | Open the fridge door | Unitree G1 |
15
- | `a7-cabinet-close/` | Close the cabinet door | Unitree G1 |
16
- | `a18-pour-water/` | Pour water from a pitcher into a glass | H2 (Sharpa hands) |
17
- | `a10-trolley-push/` | Push a trolley | Unitree G1 |
18
- | `a20-basket-transfer/` | Transfer a basket | Unitree G1 |
19
- | `a17-unscrew-bottle/` | Unscrew the cap of a bottle | H2 (Sharpa hands) |
20
- | `b7-black-clothes-hamper/` | Take the black garment off the hanger and drop it into the hamper | Unitree G1 |
21
-
22
- Each task folder has one folder per published checkpoint (`epoch-XXX/`). Every epoch folder is self-contained for
23
- evaluation and deployment (weights, configuration, normalization statistics, `protocol.json`) and also carries the
24
- simple training records kept for reproducibility (training plan, logs, metrics). Optimizer / scheduler / RNG /
25
- DeepSpeed resume state is not included. `code/` holds the adapter code used for training and inference.
26
-
27
- Model inputs are identical across all benchmark models: G1 state 44-D (29 body joint angles + 6 + 6 measured hand
28
- values + 3 projected gravity), H2 state 78-D (75 joint angles + 3 projected gravity), no joint velocities.
29
- Training used the Humanoid Benchmark online data augmentation spec v2 with the model's own image augmentation off.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  ---
7
  # ACT — Humanoid Benchmark
8
 
9
+ ACT (Action Chunking Transformer, ResNet18 backbone, CVAE; Psi0 `baselines/act`) checkpoints trained on Humanoid Benchmark tasks.
10
+
11
+ ## Checkpoints
12
+
13
+ 100 demonstrations per task. Every row is one training run; the epoch folders are its published checkpoints.
14
+
15
+ | Folder | Task | Robot | Cameras (model input) | Global batch | Proprio augmentation | Checkpoints |
16
+ |---|---|---|---|---|---|---|
17
+ | `b6-pedal-bin/` | B6 — pick up a chip barrel, step on the pedal bin, drop it in | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-020` |
18
+ | `a3-fridge-open/` | A3 — open the fridge door | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
19
+ | `a7-cabinet-close/` | A7 — close the cabinet door | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
20
+ | `a10-trolley-push/` | A10 — push a trolley | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
21
+ | `a20-basket-transfer/` | A20 — transfer a basket | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
22
+ | `b7-black-clothes-hamper/` | B7 — take the black garment off the hanger and drop it into the hamper | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
23
+ | `a18-pour-water/` | A18 — pour water from a pitcher into a glass | H2 (Sharpa hands) | head, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
24
+ | `a17-unscrew-bottle/` | A17 — unscrew the cap of a bottle | H2 (Sharpa hands) | head, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` |
25
+
26
+ ## What an epoch folder contains
27
+
28
+ Every `epoch-XXX/` folder is self-contained for evaluation and deployment: weights, model configuration, normalization
29
+ statistics and `protocol.json` (task, data, input/output dimensions, cameras, batch, steps, augmentation, file list).
30
+ Simple training records are included for reproducibility (training plan, training log, configs, metrics); optimizer /
31
+ scheduler / RNG / DeepSpeed resume state is not. `code/` holds the adapter code used for training and inference
32
+ (`code/VERSIONS.json`: upstream commits and package versions).
33
+
34
+ ## Inputs and augmentation
35
+
36
+ Model inputs are identical across all benchmark models: G1 state 44-D (29 body joint angles + 6 + 6 measured hand values
37
+ + 3 projected gravity), H2 state 78-D (75 joint angles + 3 projected gravity), no joint velocities. Cameras: G1 ego view +
38
+ chest; H2 head + chest (A14: head + chest + wrist). The third-person camera recorded in the datasets is for data review
39
+ only and is never a model input. Each camera is a separate image input, except for Delta Flow and DiT4DiT, which
40
+ place the views side by side in one frame.
41
+ Training uses the Humanoid Benchmark online data augmentation spec v2 (15 photometric / geometric / sensor / lighting
42
+ effects, one of them boosted to 1.5x per sample) with the model's own image augmentation turned off, plus joint-angle
43
+ bias and noise on the proprioception (column "Proprio augmentation"; the earlier recipe also delayed the
44
+ proprioception and dropped action-history rows).