--- tags: - robotics - humanoid - imitation-learning --- # ACT — Humanoid Benchmark ACT (Action Chunking Transformer, ResNet18 backbone, CVAE; Psi0 `baselines/act`) checkpoints trained on Humanoid Benchmark tasks. ## Checkpoints 100 demonstrations per task. Every row is one training run; the epoch folders are its published checkpoints. | Folder | Task | Robot | Cameras (model input) | Global batch | Proprio augmentation | Checkpoints | |---|---|---|---|---|---|---| | `b6-pedal-bin/` | B6 — pick up a chip barrel, step on the pedal bin, drop it in | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-020` | | `a3-fridge-open/` | A3 — open the fridge door | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | | `a7-cabinet-close/` | A7 — close the cabinet door | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | | `a10-trolley-push/` | A10 — push a trolley | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | | `a20-basket-transfer/` | A20 — transfer a basket | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | | `b7-black-clothes-hamper/` | B7 — take the black garment off the hanger and drop it into the hamper | Unitree G1 (Inspire hands) | ego_view, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | | `a18-pour-water/` | A18 — pour water from a pitcher into a glass | H2 (Sharpa hands) | head, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | | `a17-unscrew-bottle/` | A17 — unscrew the cap of a bottle | H2 (Sharpa hands) | head, chest | 696 | earlier recipe (with delay/drop) | `epoch-010`, `epoch-020` | ## What an epoch folder contains Every `epoch-XXX/` folder is self-contained for evaluation and deployment: weights, model configuration, normalization statistics and `protocol.json` (task, data, input/output dimensions, cameras, batch, steps, augmentation, file list). Simple training records are included for reproducibility (training plan, training log, configs, metrics); optimizer / scheduler / RNG / DeepSpeed resume state is not. `code/` holds the adapter code used for training and inference (`code/VERSIONS.json`: upstream commits and package versions). ## Inputs and augmentation Model inputs are identical across all benchmark models: G1 state 44-D (29 body joint angles + 6 + 6 measured hand values + 3 projected gravity), H2 state 78-D (75 joint angles + 3 projected gravity), no joint velocities. Cameras: G1 ego view + chest; H2 head + chest (A14: head + chest + wrist). The third-person camera recorded in the datasets is for data review only and is never a model input. Each camera is a separate image input, except for Delta Flow and DiT4DiT, which place the views side by side in one frame. Training uses the Humanoid Benchmark online data augmentation spec v2 (15 photometric / geometric / sensor / lighting effects, one of them boosted to 1.5x per sample) with the model's own image augmentation turned off, plus joint-angle bias and noise on the proprioception (column "Proprio augmentation"; the earlier recipe also delayed the proprioception and dropped action-history rows). ## A20 pre-reach trim v2, batch 512, step 4000 (2026-11-17) Checkpoint: [`a20-basket-transfer/b512-trimv2/step-004000/`](a20-basket-transfer/b512-trimv2/step-004000). Trained on 100 demonstrations (54,433 frames) from `humanoid-benchmark/a20-basket-transfer-pre-reach-trim-expert-v2`, with batch size 512, 4,000 optimizer steps, seed 0, EMA weights (decay 0.9999), and Humanoid Benchmark augmentation v2. These weights use the `hb_baselines/1` format. Inputs: Unitree G1 state44 and separate `ego_view` and `chest` RGB images, resized to 168 x 224. Output: 16 actions of dimension 76 (64 SONIC motion-token values and 6 measured values for each INSPIRE hand). Normalization is stored in the weights. The ResNet18 visual backbone was initialized with ImageNet weights and fine-tuned. The checkpoint contains `model.pt`, `config.json` and `protocol.json` (weight size and SHA-256). `SHA256SUMS` covers the checkpoint and the accompanying inference source snapshot. The snapshot is under `code/hb_baselines_v1/`; its requirements record the training environment. Optimizer/resume state is excluded. After downloading this repository and installing its inference dependencies, load the checkpoint from the repository root: ```python import sys sys.path.insert(0, 'code/hb_baselines_v1') from baselines.runtime import BaselinePolicy policy = BaselinePolicy( 'a20-basket-transfer/b512-trimv2/step-004000', device='cpu', policy='act', robot='g1', ) print(policy.info()) ``` The saved weights passed SHA-256 verification and strict CPU loading before upload. The offline metrics in `config.json` use training frames; they are not held-out or closed-loop task success rates. This release uses the accompanying benchmark baseline loader. The earlier `epoch-XXX` releases retain their original Psi0 ACT loader and file format. ## B6 batch 512, step 4000 (2026-11-17) Checkpoint: [`b6-pedal-bin/b512/step-004000/`](b6-pedal-bin/b512/step-004000). A new training run on B6; the earlier B6 release `b6-pedal-bin/epoch-020/` (Psi0 ACT loader, batch 696, earlier augmentation recipe) is kept unchanged. Trained with `evaluation/baselines/train.py --policy act --contract g1_sonic_dfx6` (Humanoid_benchmark commit 4385573e) on the first 100 episodes (65,127 frames) of the B6 v5t7 ready-start expert data, the data of the GR00T `b6-pedal-bin/b256` and Psi0 `b6-pedal-bin/b512` checkpoints: batch size 512, 4,000 optimizer steps (about 31.4 epochs), AdamW lr 1e-4, 200-step linear warm-up then cosine decay to step 4000, seed 0, EMA weights (decay 0.9999), bf16 autocast, ImageNet-initialized ResNet18 fine-tuned at 0.1x the learning rate (frozen BatchNorm), and Humanoid Benchmark augmentation v2 (one call per sample on both views; joint-angle bias +-2 deg plus 0.25 deg noise on the 29 body joints only; no proprio delay or history drop; native random shift off). Before training, a real batch of 512 was checked for every model: state 44-D without velocity, only the body joints perturbed, action labels bit-identical to the parquet files (`training_records/batch_check_b6_v5.json`). Final loss: L1 0.050, KL 0.006; offline check (normalized action MSE on 256 training frames) 0.0054. These weights use the `hb_baselines/1` format and the same `code/hb_baselines_v1/` loader as the A20 release above (byte-identical to the training code). Inputs: Unitree G1 state44 and separate `ego_view` and `chest` RGB images, resized to 168 x 224. Output: 16 actions of dimension 76 (64 SONIC motion-token values and 6 measured values for each INSPIRE hand). Normalization is stored in the weights. Training instruction: "Pick up the chip barrel from the cart, walk to the pedal bin, step on its pedal and drop it in." Closed-loop evaluation (Humanoid_benchmark `debug/newton/b6_policy_eval.py --protocol v5_ready_start` with the v5 task configuration; 50 unseen layouts, seeds 180000-180049, 45 policy seconds; success = the barrel inside the bin and released): **8/50** (Wilson 95 % 8-29 %); the barrel touched 45, lifted 28, pedal pressed 10, in the bin 10. Same seeds and protocol: GR00T `b6-pedal-bin/b256/step-004000` 39/50, Psi0 `b6-pedal-bin/b512/step-004000` 30/50, FM-DiT 0/50, DP 0/50 (trained with this recipe). Files: `model.pt`, `config.json`, `protocol.json` (weight size and SHA-256), `SHA256SUMS` (the checkpoint, the loader snapshot and the training records) and `training_records/` (run configuration `run.json`, `train_log.jsonl`, `train.log`, the pre-training batch check). Optimizer/resume state is excluded. Load it as the A20 example above with `'b6-pedal-bin/b512/step-004000'`.