BrutalCaesar/act_so101_redcube_wrist_front_chunk75_step25k

ACT · SO-101 · red cube → cardboard box · wrist + front · chunk 75 · step 25k

Action Chunking Transformer trained with LeRobot 0.6.0 on BrutalCaesar/phi_so101_redcube_in_box_v1.

Task: pick up a red 3D-printed cube and place it in a fixed cardboard box. Cameras: wrist (gripper) + front (desk-level Logitech Brio 101).

✅ The camera keys are correct in this model

Unlike our two 8bin models, this dataset was recorded with the index-to-name mapping fixed and verified against pixels before upload. No transposition. Wire by name.

Physical camera	Observation key
wrist (gripper module)	observation.images.wrist
front (desk-level)	observation.images.front

If you have used act_so101_8bin_wrist_front_chunk50 or act_so101_8bin_wrist_top_chunk50, do not carry their swapped mapping over to this one.

Why the 25,000-step checkpoint and not 100,000

The dataset is 11,264 frames. At batch 8:

Checkpoint	Samples	Epochs	Train loss
25,000 ← this one	200K	17.8	0.068
50,000	400K	35.5	0.045
75,000	600K	53.3	0.038
100,000	800K	71.0	0.032

100k steps is 71 epochs over 20 demonstrations of a fixed-position task. The training loss falls smoothly the whole way and cannot detect overfitting here, because all 20 episodes are in the training set and there is no held-out split to hold it honest.

25k lands at 17.8 epochs, close to the 14.6 epochs our larger 8bin dataset got at the same 100k steps. So this checkpoint is the one whose exposure per frame matches a normally-trained run. The later checkpoints exist and may well be better — but that is a question for rollouts, not loss.

Inputs / outputs

	Shape	Notes
observation.images.wrist	(3, 480, 640)	no resize anywhere in the pipeline
observation.images.front	(3, 480, 640)
observation.state	(6,)	SO-101 joint positions, use_degrees=True at record time
action	(75, 6)	75-step chunk = 2.5 s at 30 fps, all of it executed

Normalization ships as LeRobot 0.6.0 processor files (policy_preprocessor*, policy_postprocessor*), not baked into the weights. Load via ACTPolicy.from_pretrained or actions come out in the wrong units.

Training

Steps	25,000 of 100,000 (this checkpoint)
Batch size	8
Train loss here	0.068
Optimizer	AdamW, lr 1e-5, backbone lr 1e-5, weight decay 1e-4
Backbone	ResNet-18, ImageNet-pretrained, not frozen
dim_model / chunk_size / kl_weight	512 / 75 / 10.0
n_obs_steps	1 — single frame, no history, no velocity
Seed	1000
Hardware	1× H200, Northeastern Explorer, 1 h 28 m for the full 100k

All hyperparameters except chunk_size and the camera pair are LeRobot defaults, deliberately, so runs stay comparable across people training the same task.

🚨 Scene dependency: keep the cardboard box

Every one of the 11,264 training frames contains the cardboard box in the front and wrist views. Substituting a 3D-printed bin — different colour, geometry and reflectivity — is a domain shift this policy has never seen, and it will read as a policy failure when it is a scene change.

Camera pose matters equally: the mounts must sit where they sat during recording.

Evaluation

Not evaluated on hardware. No success rate is reported because none has been measured.

Note also that this dataset has no held-out split — one bin position, one object placement — so even a perfect score here measures fit, not generalization. It is a positive control for the pipeline, not a capability claim.

Usage

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("BrutalCaesar/act_so101_redcube_wrist_front_chunk75_step25k")

Optional: temporal ensembling

This checkpoint ships with temporal_ensemble_coeff=None and n_action_steps=75, i.e. fully open-loop for 2.5 s per query. The ACT paper reports +3.3% from temporal ensembling, which requires querying every step:

policy.config.n_action_steps = 1
policy.config.temporal_ensemble_coeff = 0.01

This is inference-only — no retraining needed — but costs 75× the forward passes. Worth an A/B once you are scoring rollouts.

Known limitations

• n_obs_steps=1, so no velocity information.
• The CVAE latent is collapsed (KL ≈ 0 at kl_weight=10.0), so this behaves as a deterministic chunk regressor. The ACT paper's ablation reports human-demo performance dropping 35.3% → 2% without the CVAE objective, so whether the collapse is benign here is an open question we have not settled.
• One lighting setup, one room, one operator, one object placement. No domain randomization beyond LeRobot's default image augmentation.