MilesKimRLWRLD/domino-puma

PUMA trained from scratch on the DOMINO 10-task dynamic subset

A PUMA policy trained from scratch (no PUMA-pretrained initialisation) on ten dynamic manipulation tasks from the DOMINO benchmark. It exists as the external baseline for a benchmark-validation study of another VLA, so it is published for reproducibility rather than as a tuned release.

Contents

Path	What
domino10_pumascratch_mlxp_seed0_s30000_20260804/pytorch_model.pt	final weights at step 30000 (9.4 GB)

Load with upstream PUMA — the file is a plain torch.save of the framework's state, not a transformers checkpoint, so AutoModel.from_pretrained does not apply.

Tasks

Ten tasks, the dynamic subset used by the DOMINO paper — five Dynamic Interception and five Dynamic Tracking:

grab_roller, handover_block, place_a2b_left, place_can_basket, shake_bottle, beat_block_hammer, move_pillbottle_pad, place_bread_basket, place_container_plate, press_stapler

50 demonstrations per task, all used for training. There is no validation split and no checkpoint selection: the released checkpoint is the last one, at exactly max_steps.

Training

Initialisation	from scratch, on the Qwen3-VL-4B-Instruct-Action base VLM
Steps	30000
Global batch	64 (16 per GPU x 4 GPUs, no gradient accumulation)
Hardware	4x NVIDIA H200
Wall-clock	42.6 h
Attention	flash_attention_2
Data mixture	the ten tasks above at equal weight
Final training loss	0.0012
Runtime	torch 2.6.0+cu124, flash-attn 2.7.4.post1

What the model actually sees

Worth stating explicitly, because it is easy to assume otherwise when comparing against another policy trained on the same files:

• Cameras: three — cam_high, cam_left_wrist, cam_right_wrist.
• Proprioceptive state: not part of the batch. The state modality is declared and its statistics are computed, but upstream's include_state: false keeps it out of the sample, so the policy is vision-and-language only.
• Actions: 14-dim absolute joint targets, chunk of 16.
• Temporal context: four history frames at stride 4 (a 12-step window), supplied as 64x64 optical flow rather than as full-resolution frames.
• World-model supervision: four future frames at stride 4.
• Images: cropped at scale 0.95 then resized to 224x224.

Data conversion

Converted from the benchmark's HDF5 episodes to LeRobot v2.1 with PUMA's own robotwin2lerobot scripts, unmodified. Two properties of that converter are worth knowing because they are inherited by anything trained on its output:

• frames are decoded with OpenCV and stored without a channel swap, so the video holds BGR in RGB slots — training and evaluation must agree on this, and upstream's evaluation path does not apply the swap;
• fps is recorded as 15 in the metadata while the source data is 30 Hz, and the last frame of every episode is dropped.

Differences from upstream defaults

• attn_implementation: sdpa -> flash_attention_2.
• A domino_paper10 entry added to the dataset mixture table for the ten tasks.

Everything else, including the instruction sampling and the evaluation interface, is upstream's.

Reproducibility notes

• The seed0 in the folder name is this study's run-naming convention. PUMA's own RNG seed for this run was upstream's default, 42 — it was not threaded from the study's seed. One training run, one seed: nothing here supports a claim about seed-to-seed variance.
• These weights have not been evaluated in this repository. The numbers reported in the study for a PUMA baseline came from PUMA's published checkpoint, not from this from-scratch run.

Licence

Derived from upstream PUMA and from the Qwen3-VL-4B-Instruct-Action base model; use is subject to their terms. Consult those projects before redistribution.