pi05_bi — two_tubes_all, step 8000
openpi pi05_bi checkpoint fine-tuned on a bimanual two-tube pick-and-place task
with paired RGB + tactile input. Step 8000 is the checkpoint with the best
held-out validation loss for this run (see the curve below).
Task
Firstly, use the left hand to pick up the blue tube, and then use the right hand to pick up the green tube. Next, use the left hand to place the blue tube back firstly, and then use the right hand to place the green tube back.
Data
Derived from KaiyueChen/two_tubes_01
(LeRobot v2.1, 30 fps, robot_type: bimanual).
| Episodes | 519 (train 467 / val_seen 52 / val_unseen 52) |
| Frames | 425,115 |
| 1 epoch | 2,983 steps → step 8000 ≈ 2.68 epochs |
| Observation | 6 × 224×224×3 images: camera0, camera1, tactile_{left,right}_{0,1} |
| State / action | 20-D each |
The split is per-source stratified with 10% held out (seed=42). val_unseen
episodes are never trained on; val_seen is a same-sized sample drawn from the
training episodes, so the two are directly comparable and their difference
isolates memorisation from generalisation.
Normalisation statistics were computed on the training split only (quantile
q01/q99, as PI05 expects) and ship with the checkpoint at
checkpoint/assets/two_tubes_all/norm_stats.json. Computing them over the full
dataset would leak held-out episodes into the input scaling.
Training
| Base | pi05_bi (PaliGemma + SigLIP vision tower, flow-matching action expert) |
| Adaptation | LoRA rank 16 on the LLM, rank 32 on the action expert; the vision tower is fully fine-tuned (the freeze filter only matches .*llm.*, so ~413M of 463M trainable parameters are ViT weights) |
| Hardware | 2 × A100-80GB, FSDP |
| Batch size | 128 |
| Total steps | 12,000 (this checkpoint: 8,000) |
Validation
| step | train | val_seen | val_unseen | gap |
|---|---|---|---|---|
| 2000 | 0.0547 | 0.0549 | 0.0614 | +0.0065 |
| 4000 | 0.0475 | 0.0486 | 0.0548 | +0.0062 |
| 6000 | 0.0443 | 0.0463 | 0.0551 | +0.0088 |
| 8000 | 0.0419 | 0.0436 | 0.0542 | +0.0106 |
| 10000 | 0.0403 | 0.0416 | 0.0553 | +0.0137 |
val_unseen bottoms out at step 8000 and does not improve afterwards: from step
4000 to 10000 it moves by only +0.9% while val_seen drops 14.4% and the gap
more than doubles. Everything learned past ~step 4000 is memorisation rather
than generalisation, which is why this checkpoint — not the final one — is the
one to use.
Caveat on the numbers: each validation pass covers 20 batches × 128 frames
≈ 2,560 frames, which for ~800-frame episodes is only 3–4 episodes per split.
The absolute val_unseen trend is meaningful; the exact value of gap is not.
Layout
checkpoint/
├── _CHECKPOINT_METADATA
├── assets/two_tubes_all/norm_stats.json
├── params/ # orbax OCDBT, inference weights
└── train_state/ # optimiser state, for resuming
Load with openpi by pointing the config's checkpoint directory at checkpoint/.
Inference only needs params/ and assets/; train_state/ is for resuming.