pi05_bi — two_tubes (01+02 merged), step 10000
openpi pi05_bi checkpoint from a bimanual two-tube pick-and-place run with
tactile inputs. This is the step-10000 checkpoint, which has the lowest held-out
validation loss of the run so far -- though only by 0.0001 over step 6000, so the
two are effectively tied.
Task
Single unified instruction for every episode:
Firstly, use the left hand to pick up the blue tube, and then use the right hand to pick up the green tube. Next, use the left hand to place the blue tube back firstly, and then use the right hand to place the green tube back.
two_tubes_02 shipped with the placeholder string "perform manipulation task"
in its meta/tasks.jsonl. Since prompt_from_task=True feeds this string
straight to the model, the two sources were forced onto the single instruction
above at merge time -- otherwise the model would be taught that two different
instructions mean the same motion.
Data
| source | episodes | frames |
|---|---|---|
KaiyueChen/two_tubes_01 |
519 | 425,115 |
KaiyueChen/two_tubes_02 |
500 | 377,604 |
| merged | 1,019 | 802,719 |
LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet
files (total_videos=0). Six camera streams: camera0, camera1, and four
tactile sensors (tactile_left_0/1, tactile_right_0/1).
Split
Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:
| split | episodes |
|---|---|
train |
917 |
val_seen (subset of train) |
102 |
val_unseen (held out) |
102 |
Normalization statistics (quantile q01/q99) are computed over the train split
only.
Training
| config | pi05_bi |
| hardware | 2 x A100-80GB, FSDP |
| batch size | 128 |
| this checkpoint | step 10000 (~1.77 epoch; 1 epoch = 5,639 steps) |
| planned length | 20,000 steps |
| lr | cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps |
(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule; the peak_lr=2e-4 / decay_steps=100000 block in config.py is referenced only by pi05_single*) |
|
| LoRA | rank 16 on the LLM, rank 32 on the action expert |
| vision tower | fully fine-tuned -- the freeze filter matches only .*llm.* |
Validation curve
Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.
| step | train | val_seen | val_unseen | gap |
|---|---|---|---|---|
| 0 | 0.5525 | 0.4968 | 0.5261 | 0.0293 |
| 2000 | 0.0553 | 0.0504 | 0.0608 | 0.0104 |
| 4000 | 0.0490 | 0.0467 | 0.0576 | 0.0109 |
| 6000 | 0.0460 | 0.0437 | 0.0543 | 0.0105 |
| 8000 | 0.0441 | 0.0423 | 0.0550 | 0.0127 |
| 10000 | 0.0435 | 0.0416 | 0.0542 | 0.0126 |
val_unseen fell monotonically through step 6000 and has been flat since:
0.0543 -> 0.0550 -> 0.0542 over steps 6000-10000, a spread of 0.0008. Over the
same span val_seen improved 0.0437 -> 0.0416 and the gap widened from 0.0105 to
0.0126. That combination -- train and val_seen still falling while val_unseen
sits still -- is generalization saturating: the capacity gained after step 6000
went into fitting the training episodes rather than transferring.
Each validation pass covers only ~2,560 frames (roughly 3-4 episodes of ~780 frames), so single-point moves under +-0.001 are within noise; the flat trend across three consecutive points is the reliable signal, not any one delta.
Contents
checkpoint/
params/ # inference weights
train_state/ # optimizer state, for resuming
assets/two_tubes_0102/
norm_stats.json # computed over the train split only