Aether258/pi05_bi_two_tubes_0102_step12000

pi05_bi — two_tubes (01+02 merged), step 12000

openpi pi05_bi checkpoint from a bimanual two-tube pick-and-place run with tactile inputs. Step 12000 (2.13 epochs), held-out validation loss 0.0538 (best so far is step 14000 at 0.0537).

Task

Single unified instruction for every episode:

Firstly, use the left hand to pick up the blue tube, and then use the right hand to pick up the green tube. Next, use the left hand to place the blue tube back firstly, and then use the right hand to place the green tube back.

two_tubes_02 shipped with the placeholder string "perform manipulation task" in its meta/tasks.jsonl. Since prompt_from_task=True feeds this string straight to the model, the two sources were forced onto the single instruction above at merge time -- otherwise the model would be taught that two different instructions mean the same motion.

Data

source	episodes	frames
KaiyueChen/two_tubes_01	519	425,115
KaiyueChen/two_tubes_02	500	377,604
merged	1,019	802,719

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1).

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

split	episodes
train	917
val_seen (subset of train)	102
val_unseen (held out)	102

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

config	pi05_bi
hardware	2 x A100-80GB, FSDP
batch size	128
this checkpoint	step 12000 (~2.13 epoch; 1 epoch = 5,639 steps)
planned length	20,000 steps
lr	cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps
	(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule; the peak_lr=2e-4 / decay_steps=100000 block in config.py is referenced only by pi05_single*)
LoRA	rank 16 on the LLM, rank 32 on the action expert
vision tower	fully fine-tuned -- the freeze filter matches only .*llm.*

Steps 0-10000 ran on one host; the run was then resumed from the step-10000 checkpoint on a fresh host with --resume. --resume restores model weights and optimizer state but not the data loader position, so the batch sequence after step 10000 differs from what an uninterrupted run would have seen. This is why step 12000 here reads 0.0538 while the original run's step 12000 read 0.0542 -- the weights genuinely differ, it is not measurement noise.

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

step	train	val_seen	val_unseen	gap
0	0.5525	0.4968	0.5261	0.0293	初始 run
2000	0.0553	0.0504	0.0608	0.0104	初始 run
4000	0.0490	0.0467	0.0576	0.0109	初始 run
6000	0.0460	0.0437	0.0543	0.0106	初始 run
8000	0.0441	0.0423	0.0550	0.0127	初始 run
10000	0.0435	0.0416	0.0542	0.0126	初始 run / 恢复点
12000	0.0420	0.0403	0.0538	0.0135	续训
14000	0.0404	0.0387	0.0537	0.0150	续训

val_unseen fell monotonically through step 6000 and has been effectively flat since: 0.0543 -> 0.0550 -> 0.0542 -> 0.0538 -> 0.0537 over steps 6000-14000, a total improvement of 0.0006 across 8,000 steps. Over the same span val_seen improved 11.4% (0.0437 -> 0.0387) and the gap widened 42% (0.0105 -> 0.0150). That combination -- train and val_seen still falling while val_unseen sits still -- is generalization saturating: capacity gained after step 6000 went into fitting the training episodes rather than transferring.

Each validation pass covers only ~2,560 frames (roughly 3-4 episodes of ~780 frames), so single-point moves under +-0.001 are within noise; the flat trend across five consecutive points is the reliable signal, not any one delta.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/two_tubes_0102/
    norm_stats.json            # computed over the train split only