pi05-real-clutter-60-droid-refined-lora

LoRA fine-tune of pi0.5 (pi05_droid warm-start) on IDEAS-Lab-Northwestern/real-clutter-60-droid-refined — 60 real Franka teleop trajectories of a cluttered-table pick-and-place task, DROID schema.

• Task prompt: "pick up the smallest mug and place it in the bowl"
• Warm-start: gs://openpi-assets/checkpoints/pi05_droid/params
• LoRA: gemma_2b_lora (rank 16) backbone + gemma_300m_lora (rank 32) action expert, batch 4, EMA off.
• Norm-stats: reuses pi05_droid's bundled DROID assets (no recompute) — bundled per checkpoint at assets/droid/norm_stats.json.

Checkpoint ladder

Dir	Cumulative steps	Train loss
10000/	10k	0.0056
20000/	20k	0.0036
30000/	30k	0.0032
40000/	40k	0.0038
50000/	50k (saved at step 49999, openpi 0-indexed)	0.0039

Train loss converges by ~20–30k (min 0.0032 at 30k), flat thereafter. 60 traj × 50k steps ≈ ~13 epochs, so eval-sweep the earlier checkpoints — the best real-robot step is likely 10k–30k, before overfit.

Refined real setup

Trained on data from the refined real setup — refined wrist & main-camera poses and gripper — distinct from earlier real datasets (e.g. cab / jar). The wrist camera is raw cam1, exterior third-view is raw cam0 (routed at conversion via --wrist-cam cam1). Collector: yypeng666.

Observation / action schema (DROID)

Stream	→ pi0.5
exterior third-view (raw cam0)	exterior_image_1_left
(zero-pad)	exterior_image_2_left
wrist (raw cam1)	wrist_image_left
joint_position (7) + gripper_position (1)	state
actions (8) = joint_velocity(7) + next gripper target(1)	actions

Inference

Serve with openpi pi05_droid-family config (pi05_droid_finetune_lora). train_state/ is omitted (inference-only). Eval runs on a real Franka / RT-capable client, not the SFT server.