| --- |
| license: apache-2.0 |
| tags: |
| - robotics |
| - openpi |
| - pi05 |
| - lerobot |
| - vla |
| --- |
| |
| # pi05_bi — task1_all — step 10000 |
|
|
| openpi `pi05_bi` checkpoint, snapshot at **training step 10000**. |
| Task: *"Use the left hand to pick up the black tube, and then transfer it to the |
| right hand and place it into the rack."* |
|
|
| ## Data |
|
|
| Merged from three LeRobot v2.1 datasets (30 fps, images embedded in parquet, |
| `total_videos=0`): |
|
|
| | source | episodes | frames | |
| |---|---|---| |
| | `KaiyueChen/task1_01` | 550 | 375,061 | |
| | `KaiyueChen/task1_02` | 200 | 130,902 | |
| | `KaiyueChen/task1_03` | 200 | 114,987 | |
| | **merged** | **950** | **620,950** | |
|
|
| `task1_01` shipped with the placeholder task string `perform manipulation task` |
| while 02/03 carried the full description. Since `prompt_from_task=True` feeds |
| this string to the model, all episodes were unified to the 02/03 wording at |
| merge time — otherwise 58% of the data would train against an uninformative |
| prompt. |
|
|
| Split (per-source 10% held out, seed 42): **train 855 / val_seen 95 / |
| val_unseen 95**. Normalization statistics were computed over the *train split |
| only*. |
|
|
| ## Model & training |
|
|
| - 6 camera streams: `camera0`, `camera1`, `tactile_{left,right}_{0,1}` |
| - `state_dim=20`, `action_dim=20`, `action_horizon=50` |
| - LoRA on the PaliGemma LLM (rank 16) and the action expert (rank 32) |
| - **The SigLIP vision tower is fully fine-tuned, not LoRA** — openpi's |
| `get_freeze_filter()` matches `.*llm.*` only, so `PaliGemma/img/*` stays |
| trainable. That is 413M of the 463M trainable parameters (89.8%). |
| - batch size 128, FSDP over 2×A100-80GB, `ema_decay=None` |
| - `CosineDecaySchedule` defaults: peak_lr 2.5e-5, warmup 1000, decay_steps 30000 |
| - init from `gs://openpi-assets/checkpoints/pi05_base/params` |
|
|
| One epoch is 4356 steps, so step 10000 ≈ **2.3 epochs**. |
|
|
| ## Validation at this checkpoint |
|
|
| Loss is the flow-matching training objective evaluated with `train=False` |
| (image augmentation disabled), averaged over a fixed set of batches with a |
| fixed rng so successive points differ only by the model. |
|
|
| | step | val_seen | val_unseen | |
| |---|---|---| |
| | 2000 | 0.0575 | 0.0534 | |
| | 4000 | 0.0530 | 0.0501 | |
| | 6000 | 0.0497 | 0.0482 | |
| | 8000 | 0.0483 | 0.0470 | |
| | **10000** | **0.0463** | **0.0465** | |
|
|
| `val_unseen` reached a sixth consecutive new low here, so no overfitting turn |
| had occurred. The per-step improvement was shrinking fast, though: 3.8% -> |
| 2.4% -> 1.1%, so the curve is approaching a plateau. |
|
|
| For context, an earlier run of the same recipe on a *different* dataset |
| (`pick_tube`, 1250 episodes) bottomed out at step 8000 and degraded from step |
| 10000 onward. That did not happen here. Absolute losses are **not** comparable |
| across the two datasets — they use different normalization statistics, and |
| task1's trajectories are slower and smoother (per-step |Δaction| 0.00052 vs |
| 0.00079), which lowers the achievable regression loss independently of task |
| difficulty. |
|
|
| **Caveat on the numbers:** validation used `val_batches=20` at batch size 128 = |
| 2560 frames, and episodes here average 653 frames, so each evaluation covers |
| only the first ~4 episodes of each split (~5.4%). The *absolute trend* of |
| `val_unseen` is meaningful (identical episodes and rng every time), but the |
| seen-vs-unseen difference rests on 4 episodes per side and should not be read |
| as a generalization gap. |
|
|
| ## Contents |
|
|
| Orbax checkpoint: `params/`, `train_state/`, `assets/`, `_CHECKPOINT_METADATA`. |
|
|
