| --- |
| license: apache-2.0 |
| language: |
| - en |
| library_name: lerobot |
| pipeline_tag: robotics |
| tags: |
| - lerobot |
| - robotics |
| - manipulation |
| - pi0 |
| - tactile |
| - bimanual |
| --- |
| |
| # pi05_bi — two_tubes (01+02 merged), step 6000 |
|
|
| openpi `pi05_bi` checkpoint from a bimanual two-tube pick-and-place run with |
| tactile inputs. This is the step-6000 checkpoint, which had the lowest held-out |
| validation loss of the run up to step 8000. |
|
|
| ## Task |
|
|
| Single unified instruction for every episode: |
|
|
| > Firstly, use the left hand to pick up the blue tube, and then use the right |
| > hand to pick up the green tube. Next, use the left hand to place the blue tube |
| > back firstly, and then use the right hand to place the green tube back. |
|
|
| `two_tubes_02` shipped with the placeholder string `"perform manipulation task"` |
| in its `meta/tasks.jsonl`. Since `prompt_from_task=True` feeds this string |
| straight to the model, the two sources were forced onto the single instruction |
| above at merge time -- otherwise the model would be taught that two different |
| instructions mean the same motion. |
|
|
| ## Data |
|
|
| | source | episodes | frames | |
| |---|---:|---:| |
| | `KaiyueChen/two_tubes_01` | 519 | 425,115 | |
| | `KaiyueChen/two_tubes_02` | 500 | 377,604 | |
| | **merged** | **1,019** | **802,719** | |
|
|
| LeRobot v2.1, 30 fps, `robot_type=bimanual`, images embedded in the parquet |
| files (`total_videos=0`). Six camera streams: `camera0`, `camera1`, and four |
| tactile sensors (`tactile_left_0/1`, `tactile_right_0/1`). |
|
|
| ### Split |
|
|
| Episodes are held out **per source repo** (10%, seed 42) so the held-out set |
| keeps the same source mix as train: |
|
|
| | split | episodes | |
| |---|---:| |
| | `train` | 917 | |
| | `val_seen` (subset of train) | 102 | |
| | `val_unseen` (held out) | 102 | |
|
|
| Normalization statistics (quantile q01/q99) are computed over the `train` split |
| only. |
|
|
| ## Training |
|
|
| | | | |
| |---|---| |
| | config | `pi05_bi` | |
| | hardware | 2 x A100-80GB, FSDP | |
| | batch size | 128 | |
| | this checkpoint | step 6000 (~1.06 epoch; 1 epoch = 5,639 steps) | |
| | planned length | 20,000 steps | |
| | lr | cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps | |
| | | (`CosineDecaySchedule` defaults -- `pi05_bi` does not override `lr_schedule`; the `peak_lr=2e-4 / decay_steps=100000` block in `config.py` is referenced only by `pi05_single*`) | |
| | LoRA | rank 16 on the LLM, rank 32 on the action expert | |
| | vision tower | **fully fine-tuned** -- the freeze filter matches only `.*llm.*` | |
|
|
| ## Validation curve |
|
|
| Flow-matching loss, 20 batches per split, evaluated on the same leading batches |
| each time so successive points are comparable. |
|
|
| | step | train | val_seen | val_unseen | gap | |
| |---:|---:|---:|---:|---:| |
| | 0 | 0.5525 | 0.4968 | 0.5261 | 0.0293 | |
| | 2000 | 0.0504 | 0.0504 | 0.0608 | 0.0104 | |
| | 4000 | 0.0475 | 0.0467 | 0.0576 | 0.0109 | |
| | **6000** | 0.0448 | 0.0437 | **0.0543** | 0.0105 | |
| | 8000 | 0.0439 | 0.0423 | 0.0550 | 0.0127 | |
|
|
| `val_unseen` fell monotonically through step 6000, then ticked up by 0.0007 at |
| step 8000 while `val_seen` kept improving and the gap widened. Each validation |
| pass covers only ~2,560 frames (roughly 3-4 episodes of ~780 frames), so a |
| change of that size is within noise -- whether step 6000 is the true optimum is |
| not yet settled. |
|
|
| ## Contents |
|
|
| ``` |
| checkpoint/ |
| params/ # inference weights |
| train_state/ # optimizer state, for resuming |
| assets/two_tubes_0102/ |
| norm_stats.json # computed over the train split only |
| ``` |
|
|
