README.md

---
license: apache-2.0
language:
- en
library_name: lerobot
pipeline_tag: robotics
tags:
- lerobot
- robotics
- manipulation
- pi0
- tactile
- bimanual
---

# pi05_bi — task2 (dish washing), step 8000

openpi `pi05_bi` checkpoint for a bimanual dish-and-sponge task with tactile
inputs. Step 8000 (1.24 epochs), held-out validation loss
**0.0535** (best through step 12000 is step 12000 at 0.0534).

## Task

Single unified instruction for every episode:

> Use the left hand to pick up the dish, and then use the right hand to pick up
> the sponge to brush the dish. Finally, put all things back.

Only `task2_01` shipped this string; `task2_02` and `task2_03` both carried the
placeholder `"perform manipulation task"` in their `meta/tasks.jsonl`. Since
`prompt_from_task=True` feeds that string straight to the model, all three
sources were forced onto the instruction above at merge time -- otherwise 76% of
the episodes (723 of 948) would have trained against an uninformative prompt.
A trailing space in the original string was stripped.

## Data

| source | episodes | frames |
|---|---:|---:|
| `KaiyueChen/task2_01` | 225 | 222,531 |
| `KaiyueChen/task2_02` | 298 | 291,661 |
| `KaiyueChen/task2_03` | 425 | 402,110 |
| **merged** | **948** | **916,302** |

LeRobot v2.1, 30 fps, `robot_type=bimanual`, images embedded in the parquet files
(`total_videos=0`). Six camera streams: `camera0`, `camera1`, and four tactile
sensors (`tactile_left_0/1`, `tactile_right_0/1`). Mean episode length ~966
frames.

### Split

Episodes are held out **per source repo** (10%, seed 42) so the held-out set keeps
the same source mix as train:

| split | episodes | held out from |
|---|---:|---|
| `train` | 854 | |
| `val_seen` (subset of train) | 94 | |
| `val_unseen` (held out) | 94 | 22 / 30 / 42 from sources 01 / 02 / 03 |

Normalization statistics (quantile q01/q99) are computed over the `train` split
only.

## Training

| | |
|---|---|
| config | `pi05_bi` |
| hardware | 2 x A100-80GB, FSDP |
| batch size | 128 |
| this checkpoint | step 8000 (~1.24 epoch; 1 epoch = 6,444 steps) |
| planned length | 20,000 steps |
| lr | cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps |
| | (`CosineDecaySchedule` defaults -- `pi05_bi` does not override `lr_schedule`; the `peak_lr=2e-4 / decay_steps=100000` block in `config.py` is referenced only by `pi05_single*`) |
| LoRA | rank 16 on the LLM, rank 32 on the action expert |
| vision tower | **fully fine-tuned** -- the freeze filter matches only `.*llm.*` |

## Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches
each time so successive points are comparable.

| step | train | val_seen | val_unseen | gap |
|---:|---:|---:|---:|---:|
| 0 | 0.6399 | 0.6621 | 0.6508 | -0.0113 |
| 2000 | 0.0535 | 0.0671 | 0.0662 | -0.0009 |
| 4000 | 0.0470 | 0.0574 | 0.0573 | -0.0001 |
| 6000 | 0.0448 | 0.0565 | 0.0566 | +0.0001 |
| **8000** | 0.0430 | 0.0513 | **0.0535** | +0.0022 |
| 10000 | 0.0420 | 0.0514 | 0.0552 | +0.0038 |
| 12000 | 0.0421 | 0.0493 | 0.0534 | +0.0041 |

`val_unseen` fell steeply through step 8000 (0.0535), read 0.0552 at step 10000,
then returned to 0.0534 at step 12000 -- the step-10000 reading was noise, not a
turn. Improvement has nonetheless nearly stopped: the 4,000 steps from 8000 to
12000 bought 0.0001. Over that same span `val_seen` improved 4% (0.0513 ->
0.0493) and the gap widened from 0.0022 to 0.0041.

The gap is still small in absolute terms. The sibling `two_tubes_0102` run had a
gap of 0.0135 at its step 12000 -- more than 3x larger -- and its `val_unseen` had
been flat for 6,000 steps by then. task2 is overfitting far less at the same step
count, which is why training continued past this checkpoint.

The early-step `gap` is slightly **negative** (-0.0009 at step 2000). That is not
evidence of good generalization: each validation pass covers only ~2,560 frames,
which at ~966 frames per episode is about 2-3 episodes per split, so the sign of
the gap early on is dominated by which particular episodes landed in each split.
Only the `val_unseen` absolute trend is reliable, since the same episodes and the
same rng are used at every evaluation.

Note that the training loss is measured on **augmented** images (random crop to
95%, +-5 deg rotation, colour jitter, applied to all six streams including the
four tactile ones) while validation runs with `train=False`, which skips
augmentation. The two numbers are therefore not directly comparable; on this run
`val_seen` still sits above the training loss at every step, unlike the
`two_tubes_0102` run where it dropped below after ~2.5 epochs of memorization.

## Contents

```
checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/task2_all/
    norm_stats.json            # computed over the train split only
```