Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,117 @@
+---
+license: apache-2.0
+language:
+- en
+library_name: lerobot
+pipeline_tag: robotics
+tags:
+- lerobot
+- robotics
+- manipulation
+- pi0
+- tactile
+- bimanual
+---
+
+# pi05_bi — task2 (dish washing), step 8000
+
+openpi `pi05_bi` checkpoint for a bimanual dish-and-sponge task with tactile
+inputs. Step 8000 (1.24 epochs), held-out validation loss
+**0.0535** — the lowest of the run through step 10000.
+
+## Task
+
+Single unified instruction for every episode:
+
+> Use the left hand to pick up the dish, and then use the right hand to pick up
+> the sponge to brush the dish. Finally, put all things back.
+
+Only `task2_01` shipped this string; `task2_02` and `task2_03` both carried the
+placeholder `"perform manipulation task"` in their `meta/tasks.jsonl`. Since
+`prompt_from_task=True` feeds that string straight to the model, all three
+sources were forced onto the instruction above at merge time -- otherwise 76% of
+the episodes (723 of 948) would have trained against an uninformative prompt.
+A trailing space in the original string was stripped.
+
+## Data
+
+| source | episodes | frames |
+|---|---:|---:|
+| `KaiyueChen/task2_01` | 225 | 222,531 |
+| `KaiyueChen/task2_02` | 298 | 291,661 |
+| `KaiyueChen/task2_03` | 425 | 402,110 |
+| **merged** | **948** | **916,302** |
+
+LeRobot v2.1, 30 fps, `robot_type=bimanual`, images embedded in the parquet files
+(`total_videos=0`). Six camera streams: `camera0`, `camera1`, and four tactile
+sensors (`tactile_left_0/1`, `tactile_right_0/1`). Mean episode length ~966
+frames.
+
+### Split
+
+Episodes are held out **per source repo** (10%, seed 42) so the held-out set keeps
+the same source mix as train:
+
+| split | episodes | held out from |
+|---|---:|---|
+| `train` | 854 | |
+| `val_seen` (subset of train) | 94 | |
+| `val_unseen` (held out) | 94 | 22 / 30 / 42 from sources 01 / 02 / 03 |
+
+Normalization statistics (quantile q01/q99) are computed over the `train` split
+only.
+
+## Training
+
+| | |
+|---|---|
+| config | `pi05_bi` |
+| hardware | 2 x A100-80GB, FSDP |
+| batch size | 128 |
+| this checkpoint | step 8000 (~1.24 epoch; 1 epoch = 6,444 steps) |
+| planned length | 20,000 steps |
+| lr | peak 2e-4, 1,000 warmup steps, cosine decay over 100,000 |
+| LoRA | rank 16 on the LLM, rank 32 on the action expert |
+| vision tower | **fully fine-tuned** -- the freeze filter matches only `.*llm.*` |
+
+## Validation curve
+
+Flow-matching loss, 20 batches per split, evaluated on the same leading batches
+each time so successive points are comparable.
+
+| step | train | val_seen | val_unseen | gap |
+|---:|---:|---:|---:|---:|
+| 0 | 0.6399 | 0.6621 | 0.6508 | -0.0113 |
+| 2000 | 0.0535 | 0.0671 | 0.0662 | -0.0009 |
+| 4000 | 0.0470 | 0.0574 | 0.0573 | -0.0001 |
+| 6000 | 0.0448 | 0.0565 | 0.0566 | +0.0001 |
+| **8000** | 0.0430 | 0.0513 | **0.0535** | +0.0022 |
+| 10000 | 0.0420 | 0.0514 | 0.0552 | +0.0038 |
+
+`val_unseen` bottomed at step 8000 (0.0535) and rose to 0.0552 at step 10000.
+Notably `val_seen` flattened at the same time (0.0513 -> 0.0514) while the training
+loss kept falling (0.0430 -> 0.0420) -- both validation curves stalling together
+suggests the model has extracted what it can from this data, not merely that
+generalization has saturated.
+
+The early-step `gap` is slightly **negative** (-0.0009 at step 2000). That is not
+evidence of good generalization: each validation pass covers only ~2,560 frames,
+which at ~966 frames per episode is about 2-3 episodes per split, so the sign of
+the gap is dominated by which particular episodes landed in each split. Only the
+`val_unseen` absolute trend is reliable, since the same episodes and the same rng
+are used at every evaluation.
+
+`val_seen` sits below the training loss from step 8000 onward. This is expected:
+the training loss is measured on augmented images (random crop to 95%, +-5 deg
+rotation, colour jitter, applied to all six streams including the four tactile
+ones) while validation runs with `train=False`, which skips augmentation.
+
+## Contents
+
+```
+checkpoint/
+  params/                      # inference weights
+  train_state/                 # optimizer state, for resuming
+  assets/task2_all/
+    norm_stats.json            # computed over the train split only
+```