Instructions to use Cache-SCA/pi05_teleop_sort_block with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use Cache-SCA/pi05_teleop_sort_block with LeRobot:
- Notebooks
- Google Colab
- Kaggle
Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: lerobot
|
| 4 |
+
pipeline_tag: robotics
|
| 5 |
+
tags:
|
| 6 |
+
- lerobot
|
| 7 |
+
- robotics
|
| 8 |
+
- pi05
|
| 9 |
+
- so101
|
| 10 |
+
- imitation-learning
|
| 11 |
+
datasets:
|
| 12 |
+
- CoRL2026-CSI/SO101-teleop_sort_blocks_100epi
|
| 13 |
+
base_model: lerobot/pi05_base
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# π0.5 — SO-101 `sort_blocks`
|
| 17 |
+
|
| 18 |
+
Fine-tuned [`lerobot/pi05_base`](https://huggingface.co/lerobot/pi05_base) on 100 teleop episodes of the SO-101 `sort_blocks` task.
|
| 19 |
+
|
| 20 |
+
## Model
|
| 21 |
+
|
| 22 |
+
- **Architecture**: π0.5 (PaliGemma-2B VLM + Gemma-300M action expert, flow matching, 10 inference steps)
|
| 23 |
+
- **Cameras**: `base_0_rgb`, `left_wrist_0_rgb`, `right_wrist_0_rgb` (224×224)
|
| 24 |
+
- **State / Action dim**: 32 (padded) / 6 (SO-101)
|
| 25 |
+
- **Action chunk**: 50
|
| 26 |
+
- **dtype**: bfloat16
|
| 27 |
+
|
| 28 |
+
Camera key rename (dataset → policy):
|
| 29 |
+
```
|
| 30 |
+
observation.images.top → observation.images.base_0_rgb
|
| 31 |
+
observation.images.wrist → observation.images.left_wrist_0_rgb
|
| 32 |
+
```
|
| 33 |
+
`right_wrist_0_rgb` is an empty camera slot for this single-arm setup.
|
| 34 |
+
|
| 35 |
+
Action features (SO-101): `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper` (`.pos`).
|
| 36 |
+
Normalization: `ACTION/STATE = MEAN_STD`, `VISUAL = IDENTITY`.
|
| 37 |
+
|
| 38 |
+
## Data
|
| 39 |
+
|
| 40 |
+
[`CoRL2026-CSI/SO101-teleop_sort_blocks_100epi`](https://huggingface.co/datasets/CoRL2026-CSI/SO101-teleop_sort_blocks_100epi) — 100 episodes, 94,568 frames, 30 fps, human teleop.
|
| 41 |
+
|
| 42 |
+
## Training
|
| 43 |
+
|
| 44 |
+
| | |
|
| 45 |
+
|---|---|
|
| 46 |
+
| Hardware | 4 × GPU (DDP, 🤗 Accelerate) |
|
| 47 |
+
| Per-device batch | 32 |
|
| 48 |
+
| Gradient accumulation | 2 |
|
| 49 |
+
| Effective global batch | 256 |
|
| 50 |
+
| Steps | 18,500 (~50 epochs) |
|
| 51 |
+
| Optimizer | AdamW, β=(0.9, 0.95), wd=0.01, grad clip 1.0 |
|
| 52 |
+
| LR | cosine decay, peak 2.5e-5 → 2.5e-6, warmup 1000, decay 30000 |
|
| 53 |
+
| Gradient checkpointing | on |
|
| 54 |
+
| Image aug | ColorJitter (brightness/contrast/saturation/hue), SharpnessJitter, RandomAffine — `max_num=3`, random order |
|
| 55 |
+
| Seed | 1000 |
|
| 56 |
+
|
| 57 |
+
Training script: [`scripts/train_pi05_sort_blocks.sh`](https://github.com/HyeonseokE/train_with_lerobot/blob/main/scripts/train_pi05_sort_blocks.sh).
|
| 58 |
+
|
| 59 |
+
## Usage
|
| 60 |
+
|
| 61 |
+
```python
|
| 62 |
+
from lerobot.policies.pi05.modeling_pi05 import PI05Policy
|
| 63 |
+
|
| 64 |
+
policy = PI05Policy.from_pretrained("CoRL2026-CSI/pi05_teleop_sort_block").to("cuda").eval()
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
```bash
|
| 68 |
+
lerobot-eval --policy.path=CoRL2026-CSI/pi05_teleop_sort_block --env.type=<env> --eval.n_episodes=20
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
## Limitations
|
| 72 |
+
|
| 73 |
+
- Single task, single seed; no quantitative success rate reported here.
|
| 74 |
+
- Trained on a single-arm SO-101; the right-wrist camera slot is empty.
|
| 75 |
+
- 100 episodes only — sensitive to camera/lighting domain shift.
|
| 76 |
+
|
| 77 |
+
## License
|
| 78 |
+
|
| 79 |
+
Apache 2.0 (inherits from [`lerobot/pi05_base`](https://huggingface.co/lerobot/pi05_base)).
|