README.md

---
license: apache-2.0
library_name: lerobot
pipeline_tag: robotics
tags:
- pi05
- pytorch_model_hub_mixin
- robotics
- IB-Robot
- LeRobot
- so-101
- banana-pick
- vision-language-action
base_model:
- lerobot/pi05_base
---

# Model Card for PI0.5/BananaPick (IB-Robot)

PI0.5 policy (as per [π0 and π0.5: Vision-Language-Action Flow Models for Robot Manipulation](https://arxiv.org/abs/2410.24132)) fine-tuned for banana pick-and-place on a 1-arm SO-101 robot within the [IB-Robot](https://gitcode.com/openeuler/IB_Robot) framework.

This checkpoint fine-tunes `lerobot/pi05_base` (PaliGemma 2B vision-language model + Gemma 300M action expert) on a teleoperated banana-pick dataset. The bundle ships PyTorch weights plus a BERT tokenizer for language-conditioned inference, and a single [`inference_manifest.json`](./inference_manifest.json) routing table (schema v3).

## Repository Structure

```
├── config.json                                                 # PI0.5 策略配置
├── model.safetensors                                           # torch 权重 (~8.8 GB, bf16)
├── policy_preprocessor.json
├── policy_postprocessor.json
├── policy_preprocessor_step_2_normalizer_processor.safetensors
├── policy_postprocessor_step_0_unnormalizer_processor.safetensors
├── bert-base-uncased/                                          # 语言指令 tokenizer (5 files)
│   ├── tokenizer.json
│   ├── tokenizer.model
│   ├── tokenizer_config.json
│   ├── special_tokens_map.json
│   └── added_tokens.json
├── train_config.json                                           # 完整训练超参
└── inference_manifest.json                                     # 部署路由表 (schema v3, 权威)
```

> The directory layout and every file path inside `inference_manifest.json` must stay in sync. Do not rename or relocate files — the manifest's sha256 checks and path bindings depend on them.

## Deployment Backends

Read `inference_manifest.json` → `deployments[<target>]` to route to the right backend. This bundle ships **PyTorch-only** deployments (compiled Ascend/RKNN artifacts are published separately).

| Target | Backend | Runtime | Artifact | Hardware |
|--------|---------|---------|----------|----------|
| `torch-cuda` | torch | PyTorch | `model.safetensors` (in bundle) | NVIDIA GPU |
| `torch-cpu` | torch | PyTorch | `model.safetensors` (in bundle) | CPU |

**Input tensors:** `observation.state` [6] float32, `observation.current` [6] float32, `observation.images.top` [3,480,640] NCHW, `observation.images.wrist` [3,480,640] NCHW.
**Output tensor:** `action` [6] float32 (6-dim joint action: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper).

## How to Get Started with the Model

See the [IB-Robot project](https://gitcode.com/openeuler/IB_Robot) (particularly the [inference_service](https://gitcode.com/openeuler/IB_Robot/tree/main/src/inference_service)) for instructions on how to load and deploy this model with ROS 2.

To load the **PyTorch** backend directly in Python:

```python
from lerobot.common.policies.pi05.modeling_pi05 import PI05Policy

policy = PI05Policy.from_pretrained("openEuler/pi05")
```

For ROS 2 deployment, consume `inference_manifest.json` and route to `deployments["torch-cuda"]` or `deployments["torch-cpu"]` via the IB-Robot inference_service.

## Training Details

- **Policy:** PI0.5 (PaliGemma 2B + Gemma 300M action expert)
- **Base model:** `lerobot/pi05_base`
- **Robot:** 1-arm SO-101
- **Task:** Banana pick-and-place
- **Cameras:** top, wrist (480×640, resized to 224×224 internally)
- **Action dim:** 6 (5 joints + gripper)
- **Chunk size:** 50 action steps, 50 executed per step
- **Inference steps:** 10 flow-matching denoising steps
- **Dtype:** bfloat16
- **Tokenizer:** bert-base-uncased (max_length=200)
- **Normalization:** VISUAL=IDENTITY, STATE=QUANTILES, ACTION=QUANTILES
- **Gradient checkpointing:** enabled
- **Optimizer:** AdamW (lr=2.5e-5, weight_decay=0.01, betas=[0.9, 0.95])
- **Scheduler:** cosine decay with warmup (1000 warmup, 30000 decay, decay_lr=2.5e-6)
- **Batch size:** 8
- **Training steps:** 100,000
- **Seed:** 1000

## Model Architecture

PI0.5 is a Vision-Language-Action (VLA) flow model:

- **Vision encoder:** PaliGemma (SigLIP-based, 224×224 input)
- **Language model:** Gemma 2B (PaliGemma backbone, processes text + image tokens)
- **Action expert:** Gemma 300M (separate decoder for flow-matching action generation)
- **Flow matching:** 10 denoising steps to generate action chunks
- **Parameters:** ~4.1B (533M F32 + 3.6B BF16)

## Citation

```bibtex
@software{ib_robot,
  title = {IB-Robot: Intelligence Boom Robot},
  url = {https://gitcode.com/openeuler/IB_Robot},
  license = {Apache-2.0}
}
```

```bibtex
@article{pi05,
  title = {π0 and π0.5: Vision-Language-Action Flow Models for Robot Manipulation},
  url = {https://arxiv.org/abs/2410.24132}
}
```