pi0.5 SO-101 Erythromycin-on-Tea
OpenPI pi0.5 fully fine-tuned for a dual-camera SO-101 follower on one tabletop manipulation task:
Pick up the red erythromycin ointment box and place it on top of the green Rizhao tea tin.
Author: CodeChild
This repository contains inference-only OpenPI/Orbax artifacts. It is not a standard Transformers checkpoint and cannot be loaded with transformers.AutoModel.from_pretrained().
Model details
| Field | Value |
|---|---|
| Base checkpoint | gs://openpi-assets/checkpoints/pi05_base/params |
| Fine-tuning | Full parameters, 8000 optimizer updates |
| Inference weights | EMA parameters, decay 0.99 |
| Cameras | Fixed RGB + wrist RGB, 640x480 at collection time |
| State/action | 6-D calibrated SO-101 position space |
| Action horizon | 50 steps at 30 Hz |
| Output | (50, 6) absolute SO-101 position targets |
| Code release tag | so101-pi05-erythromycin-v1 |
| W&B | https://wandb.ai/99087192-zhejiang-university/openpi/runs/xgk0h74f |
The full deployment and safety handoff is in SO101_PI05_HANDOFF.md. Read it before connecting this policy to motors.
Repository contents
params/ # EMA inference parameters, about 12 GiB
assets/ # clean-train normalization statistics
_CHECKPOINT_METADATA # original Orbax checkpoint metadata
code/so101-pi05-erythromycin-v1.patch
SO101_PI05_HANDOFF.md
LICENSE_OPENPI.txt
LICENSE_GEMMA.txt
NOTICE
The optimizer state is intentionally not published. This repository is suitable for inference, not direct training resume.
Code setup
The SO-101 adapter was developed from OpenPI commit:
15a9616a00943ada6c20a0f158e3adb39df2ccac
The release commit is tagged locally as so101-pi05-erythromycin-v1. Because the source checkout only has the upstream Physical Intelligence remote, the exact code delta is also included in this model repository:
git clone https://github.com/Physical-Intelligence/openpi.git
cd openpi
git checkout 15a9616a00943ada6c20a0f158e3adb39df2ccac
git apply /path/to/so101-pi05-erythromycin-v1.patch
GIT_LFS_SKIP_SMUDGE=1 UV_LINK_MODE=copy uv sync
The data config expects the portable dataset package next to the OpenPI checkout when running data-dependent scripts. Policy inference only needs this repository's params/ and assets/.
Download and serve
from huggingface_hub import snapshot_download
checkpoint_dir = snapshot_download("CodeChild/pi05-so101-erythromycin-tea")
print(checkpoint_dir)
From the patched OpenPI checkout:
CUDA_VISIBLE_DEVICES=<GPU_ID> .venv/bin/python scripts/serve_policy.py \
policy:checkpoint \
--policy.config pi05_so101_erythromycin \
--policy.dir <HF_SNAPSHOT_DIR>
Policy input:
observation = {
"observation/state": state_float32_6,
"observation/fixed_image": fixed_rgb_uint8_hwc,
"observation/wrist_image": wrist_rgb_uint8_hwc,
"prompt": "Pick up the red erythromycin ointment box and place it on top of the green Rizhao tea tin.",
}
The server returns result["actions"] with shape (50, 6).
Critical action semantics
During training, dimensions 0-4 are represented relative to the same current state and dimension 5 remains absolute:
delta[t, 0:5] = absolute_target[t, 0:5] - current_state[0:5]
delta[t, 5] = absolute_target[t, 5]
This is not a step-to-step increment. OpenPI applies the inverse transform before returning actions, so the policy server output is already absolute. A robot client must not take a cumulative sum and must not add the current state again.
The trajectory was collected at 30 Hz. Fifty predicted steps correspond to approximately 1.67 seconds of control ticks. For an initial supervised robot test, execute a short prefix and replan; the accompanying handoff recommends starting with 5 steps at 30 Hz. This is a deployment recommendation, not a robot-validated hyperparameter.
Training data
The source dataset has 90 episodes and two synchronized camera streams. It is not redistributed in this model repository. The authoritative split was splits/split_manifest.json:
| Split | Episodes | Frames | Use |
|---|---|---|---|
clean_train |
67 | 15070 | Training and normalization statistics |
clean_val |
18 | 3972 | Offline validation only |
recovery |
5 | 1515 | Excluded |
The default train: 0:90 field in the source LeRobot metadata was not used because it would leak validation and recovery episodes into training.
OpenPI's standard training augmentation was active: crop/rotation/color augmentation for the fixed camera and color augmentation for the wrist camera. Evaluation and inference use no random augmentation.
Offline evaluation
| Model | Split | Samples | Flow-matching loss |
|---|---|---|---|
| Original pi0.5 base | clean_val |
3972 | 0.04753249 |
| Fine-tuned checkpoint | clean_train |
15068 | 0.00407172 |
| Fine-tuned checkpoint | clean_val |
3972 | 0.01467515 |
The held-out validation loss is 69.126% lower than the base checkpoint under this evaluation. The validation/train ratio is 3.604, indicating a generalization gap. There is no independent test split.
Flow-matching loss is not a robot task-success metric. No closed-loop real-robot success rate has been measured for this checkpoint yet.
Intended use and limitations
- Intended for research and supervised evaluation on the stated SO-101 task.
- Requires the same joint order, direction, zero points, gripper calibration, camera assignment, RGB convention and 30 Hz timing used during collection.
- Before motor execution, validate finite values and shape, enforce hardware joint/gripper limits, maximum target deltas, velocity/workspace limits, timeouts and an emergency stop.
- Start with motors-off shadow inference, then low-speed supervised closed-loop tests.
- The model was trained on one task with a small dataset and may fail under new layouts, lighting, camera movement, object appearance or calibration drift.
- Do not infer safety or reliability from the offline flow-matching loss.
Licenses
OpenPI code is provided under Apache-2.0; see LICENSE_OPENPI.txt.
The model is derived from pi0.5, which includes Gemma components. Gemma use and redistribution are subject to the Gemma Terms of Use in LICENSE_GEMMA.txt, and the required notice is provided in NOTICE. For this reason the Hugging Face metadata uses license: other rather than describing the complete artifact as Apache-2.0 only.