File size: 2,590 Bytes
823bd5e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
language:
- en
library_name: lerobot
pipeline_tag: robotics
tags:
- vision-language-action
- imitation-learning
- lerobot
- pi05
- manipulation
inference: false
datasets:
- PickNikRobotics/kinova_gen3_cube_stack_sim
base_model:
- lerobot/pi05_base
license: gemma
extra_gated_heading: Access this model on Hugging Face
extra_gated_prompt: >-
This model is a Gemma Model Derivative and is provided under and subject to the
Gemma Terms of Use (ai.google.dev/gemma/terms). Its use is restricted by the
Gemma Prohibited Use Policy (ai.google.dev/gemma/prohibited_use_policy). To
access it, review and agree to those terms.
extra_gated_button_content: Acknowledge license
---
# pi0.5: Kinova Gen3 cube stacking
Color-conditioned cube stacking on a Kinova Gen3. Given a prompt such as
`stack the red cube on the green cube`, the policy picks the named cube and places it on the
named target.
## License
[Gemma Terms of Use](https://ai.google.dev/gemma/terms). `model.safetensors` contains modified
Gemma weights, derived from pi0.5's PaliGemma backbone and Gemma action expert, and is
distributed with [`LICENSE_GEMMA.txt`](LICENSE_GEMMA.txt) and [`NOTICE`](NOTICE).
Use is restricted by the
[Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). Pass these
terms on to anyone you redistribute the weights to.
## Model description
- **Inputs:** three 480x480 RGB views keyed `scene`, `wrist`, and `overview`, an 8-dimensional
joint state, and a language instruction
- **Outputs:** 8-dimensional joint position targets: 7 Kinova Gen3 arm joints and the Robotiq
2F-85 gripper
- **Training objective:** flow matching
- **Control rate:** 10 Hz, action chunks of 50
Serving needs `--state-dim 8`. `config.json` declares `observation.state` as `[32]`, pi0.5's
padded architecture width, while the normalizer statistics are 8-wide.
## Training
`lerobot-train`, pi0.5 LoRA-backbone: LoRA (r=32, alpha=64) on the PaliGemma language model's
attention and MLP projections, dense training for the action expert and projection heads, SigLIP
frozen. 50,000 steps, batch 4, lr 2.5e-5 cosine, weight decay 0.01, seed 1000, bfloat16. See
`train_config.json` for the full configuration.
Trained on
[`kinova_gen3_cube_stack_sim`](https://huggingface.co/datasets/PickNikRobotics/kinova_gen3_cube_stack_sim),
360 episodes across all six color-ordered prompts.
## Evaluation
150 held-out layouts in MuJoCo, real-time chunking on.
| gate | rate |
|---|---|
| **pick**, the named cube grasped and lifted | 94.0% |
| **stack**, placed on the named target | 81.3% |
|