README.md

---
license: apache-2.0
library_name: openpi
pipeline_tag: robotics
tags: [robotics, manipulation, pi0.5, lora, vla, lerobot, openpi]
datasets: [IDEAS-Lab-Northwestern/sim-stack-retrieve-60-joint-3cam]
---

# π0.5 LoRA — Sim Stack-Retrieve (joint controller, 2-cam)

A LoRA fine-tune of **π0.5** (`pi05_base`) for one ManiGuard simulated
manipulation task, trained with the [openpi](https://github.com/Physical-Intelligence/openpi) trainer.

**Task — stack-retrieve:** pull the bottom object out of a same-object stack and
move it into the green goal sphere. Prompts: *flat object*, *chili pepper*, *bowl*.

## Model
- **Warm-start:** openpi `pi05_base`; LoRA on `gemma_2b_lora` + `gemma_300m_lora`. `action_dim=32` (padded), `action_horizon=16`.
- **Controller:** JointController — 8-D joint state, 8-D absolute-joint action (7 arm joints trained as per-step deltas, gripper absolute; reconstructed to absolute at inference).
- **Cameras (LIBERO 2-cam):** `base_0_rgb ← image_left` (third-person overview) + `left_wrist_0_rgb ← wrist`; third image slot zero-filled + masked. Dataset is 3-cam rendered; `image_right` unused.

## Data
[`sim-stack-retrieve-60-joint-3cam`](https://huggingface.co/datasets/IDEAS-Lab-Northwestern/sim-stack-retrieve-60-joint-3cam) — 60 episodes / 48,208 frames; GELLO-teleop collected, re-rendered to joint + 3-cam.

## Training
- 1x A100-80GB, bf16, full data-parallel (`fsdp_devices=1`).
- ~2 epochs: **4000 steps**, batch **24**, cosine LR (peak `4.3e-5`, warmup 400, decay `4.3e-6`).
- Final train loss = **0.01**.

## Checkpoints
Step folders `1000/ 2000/ 3000/ 4000/` — **`4000`** is the final ~2-epoch checkpoint.

## Usage
Serve via the openpi policy server with config `pi05_base_stack_retrieve_joint_2cam_lora`; feed `observation/image_left` (overview) + `observation/wrist_image` + 8-D joint `observation/state`.