SleepMastger/pi05_place_cube

π₀.₅ — place the cube in the bowl (real Franka)

A π₀.₅ vision-language-action policy fine-tuned on 100 real Franka Emika teleoperation demonstrations of "place the cube in the bowl".

Serving code, robot client and the full I/O contract: github.com/sleepmastergx/deploy, branch pi0.5

Checkpoints

Four training steps are published so you can evaluate across them. Only params/ and assets/ are here — train_state/ (optimizer state, 32 GB per step) is not needed for inference and is not published.

folder	step	≈ epochs	size
5000/	5,000	10	12.4 GB
10000/	10,000	20	12.4 GB
15000/	15,000	31	12.4 GB
19999/	20,000 (final)	41	12.4 GB

openpi numbers steps 0-indexed, so 19999 is the completed 20,000-step model.

Start with 19999, but do not assume it is best. 100 demonstrations over ~41 epochs is a deep-overfit regime; training loss reached 0.0016, which says the model fit the data, not that it will place cubes. Only real-robot rollouts settle which checkpoint to use — if 19999 looks memorised or brittle, work backwards.

assets/george/place_cube_new_first100/norm_stats.json inside each folder holds that checkpoint's normalization statistics. They travel with the checkpoint and must not be mixed between runs — the policy cannot be served without them, and the wrong ones produce mis-scaled actions rather than an error.

Training

Base	pi05_base (openpi's robot-pretrained π₀.₅)
Data	100 episodes / 15,624 frames @ 10 Hz, single Franka + 2 RealSense cameras
Steps	20,000, batch 32, AdamW, cosine 5e-5 → 5e-6 after 1k warmup, EMA 0.999
Model	action_dim=32, action_horizon=16, discrete_state_input=True
Hardware	2 × H200, ~4 h 25 m
Final loss	0.0016 (from 0.163)

Usage

hf download SleepMastger/pi05_place_cube --local-dir ./pi05_place_cube

git clone -b pi0.5 https://github.com/sleepmastergx/deploy.git
cd deploy/pi05_place_cube
PYTHONPATH=$PWD/src python serve_pi05.py --ckpt ../../pi05_place_cube/19999

Requires openpi installed (JAX + CUDA, Python ≥ 3.11) and a GPU with > 8 GB VRAM.

Interface

The policy server expects exactly these keys — no batch dimension, no client-side resizing, no client-side normalization:

{
  "observation/image":       uint8 (256, 256, 3),   # third-person / agentview
  "observation/wrist_image": uint8 (256, 256, 3),   # wrist
  "observation/state":       float (8,),            # RAW, unnormalized
  "prompt":                  "place the cube in the bowl",
}

with

state[0:3] = eef_pos                    # metres
state[3:6] = quat2axisangle(eef_quat)   # eef_quat is (x, y, z, w), scalar-last
state[6]   = +gripper_width / 2         # metres (~0.0796 open, ~0.029 closed)
state[7]   = -gripper_width / 2

Returns (16, 7) actions, already un-normalized:

• dims 0:3 — dx, dy, dz delta end-effector position in metres per 100 ms step
• dims 3:6 — rotation, always ≈ 0; this policy cannot rotate (training saw only zero rotations)
• dim 6 — gripper, 1.0 = OPEN, 0.0 = CLOSE, threshold at 0.5

Control rate is 10 Hz. Predict 16, execute ~8, re-plan.

The gripper polarity is the easiest thing to get backwards, so to be explicit: 1 is open. It was verified against the training data — action[6] steps 1 → 0 at exactly the frame state[6] falls 0.0398 → 0.0146 (width 0.0796 → 0.029, closing on the cube).

State is not a side channel: with discrete_state_input=True it is normalized, binned into 256 buckets and written into the text prompt, so a wrong state changes what the model reads. Full spec in CONTRACT.md.

Verification

Replaying held-out recorded episodes through the served policy and scoring against logged actions:

	episode_0	episode_1
xyz RMSE	0.00019 m	0.00020 m
ground-truth per-step RMS	0.00106 m (error/signal 0.18)	0.00121 m (0.16)
max |rpy|	3.3e-09	3.6e-09
gripper match	100 % (96/96)	100 % (96/96)
inference	~51 ms	~51 ms

Reproduce with tools/replay_test.py in the deploy repo. This measures whether a deployment reproduces the training contract — not whether the policy succeeds at the task, which only real rollouts can tell you.

Limitations

• No rotation control. Rotation was constant zero in every training episode.
• One task, one scene, one camera setup. 100 demonstrations from a single rig. Expect nothing outside that distribution.
• Deep-overfit regime. See the checkpoint note above.
• Camera framing matters. The policy is sensitive to it, and a mismatch degrades behaviour silently — it keeps acting confidently and does the wrong thing. Reproduce the collection framing, and run the replay test before trusting a new setup.

License

Released under the Gemma Terms of Use. This model is fine-tuned from openpi's pi05_base, whose vision-language backbone is PaliGemma/Gemma, so Gemma's terms carry through to this derivative. The openpi code is Apache-2.0 separately.