model card

lift_leave_cubes_skin_3cam/29999/README.md

@@ -0,0 +1,66 @@
+---
+tags: [robotics, pi0, pi05, openpi, so101, lerobot, tactile]
+license: apache-2.0
+datasets: [endoard/grab_red_cube_3cam_skin, endoard/grab_green_cube_3cam_skin, endoard/grab_red_cube_distractor_green_cube_3cam_skin, endoard/grab_green_cube_distractor_red_cube_3cam_skin]
+---
+
+# PINlab/pi05-so101/lift_leave_cubes_skin_3cam/29999
+
+pi0.5 policy for a single SO-101 arm with a 5-chip magnetic tactile skin, prompts "Lift the red cube" / "Lift the green cube" / "Lift the red cube, leave the green cube on the table" / "Lift the green cube, leave the red cube on the table". Fine-tuned from `pi05_base` on `endoard/grab_red_cube_3cam_skin`, `endoard/grab_green_cube_3cam_skin`, `endoard/grab_red_cube_distractor_green_cube_3cam_skin`, `endoard/grab_green_cube_distractor_red_cube_3cam_skin` (merged, one task per source), experiment `liftleave_v1`,
+step 29999 of a 30k-step run. Not yet evaluated on the robot.
+
+## Prompts
+
+Trained on exactly these text prompts. At inference send one of them verbatim as the
+observation `prompt` (same wording, case and punctuation); other phrasings are untested.
+
+    Lift the red cube
+    Lift the green cube
+    Lift the red cube, leave the green cube on the table
+    Lift the green cube, leave the red cube on the table
+
+Which prompt each training episode carried (dataset task label -> prompt):
+
+| dataset task label | episodes | training prompt |
+|---|---|---|
+| Grab the red cube | 150 | Lift the red cube |
+| Grab the green cube | 150 | Lift the green cube |
+| Grab the red cube (green cube distractor) | 150 | Lift the red cube, leave the green cube on the table |
+| Grab the green cube (red cube distractor) | 100 | Lift the green cube, leave the red cube on the table |
+
+## Contents
+
+    params/                                weights (EMA, decay 0.99)
+    assets/grab_4cube_3cam_skin/norm_stats.json   quantile normalisation stats
+    _CHECKPOINT_METADATA                   orbax bookkeeping
+
+`train_state/` is not included: it only resumes training.
+
+## Serving
+
+The checkpoint carries no config. Serve it under `pi05_so101_lift_leave_cubes_skin_3cam` (it pins
+`asset_id=grab_4cube_3cam_skin`):
+
+    python scripts/serve_policy.py policy:checkpoint \
+        --policy.config=pi05_so101_lift_leave_cubes_skin_3cam \
+        --policy.dir=<downloaded dir>
+
+The policy returns 16 actions **spaced 2 frames apart (15 Hz)**, absolute
+joints (the delta is undone server-side). Replay at that rate or interpolate; replaying
+at 30 Hz halves the speed.
+
+## Schema
+
+- State 21-D (padded to 32): 5 joints (deg), gripper (0-100), 5 skin chips x xyz (uT).
+- Action 6-D (padded to 32): 5 joints (deg), gripper (0-100). **Delta on the 5
+  joints, absolute gripper** (`make_bool_mask(5, -1)`).
+- Images, all three every step: `base_0_rgb` = `fixed`, `left_wrist_0_rgb` = `handeye`,
+  `right_wrist_0_rgb` = `side` (mask on). The client must send all three cameras.
+- `action_horizon=16`, `action_stride=2`, prompt from the task string (mapped as above).
+
+## Training
+
+From `gs://openpi-assets/checkpoints/pi05_base/params` on 4 nodes x 1 A100-64GB,
+`--fsdp-devices=4`, batch 32, AdamW with cosine decay, EMA 0.99, seed 42,
+30000 steps. Data: 550 episodes / 191,324 frames at 30 fps,
+495 train / 55 held out.