Index 23 HF-data retraining PI0.5 peaks (pi05_hf21_20261025) with 50-episode evaluations

README.md

@@ -6,14 +6,14 @@ license: mit
 
 ## single_task
 
-One PI0.5 LoRA checkpoint per ManiLadder task (106 tasks), each finetuned on 20 demos.
+One PI0.5 LoRA checkpoint per ManiLadder task (108 tasks), each finetuned on 20 demos.
 Checkpoints are selected by the highest **50-episode** success rate; ties use the earliest step.
 
 - The 60 previously published tasks use fixed evaluation seeds 1000–1049.
 - The 22 completed retraining runs published through 2026-10-30 use seeds 800000–800049. All reached 40,000 training steps. 15 have all eight 5k–40k evaluations; 7 have seven evaluations because one intermediate checkpoint was not retained. Their selected weights are the best among the available evaluated checkpoints.
-- The 21 tasks from the HF-data retraining campaign `pi05_hf21_20261025` (added 2026-10-29; raw demos freshly downloaded from Linslab/ManiLadder_Dataset) use seeds 1000–1049. All reached 40,000 training steps and have all eight 5k–40k evaluations. See [its manifest](single_task/pi05_hf21_20261025_manifest.json).
 
 - The 3 fresh-data runs in `pi05_newdata_3tasks_20261028` use pinned dataset revision `d120457e637b01679fbb40fd4de1d2394f0d55c9`, 20 demos per task, 40,000 training steps, and all eight 5k–40k eval50 checkpoints with seeds 800000–800049. **All three scored 0/50 at every checkpoint**; step 5,000 is the earliest tied checkpoint. See [their manifest](single_task/pi05_newdata_3tasks_20261028_manifest.json).
+- The 23 tasks from the HF-data retraining campaign `pi05_hf21_20261025` (added 2026-10-29; raw demos freshly downloaded from Linslab/ManiLadder_Dataset) use seeds 1000–1049. All reached 40,000 training steps and have all eight 5k–40k evaluations. See [its manifest](single_task/pi05_hf21_20261025_manifest.json).
 
 See [retraining provenance and missing steps](single_task/pi05_retrain_eval50_20261022_manifest.json). These scores use different seed sets across the two groups and are not a controlled comparison.
 
@@ -24,7 +24,7 @@ Each `single_task/<task>/` folder contains:
 - `eval_result.json` — 50-episode evaluation of this checkpoint
 
 `single_task/results.csv` lists the selected checkpoint per task; `single_task/all_checkpoints_eval.csv` has the
-50-episode result of every evaluated checkpoint (838 total).
+50-episode result of every evaluated checkpoint (854 total).
 
 Evaluation: `exec_steps=16`, `sample_steps=10`. Tasks whose checkpoints all scored 0/50 are still included;
 for them the step choice is arbitrary.
@@ -107,6 +107,7 @@ for them the step choice is arbitrary.
 | HitBox-v1 | 3 | 25000 | 0.06 |
 | LiquidMeasuring-v1 | 3 | 35000 | 0.32 |
 | MobilePickPlace-v1 | 3 | 15000 | 0.08 |
+| PressFitSoftPeg-v1 | 3 | 20000 | 0.08 |
 | PushTargetPose-v1 | 3 | 30000 | 0.02 |
 | SeparateTwoRopesEnv-v1 | 3 | 35000 | 0.26 |
 | Stack6Blocks-v1 | 3 | 5000 | 0.00 |
@@ -135,6 +136,7 @@ for them the step choice is arbitrary.
 | RodThroughHoles-v1 | 4 | 5000 | 0.02 |
 | SpinPenHora-v1 | 4 | 5000 | 0.00 |
 | StampRoll-v1 | 4 | 5000 | 0.00 |
+| SwingRing-v1 | 4 | 15000 | 0.04 |
 | TennisRally-v1 | 4 | 15000 | 0.06 |
 | WrapRope-v1 | 4 | 5000 | 0.02 |
 

single_task/all_checkpoints_eval.csv

@@ -601,6 +601,14 @@ MobilePickPlace-v1,level_3,25000,0.02,1,nan
 MobilePickPlace-v1,level_3,30000,0.02,1,nan
 MobilePickPlace-v1,level_3,35000,0.04,2,169.25979777826228
 MobilePickPlace-v1,level_3,40000,0.04,2,nan
+PressFitSoftPeg-v1,level_3,5000,0.0,0,693.0195
+PressFitSoftPeg-v1,level_3,10000,0.0,0,1621.6199599999998
+PressFitSoftPeg-v1,level_3,15000,0.0,0,1761.0606
+PressFitSoftPeg-v1,level_3,20000,0.08,4,1640.69166
+PressFitSoftPeg-v1,level_3,25000,0.02,1,1553.72812
+PressFitSoftPeg-v1,level_3,30000,0.08,4,1815.9183600000001
+PressFitSoftPeg-v1,level_3,35000,0.08,4,1902.3642799999998
+PressFitSoftPeg-v1,level_3,40000,0.06,3,1725.1256
 PushTargetPose-v1,level_3,5000,0.0,0,-348.17943846166133
 PushTargetPose-v1,level_3,10000,0.0,0,-391.1001693305373
 PushTargetPose-v1,level_3,15000,0.0,0,-293.1471811267734
@@ -821,6 +829,14 @@ StampRoll-v1,level_4,25000,0.0,0,0.0
 StampRoll-v1,level_4,30000,0.0,0,0.0
 StampRoll-v1,level_4,35000,0.0,0,0.0
 StampRoll-v1,level_4,40000,0.0,0,0.0
+SwingRing-v1,level_4,5000,0.02,1,73.43362
+SwingRing-v1,level_4,10000,0.02,1,77.27958
+SwingRing-v1,level_4,15000,0.04,2,78.49958
+SwingRing-v1,level_4,20000,0.0,0,76.71624
+SwingRing-v1,level_4,25000,0.0,0,71.01814
+SwingRing-v1,level_4,30000,0.02,1,75.07314000000001
+SwingRing-v1,level_4,35000,0.0,0,73.35582000000001
+SwingRing-v1,level_4,40000,0.0,0,74.07002
 TennisRally-v1,level_4,5000,0.02,1,357.17158117782327
 TennisRally-v1,level_4,10000,0.02,1,582.0495989676378
 TennisRally-v1,level_4,15000,0.06,3,631.176888383422

single_task/pi05_hf21_20261025_manifest.json

@@ -155,6 +155,21 @@
       "checkpoint": "single_task/DualArmThreading-v1/lora_step_0020000.npz",
       "checkpoint_sha256": "7e815721a302d324c55dea8c9681332c939b65d432b3918e2a00d365828ffedd"
     },
+    {
+      "task": "PressFitSoftPeg-v1",
+      "level": "level_3",
+      "suite": "deformable/single_arm_dex",
+      "step": 20000,
+      "success_rate": 0.08,
+      "successes": 4,
+      "episodes": 50,
+      "episode_seed_base": 1000,
+      "ckpts_evaluated": 8,
+      "missing_eval_steps": [],
+      "campaign": "pi05_hf21_20261025",
+      "checkpoint": "single_task/PressFitSoftPeg-v1/lora_step_0020000.npz",
+      "checkpoint_sha256": "3f798221f49b418dee3fbf610282e8bbb29499d02705a072d0e4053cfcf051c4"
+    },
     {
       "task": "DualArmMakeLine-v1",
       "level": "level_3",
@@ -290,6 +305,21 @@
       "checkpoint": "single_task/CoverPattern-v1/lora_step_0030000.npz",
       "checkpoint_sha256": "0f555c7b620fa9355b6f56ee378c7531d89e630c9b6383210cdac21cd2dae020"
     },
+    {
+      "task": "SwingRing-v1",
+      "level": "level_4",
+      "suite": "deformable/single_arm_dex",
+      "step": 15000,
+      "success_rate": 0.04,
+      "successes": 2,
+      "episodes": 50,
+      "episode_seed_base": 1000,
+      "ckpts_evaluated": 8,
+      "missing_eval_steps": [],
+      "campaign": "pi05_hf21_20261025",
+      "checkpoint": "single_task/SwingRing-v1/lora_step_0015000.npz",
+      "checkpoint_sha256": "4ba44fb21dbbc9435859c93e0796840c75f7855ba3fc996e4b748f6d4360085e"
+    },
     {
       "task": "StampRoll-v1",
       "level": "level_4",

single_task/results.csv

@@ -75,6 +75,7 @@ HammerNail-v1,level_3,35000,0.12,6,8,single_task/HammerNail-v1/lora_step_0035000
 HitBox-v1,level_3,25000,0.06,3,8,single_task/HitBox-v1/lora_step_0025000.npz
 LiquidMeasuring-v1,level_3,35000,0.32,16,8,single_task/LiquidMeasuring-v1/lora_step_0035000.npz
 MobilePickPlace-v1,level_3,15000,0.08,4,8,single_task/MobilePickPlace-v1/lora_step_0015000.npz
+PressFitSoftPeg-v1,level_3,20000,0.08,4,8,single_task/PressFitSoftPeg-v1/lora_step_0020000.npz
 PushTargetPose-v1,level_3,30000,0.02,1,8,single_task/PushTargetPose-v1/lora_step_0030000.npz
 SeparateTwoRopesEnv-v1,level_3,35000,0.26,13,7,single_task/SeparateTwoRopesEnv-v1/lora_step_0035000.npz
 Stack6Blocks-v1,level_3,5000,0.0,0,8,single_task/Stack6Blocks-v1/lora_step_0005000.npz
@@ -103,5 +104,6 @@ RingTossing-v1,level_4,5000,0.0,0,8,single_task/RingTossing-v1/lora_step_0005000
 RodThroughHoles-v1,level_4,5000,0.02,1,8,single_task/RodThroughHoles-v1/lora_step_0005000.npz
 SpinPenHora-v1,level_4,5000,0.0,0,8,single_task/SpinPenHora-v1/lora_step_0005000.npz
 StampRoll-v1,level_4,5000,0.0,0,8,single_task/StampRoll-v1/lora_step_0005000.npz
+SwingRing-v1,level_4,15000,0.04,2,8,single_task/SwingRing-v1/lora_step_0015000.npz
 TennisRally-v1,level_4,15000,0.06,3,8,single_task/TennisRally-v1/lora_step_0015000.npz
 WrapRope-v1,level_4,5000,0.02,1,8,single_task/WrapRope-v1/lora_step_0005000.npz