Download README.md from cognaclee/Frank-Insert-Wipe: direct link, hf CLI and curl.
- Browser
- Download file 28.3 kB
-
https://huggingface.co/cognaclee/Frank-Insert-Wipe/resolve/main/README.md
- Command line
-
hf download hf://cognaclee/Frank-Insert-Wipe/README.md
-
curl -L -o README.md https://huggingface.co/cognaclee/Frank-Insert-Wipe/resolve/main/README.md
tags:
- robotics
- openpi
- pi0.5
- franka
library_name: openpi
Frank-Insert-Wipe: AdaSpeVLA checkpoints for Franka insert_tube, wipe_whiteboard, bread, test_tube, tennis_balls and a 5-task co-trained model (H=16)
AdaSpeVLA is a pi0.5 (OpenPI, JAX) variant. It predicts cubic B-spline action chunks with adaptive knots, plus a timing head that decides when to query the policy again. This repo holds two variants that differ only in the generator that produces the 11 control points:
- flow_head (
<task>/, 2026-11-12): a 20.4 M FlowPolicy trained from scratch. - pi05_expert (
<task>_expert/, 2026-11-14): pi0.5's pretrained Gemma-300M action expert (427.9 M with adaRMS, frompi05_base) runs on the 11 control-point tokens, as pi0.5 does on its action tokens.
Everything else (data, norm stats, H=16, K=11, labels, knot head, timing head, 40k + 10k steps) is the same.
The training code for both is in code/.
These are params-only Orbax checkpoints in the OpenPI layout. The optimizer train_state is not included.
| Folder | Training stage | Config name (with TASK=<task>) |
Use |
|---|---|---|---|
<task>/timing-from40k-step-9999/ |
Timing head, 10k steps, from Action step 39999 (everything except freq_head frozen) |
<task>_adaspe_timing_from40k |
Deployment: full model with the trained timing head |
<task>/action-step-39999/ |
Action stage, 40k steps (EMA params) | <task>_adaspe_action |
Action generator / knot head only. Its timing head is untrained. |
<task>_expert/timing-from40k-step-9999/ |
Same, pi05_expert generator | <task>_adaspe_expert_timing_from40k |
Deployment (expert variant) |
<task>_expert/action-step-39999/ |
Same, pi05_expert generator | <task>_adaspe_expert_action |
Action generator / knot head only (expert variant) |
<task>/timing-from40k-lr3e-4-step-9999/ |
Timing head only, retrained 2026-11-14 with peak LR 3e-4 (same 10k steps; see below) | <task>_adaspe_timing_from40k_lr3e-4 |
Deployment candidate |
<task>/timing-from40k-lr1e-3-step-9999/ |
Timing head only, retrained 2026-11-14 with peak LR 1e-3 (same 10k steps; see below) | <task>_adaspe_timing_from40k_lr1e-3 |
Deployment candidate |
<task>_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, retrained 2026-11-14 with peak LR 3e-4 (same 10k steps; see below) | <task>_adaspe_expert_timing_from40k_lr3e-4 |
Deployment candidate (expert variant) |
<task>_expert/timing-from40k-lr1e-3-step-9999/ |
Timing head only, retrained 2026-11-14 with peak LR 1e-3 (same 10k steps; see below) | <task>_adaspe_expert_timing_from40k_lr1e-3 |
Deployment candidate (expert variant) |
<task> is insert_tube or wipe_whiteboard. Each folder has params/, assets/dddddsds31341/<dataset>/norm_stats.json
(the normalization stats used in training) and _CHECKPOINT_METADATA. SHA256SUMS lists every file.
Loading
You need the AdaSpeVLA OpenPI plugin, included here as code/adaspe_train (run with TASK=<task>; it registers
these config names). It runs on OpenPI at the commit in code/OPENPI_COMMIT. Setup is in code/README.md, and
code/adaspe_train/EXPERT_VARIANT.md documents the expert variant.
The folders use the standard OpenPI checkpoint layout (params/ + assets/).
from huggingface_hub import snapshot_download
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["insert_tube/timing-from40k-step-9999/*"])
# higher-LR timing head: "insert_tube/timing-from40k-lr3e-4-step-9999/*" with config insert_tube_adaspe_timing_from40k_lr3e-4
ckpt_dir = f"{root}/insert_tube/timing-from40k-step-9999" # with config insert_tube_adaspe_timing_from40k
Data
| Task | Dataset (revision) | Size | pi0.5 baseline (same norm stats) |
|---|---|---|---|
| insert_tube | dddddsds31341/insert-tube-lerobot (70bcd1f86ed35fc93a0422a564472355242a51ab) |
102 episodes / 16,403 frames | dddddsds31341/Franka-Insert-Tube-pi05-step-20000 |
| wipe_whiteboard | dddddsds31341/wipe-whiteboard-lerobot (2c8cfa8ddef1f83cb09ca7881a0944cd637cd6d0) |
112 episodes / 29,795 frames | dddddsds31341/Franka-Wipe-Whiteboard-pi05-step-20000 |
Two 128Γ128 cameras, 19-D state, 7-D action, 10 Hz.
Recipe
- Action horizon H=16 (1.5 s), 11 cubic B-spline control points, query interval 0.2β1.5 s.
- Gripper events for the timing and knot labels come from the gripper command (the measured width lags by about 1.9 s).
- Knot density is tanh(motion_variation / e_crit), with e_crit calibrated per task at H=16: insert_tube 11.98, wipe_whiteboard 8.96.
- Initialized from
pi05_base. Batch 32, Action 40k steps (LR decay over 30k), then timing 10k steps.
Training loss
Mean training loss over the last 1kβ2k steps of each stage.
| Task | Variant | Action loss, steps 38kβ40k | Timing loss (log-interval MSE), steps 9kβ10k |
|---|---|---|---|
| insert_tube | flow_head | 0.086 | 0.206 |
| insert_tube | pi05_expert | 0.030 | 0.255 |
| wipe_whiteboard | flow_head | 0.118 | 0.196 |
| wipe_whiteboard | pi05_expert | 0.040 | 0.271 |
Action loss over training (mean per 2k-step window):
| Steps | insert_tube flow_head | insert_tube expert | wipe flow_head | wipe expert |
|---|---|---|---|---|
| 0β2k | 0.624 | 0.334 | 0.597 | 0.295 |
| 8β10k | 0.238 | 0.121 | 0.238 | 0.121 |
| 18β20k | 0.156 | 0.067 | 0.169 | 0.083 |
| 28β30k | 0.105 | 0.038 | 0.130 | 0.049 |
| 38β40k | 0.086 | 0.030 | 0.118 | 0.040 |
Notes:
- The two variants use different flow-matching conventions (flow_head: t ~ U(0,1); pi05_expert: pi0.5's Beta(1.5,1)), so the action losses are not exactly the same metric.
- The expert's timing head has a higher loss. It reads a pooled observation feature that, in the expert
variant, is trained only by the knot loss (see
code/adaspe_train/EXPERT_VARIANT.md). - These are training losses. No held-out or real-robot evaluation results are included here.
Timing head retrained with a higher learning rate (2026-11-14)
The original timing heads (timing-from40k-step-9999) used OpenPI's default schedule: peak LR 2.5e-5 with a cosine
decay over 30k steps. The timing stage only runs 10k steps, so the LR stayed at about 2.0β2.5e-5, which is low for
freq_head, a freshly initialized 2-layer MLP (633 β 128 β 1, ~81k params) and the only trainable part. The loss was
still falling at 10k.
The timing-from40k-lr<LR>-step-9999 folders retrain only freq_head, with the same data, labels, 10k steps,
batch 32 and 1k warmup, a cosine decay to LR/10 over the 10k steps, and peak LR 3e-4 or 1e-3. They start from the
same Action step-39999 params, and everything except freq_head is frozen. The predicted actions, knots and
B-spline are therefore identical to timing-from40k-step-9999. Only the query interval (and so execute_steps)
changes.
Timing loss (log-interval MSE, training loss, mean over steps 9kβ10k):
| Task | Variant | LR 2.5e-5 (original) | LR 3e-4 | LR 1e-3 |
|---|---|---|---|---|
| insert_tube | flow_head | 0.206 | 0.157 | 0.091 |
| insert_tube | pi05_expert | 0.255 | 0.182 | 0.100 |
| wipe_whiteboard | flow_head | 0.196 | 0.158 | 0.112 |
| wipe_whiteboard | pi05_expert | 0.271 | 0.214 | 0.153 |
These are training losses. With the higher LR, part of the drop may come from fitting the training frames; a
held-out comparison has not been done yet. The configs are configs.TIMINGS_LR in code/adaspe_train/configs.py,
and code/adaspe_train/slurm/timing_lr.slurm runs one of them.
Bread (2026-11-15, pi05_expert only)
bread_expert/ holds the bread task trained with the same H=16 recipe and the pi05_expert generator. There
is no flow_head version of bread. Unlike the other tasks, every saved checkpoint is included (params only):
| Folder | Training stage | Config name (with TASK=bread) |
Use |
|---|---|---|---|
bread_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, peak LR 3e-4, 10k steps, from Action step 39999 | bread_adaspe_expert_timing_from40k_lr3e-4 |
Deployment (recommended) |
bread_expert/timing-from40k-step-9999/ |
Timing head, default LR (peak 2.5e-5), from Action step 39999 | bread_adaspe_expert_timing_from40k |
Deployment, original timing LR |
bread_expert/timing-from30k-step-9999/ |
Timing head, default LR, from Action step 30000 | bread_adaspe_expert_timing_from30k |
Ablation |
bread_expert/timing-from20k-step-9999/ |
Timing head, default LR, from Action step 20000 | bread_adaspe_expert_timing_from20k |
Ablation |
bread_expert/action-step-{10000,20000,30000,39999}/ |
Action stage (EMA params) at that step | bread_adaspe_expert_action |
Action generator / knot head only. The timing head is untrained. |
Every timing folder is a full model. Only freq_head was trained in the timing stage, so actions and knots equal
those of the Action checkpoint it starts from.
Data: dddddsds31341/bread-lerobot (revision a45b8ba49fc164fb05d8a9c8bdf871ff97004357), 100 episodes /
28,380 frames, 10 Hz. e_crit = 10.01 (calibrated at H=16). Training: batch 32, Action 40k steps (OpenPI default LR:
1k warmup, peak 2.5e-5, cosine decay to 2.5e-6 at 30k), then the timing stage (10k steps).
Action loss (training, mean per 2k-step window):
| Steps | bread expert |
|---|---|
| 0β2k | 0.357 |
| 8β10k | 0.161 |
| 18β20k | 0.114 |
| 28β30k | 0.093 |
| 38β40k | 0.083 |
Timing loss (log-interval MSE, training, steps 9kβ10k): from20k 0.217, from30k 0.217, from40k 0.212, from40k lr3e-4 0.159.
These are training losses. No held-out or real-robot evaluation results are included. code/adaspe_train has the
bread settings (tasks.py, with checkpoints saved every 10k steps).
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["bread_expert/timing-from40k-lr3e-4-step-9999/*"])
ckpt_dir = f"{root}/bread_expert/timing-from40k-lr3e-4-step-9999" # TASK=bread, config bread_adaspe_expert_timing_from40k_lr3e-4
Five-task co-training (2026-11-15, pi05_expert only)
cotrain5_expert/ is one model for all five tasks of the dddddsds31341/franka-data collection, trained with the
same H=16 recipe and the pi05_expert generator. The task is chosen by the prompt. Use the dataset's task string
exactly:
| Task | Dataset | Revision | Episodes | Frames | Share of training | Prompt |
|---|---|---|---|---|---|---|
bread |
dddddsds31341/bread-lerobot |
a45b8ba49fc1 |
100 | 28,380 | 19.3% | Pick up both slices of bread from the toaster and place them on the plate. |
insert_tube |
dddddsds31341/insert-tube-lerobot |
70bcd1f86ed3 |
102 | 16,403 | 11.2% | Insert the gray tube into the empty black holder. |
wipe_whiteboard |
dddddsds31341/wipe-whiteboard-lerobot |
2c8cfa8ddef1 |
112 | 29,795 | 20.3% | Erase the writing from the whiteboard. |
tennis_balls |
dddddsds31341/tennis-balls-lerobot |
04a27cd5ffe2 |
100 | 36,950 | 25.1% | Place all tennis balls into the green basket. |
test_tube |
dddddsds31341/test-tube-lerobot |
29d821a1a711 |
99 | 35,425 | 24.1% | Place the test tube into the test tube rack. |
The five datasets are merged into one local LeRobot v2.1 dataset (code/adaspe_train/merge_datasets.py; record in
cotrain5_expert/merge_provenance.json): 513 episodes, 146,953 frames, 10 Hz. Batches are sampled
uniformly over frames (no per-task reweighting), so each task's share equals its share of frames.
Differences from the single-task models:
- Norm stats were computed on the merged data. They are in each folder at
assets/local/franka-cotrain5/norm_stats.json. The per-task stats do not apply to this model. - One set of knot/timing constants for all tasks, calibrated on the merged data (
cotrain5_expert/calibration_cotrain5_H16.json): e_crit = 9.216; timing labels 0.2 s 10.9%, 1.5 s 57.9%. - Same step budget as one single-task run (Action 40k steps, batch 32), so each frame is seen about 8.7 times (single-task runs see each frame about 45 times).
| Folder | Training stage | Config name (with TASK=cotrain5) |
Use |
|---|---|---|---|
cotrain5_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, peak LR 3e-4, 10k steps, from Action step 39999 | cotrain5_adaspe_expert_timing_from40k_lr3e-4 |
Deployment (recommended) |
cotrain5_expert/timing-from40k-step-9999/ |
Timing head, default LR (peak 2.5e-5), from Action step 39999 | cotrain5_adaspe_expert_timing_from40k |
Deployment, original timing LR |
cotrain5_expert/timing-from30k-step-9999/ |
Timing head, default LR, from Action step 30000 | cotrain5_adaspe_expert_timing_from30k |
Ablation |
cotrain5_expert/timing-from20k-step-9999/ |
Timing head, default LR, from Action step 20000 | cotrain5_adaspe_expert_timing_from20k |
Ablation |
cotrain5_expert/action-step-{10000,20000,30000,39999}/ |
Action stage (EMA params) at that step | cotrain5_adaspe_expert_action |
Action generator / knot head only. The timing head is untrained. |
Every timing folder is a full model. Only freq_head was trained in the timing stage, so actions and knots equal
those of the Action checkpoint it starts from.
Action loss (training, all five tasks mixed, mean per 2k-step window):
| Steps | cotrain5 expert |
|---|---|
| 0β2k | 0.345 |
| 8β10k | 0.192 |
| 18β20k | 0.146 |
| 28β30k | 0.123 |
| 38β40k | 0.110 |
Timing loss (log-interval MSE, training, steps 9kβ10k): from20k 0.287, from30k 0.269, from40k 0.267, from40k lr3e-4 0.232.
These are training losses. No held-out or real-robot evaluation results are included. code/adaspe_train now has the
cotrain5 task (tasks.py) and merge_datasets.py. To retrain, download the five datasets, run
TASK=cotrain5 python adaspe_train/merge_datasets.py, then train with TASK=cotrain5 VARIANT=expert.
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["cotrain5_expert/timing-from40k-lr3e-4-step-9999/*"])
ckpt_dir = f"{root}/cotrain5_expert/timing-from40k-lr3e-4-step-9999" # TASK=cotrain5, config cotrain5_adaspe_expert_timing_from40k_lr3e-4
prompt = "Place all tennis balls into the green basket." # one of the five task strings above
Test tube (2026-11-16, pi05_expert only)
test_tube_expert/ holds a single-task model for the test-tube task, trained with the same H=16 recipe and the
pi05_expert generator. (cotrain5_expert/ also covers this task, but it is shared with four other tasks.) There
is no flow_head version. Every saved checkpoint is included (params only):
| Folder | Training stage | Config name (with TASK=test_tube) |
Use |
|---|---|---|---|
test_tube_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, peak LR 3e-4, 10k steps, from Action step 39999 | test_tube_adaspe_expert_timing_from40k_lr3e-4 |
Deployment (recommended) |
test_tube_expert/timing-from40k-step-9999/ |
Timing head, default LR (peak 2.5e-5), from Action step 39999 | test_tube_adaspe_expert_timing_from40k |
Deployment, original timing LR |
test_tube_expert/timing-from30k-step-9999/ |
Timing head, default LR, from Action step 30000 | test_tube_adaspe_expert_timing_from30k |
Ablation |
test_tube_expert/timing-from20k-step-9999/ |
Timing head, default LR, from Action step 20000 | test_tube_adaspe_expert_timing_from20k |
Ablation |
test_tube_expert/action-step-{10000,20000,30000,39999}/ |
Action stage (EMA params) at that step | test_tube_adaspe_expert_action |
Action generator / knot head only. The timing head is untrained. |
Every timing folder is a full model. Only freq_head was trained in the timing stage, so actions and knots equal
those of the Action checkpoint it starts from.
Data: dddddsds31341/test-tube-lerobot (revision 29d821a1a71180948773e29c791f906306ac8018), 99 episodes /
35,425 frames, 10 Hz. Prompt (the dataset's task string): Place the test tube into the test tube rack.
- Norm stats were computed on this dataset (there is no baseline-repo copy). They are in each folder at
assets/dddddsds31341/test-tube-lerobot/norm_stats.json, and also incode/adaspe_train/assets/pi05_test_tube_franka_ref/. - Knot/timing constants were calibrated on this dataset at H=16 (
test_tube_expert/calibration_test_tube_H16.json): e_crit = 9.930; timing labels 0.2 s 10.6%, 1.5 s 58.0%. - Training: batch 32, Action 40k steps (OpenPI default LR: 1k warmup, peak 2.5e-5, cosine decay to 2.5e-6 at 30k), then the timing stage (10k steps).
Action loss (training, mean per 2k-step window):
| Steps | test_tube expert |
|---|---|
| 0β2k | 0.328 |
| 8β10k | 0.151 |
| 18β20k | 0.102 |
| 28β30k | 0.065 |
| 38β40k | 0.056 |
Timing loss (log-interval MSE, training, steps 9kβ10k): from20k 0.223, from30k 0.217, from40k 0.216, from40k lr3e-4 0.171.
These are training losses. No held-out or real-robot evaluation results are included. code/adaspe_train has the
test_tube settings (tasks.py) and its calibration, timing-label and norm-stats files.
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["test_tube_expert/timing-from40k-lr3e-4-step-9999/*"])
ckpt_dir = f"{root}/test_tube_expert/timing-from40k-lr3e-4-step-9999" # TASK=test_tube, config test_tube_adaspe_expert_timing_from40k_lr3e-4
prompt = "Place the test tube into the test tube rack."
Tennis balls (2026-11-17, pi05_expert only)
tennis_balls_expert/ holds a single-task model for the tennis-balls task, trained with the same H=16 recipe and the
pi05_expert generator. (cotrain5_expert/ also covers this task, but it is shared with four other tasks.) There
is no flow_head version. Every saved checkpoint is included (params only):
| Folder | Training stage | Config name (with TASK=tennis_balls) |
Use |
|---|---|---|---|
tennis_balls_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, peak LR 3e-4, 10k steps, from Action step 39999 | tennis_balls_adaspe_expert_timing_from40k_lr3e-4 |
Deployment (recommended) |
tennis_balls_expert/timing-from40k-step-9999/ |
Timing head, default LR (peak 2.5e-5), from Action step 39999 | tennis_balls_adaspe_expert_timing_from40k |
Deployment, original timing LR |
tennis_balls_expert/timing-from30k-step-9999/ |
Timing head, default LR, from Action step 30000 | tennis_balls_adaspe_expert_timing_from30k |
Ablation |
tennis_balls_expert/timing-from20k-step-9999/ |
Timing head, default LR, from Action step 20000 | tennis_balls_adaspe_expert_timing_from20k |
Ablation |
tennis_balls_expert/action-step-{10000,20000,30000,39999}/ |
Action stage (EMA params) at that step | tennis_balls_adaspe_expert_action |
Action generator / knot head only. The timing head is untrained. |
Every timing folder is a full model. Only freq_head was trained in the timing stage, so actions and knots equal
those of the Action checkpoint it starts from.
Data: dddddsds31341/tennis-balls-lerobot (revision 04a27cd5ffe22c250f0a23966256dc646b724a98), 100 episodes /
36,950 frames, 10 Hz. Prompt (the dataset's task string): Place all tennis balls into the green basket.
- Norm stats were computed on this dataset (there is no baseline-repo copy). They are in each folder at
assets/dddddsds31341/tennis-balls-lerobot/norm_stats.json, and also incode/adaspe_train/assets/pi05_tennis_balls_franka_ref/. - Knot/timing constants were calibrated on this dataset at H=16 (
tennis_balls_expert/calibration_tennis_balls_H16.json): e_crit = 8.338; timing labels 0.2 s 12.0%, 1.5 s 49.2%. - Training: batch 32, Action 40k steps (OpenPI default LR: 1k warmup, peak 2.5e-5, cosine decay to 2.5e-6 at 30k), then the timing stage (10k steps).
Action loss (training, mean per 2k-step window):
| Steps | tennis_balls expert |
|---|---|
| 0β2k | 0.404 |
| 8β10k | 0.182 |
| 18β20k | 0.126 |
| 28β30k | 0.084 |
| 38β40k | 0.071 |
Timing loss (log-interval MSE, training, steps 9kβ10k): from20k 0.295, from30k 0.290, from40k 0.288, from40k lr3e-4 0.231.
These are training losses. No held-out or real-robot evaluation results are included. code/adaspe_train has the
tennis_balls settings (tasks.py) and its calibration, timing-label and norm-stats files.
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["tennis_balls_expert/timing-from40k-lr3e-4-step-9999/*"])
ckpt_dir = f"{root}/tennis_balls_expert/timing-from40k-lr3e-4-step-9999" # TASK=tennis_balls, config tennis_balls_adaspe_expert_timing_from40k_lr3e-4
prompt = "Place all tennis balls into the green basket."
Bread retrain: no state (2026-11-19, pi05_expert only)
bread_nostate_expert/ is a retrain of the bread task on the same data as bread_expert/, with two recipe changes made after a
real-robot test of bread_expert. In that test the first close landed on air about one slice-height too high, the
policy still carried the empty gripper to the plate and opened it, and then closed around both slices (still in the
toaster) at the slice-2 position. pi0.5 has no memory, and the 19-D state in its prompt (gripper width, force, end-effector
pose, velocity) lets it tell the task phase from the arm pose rather than from the images.
- State input (no state):
state_keep=(): the whole 19-D state is zeroed after normalization, so the prompt carries no proprioception and the policy must read the task phase from the two cameras. The mask (configs.MaskState) is the first model transform, after normalization, so a policy built from this config (TASK=bread_nostate) applies it at inference as well. If your client builds the model input another way, apply the same mask. - Grasp-window oversampling (Action stage only): frames from 1.5 s before to 2 s after each gripper close command
(6,700 of 28,380 frames, 200 closes) are sampled 3Γ as often, so they make up
48% of the samples
(
code/adaspe_train/adaspe/oversample.py, indexcode/adaspe_train/assets/sample_index/bread_grasp_x3.npy). The timing stage samples uniformly.
Everything else is as in bread_expert/: data dddddsds31341/bread-lerobot (revision a45b8ba49fc164fb05d8a9c8bdf871ff97004357),
100 episodes / 28,380 frames, same norm stats (assets/dddddsds31341/bread-lerobot/norm_stats.json in each folder), same calibration
(e_crit = 10.008, bread_nostate_expert/calibration_bread_nostate_H16.json), H=16, K=11, batch 32, Action 40k steps, timing 10k steps.
Prompt: Pick up both slices of bread from the toaster and place them on the plate.
| Folder | Training stage | Config name (with TASK=bread_nostate) |
Use |
|---|---|---|---|
bread_nostate_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, peak LR 3e-4, 10k steps, from Action step 39999 | bread_nostate_adaspe_expert_timing_from40k_lr3e-4 |
Deployment (recommended) |
bread_nostate_expert/timing-from40k-step-9999/ |
Timing head, default LR, from Action step 39999 | bread_nostate_adaspe_expert_timing_from40k |
Deployment, original timing LR |
bread_nostate_expert/timing-from30k-step-9999/ |
Timing head, default LR, from Action step 30000 | bread_nostate_adaspe_expert_timing_from30k |
Ablation |
bread_nostate_expert/timing-from20k-step-9999/ |
Timing head, default LR, from Action step 20000 | bread_nostate_adaspe_expert_timing_from20k |
Ablation |
bread_nostate_expert/action-step-{10000,20000,30000,39999}/ |
Action stage (EMA params) at that step | bread_nostate_adaspe_expert_action |
Action generator / knot head only. The timing head is untrained. |
Action loss (training, mean per 2k-step window; grasp windows are oversampled, so not directly comparable with
bread_expert):
| Steps | bread_nostate expert |
|---|---|
| 0β2k | 0.329 |
| 8β10k | 0.191 |
| 18β20k | 0.138 |
| 28β30k | 0.097 |
| 38β40k | 0.090 |
Timing loss (log-interval MSE, training, steps 9kβ10k): from20k 0.224, from30k 0.213, from40k 0.206, from40k lr3e-4 0.158.
These are training losses. No held-out or real-robot evaluation results are included.
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["bread_nostate_expert/timing-from40k-lr3e-4-step-9999/*"])
ckpt_dir = f"{root}/bread_nostate_expert/timing-from40k-lr3e-4-step-9999" # TASK=bread_nostate, config bread_nostate_adaspe_expert_timing_from40k_lr3e-4
prompt = "Pick up both slices of bread from the toaster and place them on the plate."
Bread retrain: gripper width only (2026-11-19, pi05_expert only)
bread_gripstate_expert/ is a retrain of the bread task on the same data as bread_expert/, with two recipe changes made after a
real-robot test of bread_expert. In that test the first close landed on air about one slice-height too high, the
policy still carried the empty gripper to the plate and opened it, and then closed around both slices (still in the
toaster) at the slice-2 position. pi0.5 has no memory, and the 19-D state in its prompt (gripper width, force, end-effector
pose, velocity) lets it tell the task phase from the arm pose rather than from the images.
- State input (gripper width only):
state_keep=(0,): onlystate[0](gripper width) is kept; the other 18 dims are zeroed after normalization. The mask (configs.MaskState) is the first model transform, after normalization, so a policy built from this config (TASK=bread_gripstate) applies it at inference as well. If your client builds the model input another way, apply the same mask. - Grasp-window oversampling (Action stage only): frames from 1.5 s before to 2 s after each gripper close command
(6,700 of 28,380 frames, 200 closes) are sampled 3Γ as often, so they make up
48% of the samples
(
code/adaspe_train/adaspe/oversample.py, indexcode/adaspe_train/assets/sample_index/bread_grasp_x3.npy). The timing stage samples uniformly.
Everything else is as in bread_expert/: data dddddsds31341/bread-lerobot (revision a45b8ba49fc164fb05d8a9c8bdf871ff97004357),
100 episodes / 28,380 frames, same norm stats (assets/dddddsds31341/bread-lerobot/norm_stats.json in each folder), same calibration
(e_crit = 10.008, bread_gripstate_expert/calibration_bread_gripstate_H16.json), H=16, K=11, batch 32, Action 40k steps, timing 10k steps.
Prompt: Pick up both slices of bread from the toaster and place them on the plate.
| Folder | Training stage | Config name (with TASK=bread_gripstate) |
Use |
|---|---|---|---|
bread_gripstate_expert/timing-from40k-lr3e-4-step-9999/ |
Timing head only, peak LR 3e-4, 10k steps, from Action step 39999 | bread_gripstate_adaspe_expert_timing_from40k_lr3e-4 |
Deployment (recommended) |
bread_gripstate_expert/timing-from40k-step-9999/ |
Timing head, default LR, from Action step 39999 | bread_gripstate_adaspe_expert_timing_from40k |
Deployment, original timing LR |
bread_gripstate_expert/timing-from30k-step-9999/ |
Timing head, default LR, from Action step 30000 | bread_gripstate_adaspe_expert_timing_from30k |
Ablation |
bread_gripstate_expert/timing-from20k-step-9999/ |
Timing head, default LR, from Action step 20000 | bread_gripstate_adaspe_expert_timing_from20k |
Ablation |
bread_gripstate_expert/action-step-{10000,20000,30000,39999}/ |
Action stage (EMA params) at that step | bread_gripstate_adaspe_expert_action |
Action generator / knot head only. The timing head is untrained. |
Action loss (training, mean per 2k-step window; grasp windows are oversampled, so not directly comparable with
bread_expert):
| Steps | bread_gripstate expert |
|---|---|
| 0β2k | 0.348 |
| 8β10k | 0.190 |
| 18β20k | 0.136 |
| 28β30k | 0.085 |
| 38β40k | 0.076 |
Timing loss (log-interval MSE, training, steps 9kβ10k): from20k 0.224, from30k 0.212, from40k 0.208, from40k lr3e-4 0.156.
These are training losses. No held-out or real-robot evaluation results are included.
root = snapshot_download("cognaclee/Frank-Insert-Wipe", allow_patterns=["bread_gripstate_expert/timing-from40k-lr3e-4-step-9999/*"])
ckpt_dir = f"{root}/bread_gripstate_expert/timing-from40k-lr3e-4-step-9999" # TASK=bread_gripstate, config bread_gripstate_adaspe_expert_timing_from40k_lr3e-4
prompt = "Pick up both slices of bread from the toaster and place them on the plate."