|
Download code/adaspe_train/EXPERT_VARIANT.md from cognaclee/Frank-Insert-Wipe: direct link, hf CLI and curl.
- Browser
- Download file 8.65 kB
-
https://huggingface.co/cognaclee/Frank-Insert-Wipe/resolve/main/code/adaspe_train/EXPERT_VARIANT.md
- Command line
-
hf download hf://cognaclee/Frank-Insert-Wipe/code/adaspe_train/EXPERT_VARIANT.md
-
curl -L -o EXPERT_VARIANT.md https://huggingface.co/cognaclee/Frank-Insert-Wipe/resolve/main/code/adaspe_train/EXPERT_VARIANT.md
8.65 kB
| # AdaSpeVLA expert variant (`VARIANT=expert`) | |
| Status as of 2026-11-14 10:00: **training finished** (Action 40k + 3 timing heads per task, ALL DONE 09:54 / 09:55). | |
| Final checkpoints (params only) are on HF: `cognaclee/Frank-Insert-Wipe`, folders `<task>_expert/`. | |
| ## Why | |
| On the real robot, the baseline pi0.5 (`dddddsds31341/Franka-*-pi05-step-20000`, action_horizon=10, 20k steps) | |
| was clearly more precise than AdaSpeVLA (the Nov 12 `flow_head` run). Offline checks ruled out the B-spline as the | |
| cause: over a 9-step prefix, the fitting error at p95 is β€ 0.05 of one typical step. The main difference was the | |
| action generator: | |
| - AdaSpeVLA `flow_head`: a 20.4 M flow transformer **trained from scratch**. `PluginWeightLoader` discarded | |
| pi0.5's pretrained action expert. | |
| - pi0.5: a 427.9 M pretrained action expert (311.4 M Gemma-300M transformer + 116.5 M adaRMS). | |
| Option A: put the pretrained pi0.5 expert back and have it predict the 11 control points. | |
| ## What changed | |
| Only the part that **produces the control points** changed. | |
| | | `flow_head` (Nov 12 run) | `pi05_expert` (this run) | | |
| |---|---|---| | |
| | Generator | AdaSpeVLA flow transformer, 20.4 M, random init | pi0.5 Gemma-300M action expert + adaRMS, 427.9 M, loaded from `pi05_base` | | |
| | Observation access | cross-attention to projected last-layer prefix tokens (`token_proj`, `view_embedding`, `text_token_proj`) | attends to the PaliGemma prefix KV cache at every layer, as in pi0.5 (gradients flow through the cache) | | |
| | Tokens | K=11 control-point tokens + global context | K=11 control-point tokens via pi0.5's `action_in_proj`; time via pi0.5's `time_mlp_in/out` (adaRMS); output via `action_out_proj` | | |
| | Knot information | knot vector + log horizon concatenated into the context | **new** `cp_knot_proj` (zero-init): sin-cos of each control point's Greville abscissa on the predicted knots, plus the knot vector and log horizon, added to each token. Zero init means step 0 receives exactly pi0.5's inputs | | |
| | Flow convention | x_t = tΒ·data + (1βt)Β·noise, u = data β noise, t ~ U(0,1) | pi0.5: x_t = tΒ·noise + (1βt)Β·data, u = noise β data, t ~ Beta(1.5,1)Β·0.999+0.001 | | |
| | ODE steps (inference / timing stage) | 15 / 15 | 10 / 10 (pi0.5 default) | | |
| | Weight loading | PaliGemma only | PaliGemma + action expert (`llm/*_1`) + `action_in_proj`, `action_out_proj`, `time_mlp_in`, `time_mlp_out`. **Fails** if any pretrained param is missing or has the wrong shape | | |
| **Unchanged:** data, norm stats (`assets/baseline`), H=16, K=11, cubic B-spline, query interval 0.2β1.5 s, | |
| timing and knot labels (gripper *command* events, `adaspe/labels.py`), knot head, KL knot loss, timing head, | |
| spline fitting, `sample_spline` execute_steps, optimizer/LR/batch (`_COMMON`), 40k Action steps, and | |
| timing heads trained from the 20k/30k/40k checkpoints. | |
| ### Code | |
| - `adaspe/model.py`: `AdaSpePi05Config.generator: "flow_head" | "pi05_expert"`. Expert path in `_cp_token_bias`, | |
| `_expert_velocity`, and the expert branches of `sample_control_points` and `compute_loss`. | |
| - `configs.py`: `GENERATORS`, `_EXPERT_KNOBS` (10 ODE steps), configs `<TASK>_adaspe_expert_action` and | |
| `<TASK>_adaspe_expert_timing_from{20,30,40}k`. Strict loader check in `PluginWeightLoader`. | |
| - `names.sh`: `VARIANT=expert` sets `TAG=<TASK>_adaspe_expert` (configs, checkpoints, logs) and | |
| `JOB_BASE=adaspe-expert` (Slurm job names). | |
| - Tests: `tests/test_expert_parity.py` (the expert pass matches OpenPI's own Pi0 suffix code), | |
| `tests/check_config_names.py`, and `tests/smoke_model.py` for both generators. The `flow_head` smoke losses are | |
| identical to job 34962, so the old path is unchanged. | |
| - Not committed to git yet. | |
| ## Run | |
| | | insert_tube | wipe_whiteboard | | |
| |---|---|---| | |
| | Account / QoS | yfl@h100, dev chain (2 h links, 4 H100, fsdp=4) | same | | |
| | First link | 130181, started 2026-11-14 02:02, preflight passed 02:07 | 130182, started 02:10, preflight passed 02:15 | | |
| | Links | 130181 β 170553 β 172225 β 176486 (ALL DONE 09:54) | 130182 β 170656 β 172371 β 176654 (ALL DONE 09:55) | | |
| | Action done | 2026-11-14 08:36 | 2026-11-14 08:36 | | |
| | Timing heads done | 09:54 (2.2 it/s, 1 GPU each) | 09:55 | | |
| | Speed | 2.1 it/s (flow_head: 2.2 it/s) | 2.1 it/s | | |
| | Checkpoints | `checkpoints/adaspe/insert_tube_adaspe_expert_action/formal/` 10000, 20000, 30000, 39999; `insert_tube_adaspe_expert_timing_from{20,30,40}k/formal/9999` | same under `wipe_whiteboard_adaspe_expert_*` | | |
| | Logs | `test/franka/insert_tube_adaspe_expert/` | `test/franka/wipe_whiteboard_adaspe_expert/` | | |
| The backup t3 chains 135256 / 135257 were cancelled automatically when the dev chains finished. | |
| ## Results: training loss (Action stage) | |
| Mean training loss over each 2k-step window, from `test/franka/<TASK>_<variant>/action.log`. | |
| | Steps | insert_tube flow_head | insert_tube **expert** | wipe flow_head | wipe **expert** | | |
| |---|---|---|---|---| | |
| | 0β2k | 0.624 | **0.334** | 0.597 | **0.295** | | |
| | 2β4k | 0.371 | **0.180** | 0.358 | **0.170** | | |
| | 4β6k | 0.300 | **0.152** | 0.296 | **0.146** | | |
| | 6β8k | 0.266 | **0.134** | 0.260 | **0.131** | | |
| | 8β10k | 0.238 | **0.121** | 0.238 | **0.121** | | |
| | 10β12k | 0.218 | **0.105** | 0.218 | **0.114** | | |
| | 12β14k | 0.202 | **0.098** | 0.207 | **0.105** | | |
| | 14β16k | 0.186 | **0.087** | 0.192 | **0.098** | | |
| | 16β18k | 0.173 | **0.076** | 0.182 | **0.087** | | |
| | 18β20k | 0.156 | **0.067** | 0.169 | **0.083** | | |
| | 20β22k | 0.144 | **0.058** | 0.160 | **0.073** | | |
| | 22β24k | 0.132 | **0.054** | 0.151 | **0.066** | | |
| | 24β26k | 0.120 | **0.047** | 0.140 | **0.058** | | |
| | 26β28k | 0.111 | **0.044** | 0.135 | **0.053** | | |
| | 28β30k | 0.105 | **0.038** | 0.130 | **0.049** | | |
| | 30β32k | 0.100 | **0.040** | 0.127 | **0.046** | | |
| | 32β34k | 0.095 | **0.032** | 0.125 | **0.045** | | |
| | 34β36k | 0.092 | **0.030** | 0.123 | **0.043** | | |
| | 36β38k | 0.089 | **0.031** | 0.120 | **0.042** | | |
| | 38β40k | 0.086 | **0.030** | 0.118 | **0.040** | | |
| Summary: | |
| - At matched steps, the expert's loss is about **0.4Γ** the flow head's for both tasks (28β30k: 0.038 vs 0.105 for | |
| insert_tube, 0.049 vs 0.130 for wipe_whiteboard). | |
| - The expert passed the flow head's **final** (38β40k) loss early: by 16β18k for insert_tube (0.076 vs 0.086) and by 10β12k for wipe_whiteboard (0.114 vs 0.118). | |
| - Final (38β40k): insert_tube 0.030 vs 0.086 (0.35Γ), wipe_whiteboard 0.040 vs 0.118 (0.34Γ). insert_tube flattens | |
| after ~32k; wipe_whiteboard is still falling slowly. | |
| Caveats: | |
| 1. **Training loss only.** There is no held-out eval or robot test yet. The expert has ~21Γ more trainable | |
| generator parameters on the same small dataset, so part of the gap could be memorization. | |
| 2. **Different flow conventions.** Both losses are MSE of the velocity (noise β data, up to sign) in the same normalized | |
| control-point space, but the time sampling differs: Beta(1.5,1) puts more weight near the noise end, which is the harder | |
| part. If anything, this makes the expert's loss look higher, not lower. | |
| 3. The loss includes the knot KL term (same coefficient and labels in both variants). | |
| ## Results: timing stage | |
| Timing loss = MSE of log(query interval) against the same labels; mean over steps 9kβ10k. | |
| | Timing head trained from | insert_tube flow_head | insert_tube expert | wipe flow_head | wipe expert | | |
| |---|---|---|---|---| | |
| | Action 20k | 0.210 | 0.262 | 0.203 | 0.280 | | |
| | Action 30k | 0.207 | 0.255 | 0.197 | 0.273 | | |
| | Action 40k | **0.206** | **0.255** | **0.196** | **0.271** | | |
| **The expert's timing head is worse** (+24 % on insert_tube, +38 % on wipe_whiteboard), even though its action loss is | |
| much lower. The starting checkpoint barely matters for either variant (β€ 0.01). | |
| Likely cause (from the code; not tested): the timing head reads the pooled condition `cond` (from `fusion_proj`, | |
| `text_proj`, `cross_gate`, `final_proj`). In `flow_head`, `cond` also feeds the flow transformer, so the flow loss | |
| trains it. In `pi05_expert`, the expert reads the KV cache instead, so `cond` is trained only by the knot KL term and | |
| carries less information. Possible fixes if timing matters on the robot: feed the timing head expert features | |
| (e.g. pooled suffix outputs), or train `cond` with the timing stage unfrozen. | |
| ## Not measured yet | |
| - **Inference latency.** The expert runs a 300M transformer for 10 ODE steps, against 20M for 15 steps, so each query | |
| costs more. This matters because the timing head can ask for a new query every 0.2 s. | |
| - Real-robot success and precision against pi0.5 and against `flow_head`. | |
| - Open-loop length confound: AdaSpeVLA executes up to 1.5 s per query, while the baseline replans on its own schedule. | |