M9 — Simple Bootstrap Counterfactual Steering (RoboWarp)
The minimal member of the steering family: π₀.₅ base finetuned on LIBERO-Spatial with a flow-matching target that is a plain signed, scaled add/subtract of the bootstrap counterfactual velocity — no CFG-Zero★ projection, no guidance-delta construction. Teacher is an M4-style hard snapshot of the student (DQN-style, refreshed every N steps); counterfactual is emb-zero (top-attended SigLIP patch tokens zeroed). Trained with openpi (JAX) on 2× NVIDIA GH200.
Contents: params/ + assets/ (spatial-suite quantile norm stats). No train_state/.
Method (M9)
eps ~ N(0, I), t ~ Beta(1.5, 1)·0.999 + 0.001
x_t = t·eps + (1−t)·a u = eps − a
v_neg = v_θ̄(x_t, t | o, emb-zero mask) θ̄ = hard snapshot of student, stop-grad
v_tgt = u + λ·v_neg w.p. p, else u λ signed: λ<0 subtracts (steer away)
L = ‖v_θ(x_t, t | o) − v_tgt‖²
every N steps: θ̄ ← bf16(student)
Known trade-off (why the fancier variants exist): v_neg shares the noise-dominated direction with u (cos ≈ 0.95+), so large |λ| mostly rescales that shared component instead of steering along the evidence dimension — s★ projected it out; M9 instead keeps |λ| small. Inference unchanged: single pass from N(0, I).
Hyperparameters
| Init | gs://openpi-assets/checkpoints/pi05_base |
| Teacher | bf16 snapshot of the student, hard-refreshed every N = 1,000 steps |
| Steering | λ = −0.1, p = 0.5 (no s★, no delta) |
| Counterfactual | emb-zero: 16×16 patch grid, ≥ 50 % attended pixels (robowarp layer-12 masks) |
| Data | LIBERO-Spatial only: 432/1693 episodes, 52,970 frames |
| Batch / steps | 112 / 5,000 (≈10.7 epochs — family standard) |
| LR / optimizer | 3.3e-5 → 3.3e-6 cosine, warmup 200 · AdamW, clip 1.0, EMA 0.999 |
Caveats
- Training loss not comparable across arms; evaluate by sim success (LIBERO-PRO spatial/swap).
- λ, N, p not swept. Family:
pi05-base-spatial-fft-baseline(vanilla twin), M4/M5 (s★ steering), M6 (contrastive), M7 (guidance delta).