M4 — Bootstrapped Counterfactual Velocity Steering (RoboWarp)
Self-bootstrapped variant of M3 (adipotnis/m3-train-time-cfsteer-robowarp): π₀.₅ base
(not the released LIBERO checkpoint) finetuned on LIBERO-Spatial with the flow-matching
target steered away from a counterfactual velocity — but the counterfactual teacher is a
DQN-style target network: a frozen bfloat16 snapshot of the student's own weights,
hard-refreshed from the student every N steps. No pretrained LIBERO teacher is used
anywhere. Trained with openpi (JAX)
on 2× NVIDIA GH200.
Contents: params/ (Orbax weights) + assets/ (spatial-suite quantile normalization
stats — required at inference). No train_state/, so this cannot be resumed.
Method (M4)
Per training sample (openpi convention, t=1 noise):
eps ~ N(0, I), t ~ Beta(1.5, 1)·0.999 + 0.001
x_t = t·eps + (1−t)·a u = eps − a (standard CFM pair)
v_neg = v_target-net(x_t, t | o_cf) LAGGED SNAPSHOT of the student,
on the attention-ERASED base image
s* = ⟨u, v_neg⟩ / ‖v_neg‖² CFG-Zero* scale
v_tgt = u + ω·(u − s*·v_neg) w.p. p, else u
loss = ‖v_θ(x_t, t | o_factual) − v_tgt‖²
every N steps: target-net ← bf16(student) DQN-style hard update
At step 0 the counterfactual is π₀.₅-base's own (LIBERO-naive) velocity on the erased view;
as the student learns the spatial tasks, the lagged teacher tracks it, so v_neg converges
to "what the current policy would do without the attended evidence" — a self-distilled
counterfactual. The teacher is frozen between refreshes, forward-only, under stop-gradient.
Inference is unchanged: single pass from N(0, I), no masking, no second forward.
Approximations on record: the attention-erasure masks are static (extracted once from
the released pi05_libero via the RoboWarp masking pipeline — layer-12 attention,
register-mask + spike cleaning, top-15 % pixels, fill 128; they encode scene/task saliency,
not teacher identity) and are not refreshed with the teacher.
Hyperparameters
| Init | gs://openpi-assets/checkpoints/pi05_base (NOT the released LIBERO checkpoint) |
| Teacher | bf16 snapshot of the student, hard-refreshed every N = 1000 steps (2 syncs: steps 1000 and 2000) |
| Model | π₀.₅ (pi05=True, action_horizon 10, discrete_state_input=False — state-blind) |
| Data | physical-intelligence/libero v2.0, LIBERO-Spatial suite only: 432/1693 episodes, 52,970 frames (task indices 30–39) |
| Normalization | spatial-suite quantile stats computed from the training data (pi05_base ships no LIBERO stats — differs from M2/M3, which use the released checkpoint's stats) |
| Steering | ω = 0.5, p = 0.5, CFG-Zero* scale on |
| Batch / steps | 112 (2× GH200, FSDP; resident teacher) / 2,300 (≈4.9 epochs) |
| LR | 3.3e-5 → 3.3e-6 cosine, warmup 200 (sqrt-rescaled from 256@5e-5) |
| Optimizer | AdamW β=(0.9, 0.95), eps 1e-8, wd 1e-10, grad-clip 1.0, EMA 0.999 |
| Masking params | layer 12, erase_pct 0.85, fill 128, ratio 2.0, iters 15, σ 0.8 |
| Precision | bfloat16 (teacher snapshot also bfloat16) |
Caveats
- Training loss is not comparable to any other arm (different init, different norm stats, steered target). Evaluate by simulator success rate (LIBERO-PRO spatial/swap) only.
- Not evaluated in sim at upload time. ~4.9 epochs from BASE (vs ~48 in the reference recipe) — expect weaker absolute performance than the pi05_libero-init arms.
- N = 1000 and ω = 0.5 are initial estimates, not swept.
- Comparison set:
adipotnis/pi05-libero-spatial-fft-control(plain FFT of pi05_libero),adipotnis/m2-train-time-cf-robowarp(source tilt),adipotnis/m3-train-time-cfsteer-robowarp(fixed-teacher steering),adipotnis/pi05-libero-spatial-run1(plain FFT of pi05_base — the matching-init control for THIS run, though it is state-tokenized).