M5 — BYOL-Style Counterfactual Velocity Steering (RoboWarp)
BYOL-flavored sibling of M4 (adipotnis/m4-bootstrap-cfsteer-robowarp): π₀.₅ base
finetuned on LIBERO-Spatial with the flow-matching target steered away from a counterfactual
velocity, where the counterfactual teacher is the run's own EMA of the student — a
BYOL-style target network, soft-updated every step with τ = 0.999, instead of M4's periodic
hard (DQN-style) snapshot. No pretrained LIBERO teacher is used anywhere. Trained with
openpi (JAX) on 2× NVIDIA GH200.
Contents: params/ (Orbax weights) + assets/ (spatial-suite quantile normalization
stats — required at inference). No train_state/, so this cannot be resumed.
Method (M5)
Per training sample (openpi convention, t=1 noise):
eps ~ N(0, I), t ~ Beta(1.5, 1)·0.999 + 0.001
x_t = t·eps + (1−t)·a u = eps − a (standard CFM pair)
v_neg = v_ξ(x_t, t | o_cf) EMA target network ξ, on the emb-zero
counterfactual view (patch tokens zeroed)
s* = ⟨u, v_neg⟩ / ‖v_neg‖² CFG-Zero* scale
v_tgt = u + ω·(u − s*·v_neg) w.p. p, else u
loss = ‖v_θ(x_t, t | o_factual) − v_tgt‖²
every step: ξ ← τ·ξ + (1−τ)·θ, τ = 0.999 BYOL-style soft target update
BYOL correspondence: online network = student on the factual view; target network = EMA of the student on the counterfactual view; with the sign flipped — where BYOL attracts the online branch to the target's output across augmented views (invariance), M5 repels the student's velocity from the target's velocity on the evidence-erased view (evidence sensitivity). τ = 0.999 gives an effective teacher lag of ~1000 steps, smooth rather than M4's staircase. The target network is forward-only, under stop-gradient, and costs no extra weight memory (the EMA already lives in the train state; it is also what openpi serves). Inference is unchanged: single pass from N(0, I), no masking, no second forward.
Approximations on record: attention-erasure masks are static (extracted once from the
released pi05_libero via the RoboWarp masking pipeline — layer-12 attention, register-mask
- spike cleaning, top-15 % pixels, fill 128) and are not tied to the teacher.
Hyperparameters
| Init | gs://openpi-assets/checkpoints/pi05_base (NOT the released LIBERO checkpoint) |
| Teacher | EMA of the student, τ = 0.999, soft-updated every step (BYOL-style) |
| Model | π₀.₅ (pi05=True, action_horizon 10, discrete_state_input=False — state-blind) |
| Data | physical-intelligence/libero v2.0, LIBERO-Spatial suite only: 432/1693 episodes, 52,970 frames (task indices 30–39) |
| Normalization | spatial-suite quantile stats computed from the training data (same file as M4/baseline) |
| Steering | ω = 0.5, p = 0.5, CFG-Zero* scale on |
| Batch / steps | 112 (2× GH200, FSDP) / 2,300 (≈4.9 epochs) |
| LR | 3.3e-5 → 3.3e-6 cosine, warmup 200 (matched to M4/baseline) |
| Optimizer | AdamW β=(0.9, 0.95), eps 1e-8, wd 1e-10, grad-clip 1.0, EMA 0.999 (doubles as the teacher) |
| Counterfactual | emb-zero: 16×16 patch grid, patch zeroed at ≥ 50 % attended pixels (masks: robowarp layer-12, top-15 %, from released pi05_libero) |
| Precision | bfloat16 compute, fp32 params/EMA |
Caveats
- Training loss is not comparable to any other arm. Evaluate by simulator success rate (LIBERO-PRO spatial/swap) only.
- Not evaluated in sim at upload time. ~4.9 epochs from BASE — expect weaker absolute performance than the pi05_libero-init arms.
- τ = 0.999 (inherited from the EMA), ω = 0.5 not swept.
- Comparison set:
adipotnis/pi05-base-spatial-fft-baseline(matched vanilla twin),adipotnis/m4-bootstrap-cfsteer-robowarp(hard-snapshot sibling),adipotnis/m3-train-time-cfsteer-robowarp,adipotnis/m2-train-time-cf-robowarp,adipotnis/pi05-libero-spatial-fft-control.
v2 (emb-zero) redo. The original pixel-erase run of M5 lives in the earlier repo; this repo holds the emb-zero variant, all other knobs identical.