RESULTS.md

# Final table — 4 systems on the same val episodes (GT-action replay)

| system | psnr | ssim | lpips | fid | fvd_s3d | psnr_arm |
|---|---|---|---|---|---|---|
| base_10k (60eps) | 19.345 | 0.706 | 0.262 | 35.478 | 32.973 | 13.883 |
| vanilla_40k | — | — | — | — | — | — |
| stageB_geo_40k (60eps) | 19.133 | 0.705 | 0.265 | 42.878 | 41.132 | 13.657 |
| stageC_aux_40k (60eps) | 19.010 | 0.703 | 0.266 | 40.189 | 36.620 | 13.563 |

↑ psnr/ssim/psnr_arm better · ↓ lpips/fid/fvd better


## Verdict (2026-07-04)
Geometry grounding (tokens = Stage B; tokens+aux = Stage C) does NOT improve the
world model on in-distribution GT-action replay — both are ~0.2-0.3 dB worse than
base-10k on every metric. Mechanisms verified active (adapter fk_to_out‖W‖~0.9,
aux q-decode 0.17 rad). FID/FVD drop is consistent with finetune-on-subset drift
(aux partially recovers it), not geometry per se — a vanilla control would confirm.
arm-PSNR flat because GT actions already determine the arm; the zero-shot-camera
thesis (where geometry should help) was NOT tested here.

---

## Follow-up: WHY does geometry not help? (redundancy vs conflict)

Three experiments isolate the cause:

| system | psnr | fid | fvd | psnr_arm | geometry Δpix (toggle) |
|---|---|---|---|---|---|
| base_10k | 19.35 | 35.5 | 33.0 | 13.88 | — |
| geo AUGMENT (Stage B) | 19.13 | 42.9 | 41.1 | 13.66 | **0.0075** (inert) |
| geo REPLACE (drop action) | 15.26 | 61.6 | 61.3 | 11.03 | **0.115** (controls) |

1. **AUGMENT** (geometry + action): toggling geometry on/off changes <1% of pixels
   (Δpix 0.0075) and arm-PSNR by +0.03 dB — geometry is **inert**.
2. **REPLACE** (geometry is the sole conditioner, action dropped): a counterfactual
   swap (episode A's video context + episode B's geometry) changes **11.5%** of
   pixels — **15× the augment toggle**. Geometry is **causally load-bearing**: the
   generated arm follows the (wrong) geometry.
3. Therefore the augment failure is **pure redundancy with the action token**, not
   an inability to read geometry: on GT-action replay the 7-D EE action already
   determines the arm, so gradient descent never uses the (redundant) geometry.
   Geometry **can** be read, but it is a **weaker standalone conditioner** than the
   dense EE action (REPLACE reconstructs worse: psnr 15.3 vs 19.4).

**Bottom line:** not destructive conflict — redundancy + a weaker signal. The
model uses geometry when forced (REPLACE), ignores it when action makes it
redundant (AUGMENT). Geometry's real edge — viewpoint invariance / zero-shot new
camera — is untested here (DROID has no extrinsics) and is where it would beat the
robot-frame EE action.

## Convergence / undertraining check (REPLACE)

Is REPLACE worse because it is undertrained? Eval of replace checkpoints (n=20):

| step | psnr | psnr_arm |
|---|---|---|
| 8k | 17.90 | 11.48 |
| 24k | 17.67 | 11.62 |
| 40k | 17.19 | 11.50 |

arm-PSNR is **flat** 8k→40k (already converged by ~8-24k); whole-frame psnr even
declines slightly (mild overfit). So **more steps at this finetune scale do not
close the gap to base**. BUT the deeper confound stands: geometry conditioning was
only ever *finetuned* for ~8 GPU-h at unet_lr 5e-6, whereas the action conditioning
is baked into the base **pretrain (~8×H100 × 3 days ≈ 570 GPU-h)**. A modest
finetune cannot re-teach the cross-attn to read geometry as well as 3 days of
pretraining taught it to read action. Fairly answering "can geometry match action"
requires pretraining a model with geometry as the *native* conditioner at
comparable scale — out of scope here.

**Robust regardless of compute:** (1) geometry is readable/controllable
(counterfactual Δpix 0.115), (2) the augment failure is redundancy with action.
Only the claim "geometry is inherently weaker than action" is unresolved (never
trained at a fair budget).