Update NVFP4 notes: flashinfer AOT-dir + HF bucket, speed/memory table

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

NVFP4_ZEROGPU.md

@@ -41,19 +41,40 @@ and the kernels come from **flashinfer**, which ships a dedicated **sm_120
 4. **`FLASHINFER_CUDA_ARCH_LIST=12.0f`** — pins the arch so flashinfer doesn't
    need nvcc just to *detect* capability (a stray system CUDA 12.3 otherwise
    mis-detects and forces a rebuild).
-5. **Prebake + bake the cache.** Build the FP4 kernels once, then ship
-   `~/.cache/flashinfer/0.6.12/...` (the `.so`). Runtime loads it — no compile.
-   `app.py::_setup_nvfp4_env()` restores it from `flashinfer_cache/` at startup.
+5. **Use flashinfer's AOT path (relocatable), not the JIT cache.** The JIT cache
+   dir (`~/.cache/flashinfer/...`) is **not** relocatable — copy/tar/download it
+   and ninja recompiles (~80 s) regardless of mtimes. But flashinfer's
+   `build_and_load()` short-circuits: `if self.is_aot: return load(aot_path)` —
+   if `FLASHINFER_AOT_DIR/<op>/<op>.so` exists it **loads directly, no ninja, no
+   nvcc**. So we ship just the two `.so`
+   (`fp4_quantization_120f`, `fp4_gemm_cutlass_sm120`) in an
+   `aot/<op>/<op>.so` layout in an **HF bucket**
+   (`multimodalart/dreamverse-flashinfer-cache`), and `_setup_nvfp4_env()`
+   downloads them and sets `flashinfer.jit.env.FLASHINFER_AOT_DIR`. **Verified:
+   0.2 s load, no JIT** (even with the JIT cache wiped).
 
-At runtime there is **no compilation** — only a `.so` load and an `nvcc --version`
-read for the cache key. That's the ZeroGPU property: the GPU window is never
-spent compiling.
+At runtime there is **no compilation and no nvcc** — just a `.so` load. That's
+the ZeroGPU property: the GPU window is never spent compiling, and a fork loads
+prebaked weights + prebaked FP4 kernels and runs immediately.
+
+## Speed / memory (measured, warm `generate_video`, 512x768x65, 5 steps)
+
+| Mode | per-segment | vs bf16 eager |
+|---|---|---|
+| bf16 eager | ~8.5 s | 1.0x |
+| bf16 + AOTI | ~3.8 s | ~2.2x |
+| NVFP4 (eager + prebaked) | ~3.7 s | ~2.3x |
+
+NVFP4 ≈ AOTI on speed here, and they **don't stack** (AOTInductor can't C++-
+compile the FP4 op). Peak memory didn't drop (~72 vs ~69 GB) because the
+unquantized Gemma text encoder (~25 GB) dominates. NVFP4's win on this box is
+ZeroGPU-cleanliness; bigger gains need B200 (FA4 + NVFP4 together).
 
 ## Files
 
-- `flashinfer_cache/0.6.12/...` — prebaked sm_120 FP4 kernels
-  (`fp4_quantization_120f.so`, `fp4_gemm_cutlass_sm120.so`), committed.
-- `app.py::_setup_nvfp4_env()` — env + cache restore (NVFP4 mode only).
+- HF bucket `multimodalart/dreamverse-flashinfer-cache` (dataset),
+  `aot/<op>/<op>.so` — prebaked sm_120 FP4 kernels (relocatable, no ninja).
+- `app.py::_setup_nvfp4_env()` — pulls the AOT `.so` + sets `FLASHINFER_AOT_DIR`.
 - `aoti_dit.py` — skips the AOTI hook when `DREAMVERSE_NVFP4=1`.
 
 ## Run it