PERFORMANCE.md

# Native W4A4 performance correction

Status: validated and released with [OrbitQuant 0.9.1](https://github.com/iamwavecut/OrbitQuant/releases/tag/v0.9.1) at immutable source revision [`73bb2be9add3c86ad60430ec8e2d0a28d7126fc7`](https://github.com/iamwavecut/OrbitQuant/commit/73bb2be9add3c86ad60430ec8e2d0a28d7126fc7).

## Root cause

The prequantized Diffusers loader preserved FP32 when no compute dtype was supplied. H3's non-quantized modules therefore produced FP32 activations, and all 300 OrbitQuant W4 linears rejected the native BF16/FP16 packed kernel. The runtime silently used the generic `triton_cuda` FP32 path instead of `triton_cuda_packed_w4`.

OrbitQuant 0.9.1 changes only the default for prequantized Diffusers artifacts: an omitted dtype becomes BF16, while an explicit dtype remains authoritative. The H3 runner also passes BF16 explicitly for `transformer`, `transformer_ref`, and `text_encoder`. Source `vae` and `audio_vae` loads remain unmodified FP32.

No CUDA/Triton kernel source change was required. The existing packed W4 kernel reached the expected path and timing once activations were BF16. The package also stops independently pinning Triton, leaving PyTorch to provide its matching Triton build.

## Measurements

All GPU measurements used a RunPod NVIDIA RTX PRO 6000 Blackwell Server Edition with PyTorch 2.8.0+cu128 and CUDA 12.8.

| check | result |
|---|---:|
| exact full 50-block forward, old FP32 default | 114.1176 s |
| exact full 50-block forward, explicit BF16 | 2.8687 s |
| exact full 50-block forward, fixed package default | 2.8668 s |
| dispatch speedup | 39.8x |
| fixed packed dispatch | 300 / 300 linears |
| `transformer_ref` first-process forward with Triton compilation | 5.5383 s |
| `transformer_ref` hot compiled-cache forward | 2.8904 s |
| `transformer_ref` packed dispatch | 300 / 300 linears |
| complete 608x480, 124-frame, 49-forward generation-to-latents stage | 114.6722 s |
| complete generation peak VRAM | 46,359,644,160 bytes (43.18 GiB) |
| source-FP32 VAE decode | 4.7095 s |
| decode peak VRAM | 12,139,359,744 bytes (11.31 GiB) |
| previous W4A16 compatibility generation-to-latents stage | 2145.7408 s |
| generation-to-latents speedup vs W4A16 | 18.7x |
| measured source-BF16 control | 451.5754 s |
| generation-to-latents speedup vs source BF16 | 3.94x |

The early step-1 native-vs-W4A16 comparison measured 0.0416% video NRMSE and 0.0734% audio NRMSE. The final quality gate passed all 49 model evaluations and 19 reviewed frames. Full-timeline and dense-late inspection found no face breakup, crossed eyes, lip disconnection, ghosting, tile grid, or texture collapse. The final H.265 file is 608x480, 124 readable frames, `hvc1`, with AAC 32 kHz stereo.

## Verification

- OrbitQuant: `uv run ruff check .` passed.
- OrbitQuant: `uv run pytest -ra` passed with 570 passed and 85 skipped.
- Linux, Windows, and CPU package CI passed on the integrated release revision.
- The 0.9.1 wheel was clean-installed from PyPI and reported the expected version.
- The H3 component-load policy passed on RunPod, including BF16 transformer/text compute and untouched VAE precision.
- The early native-W4 dtype guard, component-load policy, and media packaging checks passed 14/14 tests on the second RunPod environment.
- Final HEVC and H.264 fallback independently probe as 608x480, 124 frames, AAC 32 kHz stereo.

The canonical workflow remains two-stage: atomically persist each denoise step and final latents, terminate generation, then decode in a fresh process with source-FP32 VAEs. A direct in-process FP16-latent/FP32-VAE decode is not the validated path.