--- license: mit pipeline_tag: text-to-image base_model: microsoft/Mage-Flow-Turbo base_model_relation: quantized tags: - xpo3 - comfyui - mage-flow - mage-flow-turbo - nvfp4 - fp8 - blackwell - qwen3-vl - text-to-image - quantization ---

XPO3 NVFP4 speed mascot

# Mage-Flow Turbo NVFP4 — V2 > **Follow Mage-Flow NVFP4 development, benchmarks, and release updates at > [Arands.com](https://arands.com/).** **Speed and quality through cross-precision optimization.** ### AJH Mage-Flow releases | Release | Use it when | |---|---| | **Mage-Flow Turbo NVFP4 — XPO3 V2** (this release) | Recommended: the fastest validated Turbo runtime and the best current Turbo choice. | | [Mage-Flow Turbo NVFP4 Balanced](https://huggingface.co/ajh-code/Mage-Flow-Turbo-NVFP4-Balanced-AJH) | You specifically need the earlier standard packaged Turbo variant. | | [Mage-Flow NVFP4 Fast](https://huggingface.co/ajh-code/Mage-Flow-NVFP4-AJH) | 20-step Mage-Flow with maximum compression. | | [Mage-Flow NVFP4 Balanced](https://huggingface.co/ajh-code/Mage-Flow-NVFP4-Balanced-AJH) | 20-step Mage-Flow with the balanced precision policy. | | [Mage-Flow NVFP4 Quality](https://huggingface.co/ajh-code/Mage-Flow-NVFP4-Quality-AJH) | 20-step Mage-Flow with more BF16 image-path retention. | Only the Balanced transformer policy was packaged for Turbo before XPO3. Fast and Quality are non-Turbo Mage-Flow releases. This is the V2 runtime release for the existing model files. **XPO3 is the name of the bundled runtime profile, not a new model format or loader.** With the current **ComfyUI-XPO3 v0.3.0** custom node, the weights run through the promoted XPO3 runtime stack: a four-step, ComfyUI-oriented derivative of [`microsoft/Mage-Flow-Turbo`](https://huggingface.co/microsoft/Mage-Flow-Turbo) optimized for native Blackwell NVFP4 execution. The additional speed is a runtime update and does not require rebuilding or redownloading the model weights. **Recommended AJH Mage-Flow Turbo release.** In the reviewed general-prompt set, XPO3 follows the original BF16 Turbo closely while delivering the fastest validated runtime in this project. Individual trajectories can occasionally improve a detail as well: the published golden-retriever comparison avoids the duplicate-tail artifact present in its BF16 counterpart. That is an observed example, not a claim that XPO3 is universally better than BF16. Microsoft reports **0.59 s/image at 1024×1024 on one A100** for the original Mage-Flow-Turbo, at roughly **18–20 GB** peak memory. This V2 runtime measured **0.5943 s** median at 1024×1024 on an RTX 5080 with **11.85 GiB** peak allocated. These are comparable latency points, not an apples-to-apples cross-hardware benchmark. See the [upstream model card](https://huggingface.co/microsoft/Mage-Flow-Turbo). > **Three precisions. One fused runtime.** > > NVFP4 supplies transformer speed, scaled FP8 compresses the Qwen3-VL text > encoder, and BF16 is retained only where matched testing found that four-bit > execution caused visible quality damage. ## Native W4A4 runtime—not a weight-only dump XPO3 runs its quantized transformer projections through the native Blackwell **W4A4** path: weights are stored as block-16 NVFP4, BF16 activations are dynamically quantized to NVFP4 with role-specific scaling, and the matrix multiplication executes as native NVFP4. It does not expand the packed weights to BF16 and fall back to a BF16 GEMM. That fast path is one part of the promoted measured runtime stack: - 44 MLP projections and 24 offline-fused QKV groups use native W4A4; - four quality-sensitive late image MLP projections remain BF16; - 23 eligible up projections use fused bias-plus-GELU NVFP4 execution; - calibrated FP4 image-MLP bridge routing is enabled on blocks `0-8`; - official Sparge/Sage2 routing runs on steps `1-2` across all 12 blocks; - the CFG1, batch-one direct-HND processor is used on that routed path; - steps `0` and `3`, disabled toggles, and unsupported envelopes stay on the packaged exact fallback path; - the complete pipeline stays resident, while prompt conditioning and the fixed policy prefix are cached. | Result | Measured value | What it establishes | |---|---:|---| | RTX 5080, 1024×1024 V2 hot generation | **0.5943 s** median; **0.5818 s** best | Current V2 repeat, resident model and reusable prompt/screening cache | | RTX 5080, 1280×1280 promoted-stack median | **0.9570 s** | Validated 3-prompt x 3-seed matrix result | | RTX 5080, 1024×1024 fresh-prompt generation | **2.7974 s** median | Resident model; empty prompt and screening caches | | RTX 5080, 1280×1280 fresh-prompt generation | **3.1302 s** median | Resident model; empty prompt and screening caches | | RTX 5060 Ti, 1024×1024 promoted-stack median | **1.3012 s** | Validated 16 GB entry-level Blackwell result | | RTX 5060 Ti, 1280×1280 promoted-stack median | **2.1308 s** | Validated 16 GB entry-level Blackwell result | | Full promoted stack versus the packaged resident FA2 baseline | **1.177x-1.212x faster** | Current release-runtime gain, not a BF16 claim | | Balanced NVFP4 versus BF16 in the matched pre-fusion study | **1.39× faster** | Transformer precision-policy contribution | | Scaled-FP8 text conditioning versus BF16 | **0.9932–0.9971 cosine** | Frozen prompt-suite fidelity | | Exact-text pooled conditioning versus BF16 | **0.9986 cosine** | Text-encoder pooled-vector fidelity | | Balanced final image versus matched BF16 | **23.78 dB / 0.0283 MAE** | Pixel-level precision-study comparison | The conditioning cosines describe text-encoder outputs, not an overall image quality score. Image quality was also checked with frozen same-seed prompt suites and direct visual review. ## Download This repository intentionally contains the three model files useful to ComfyUI: | File | Place in ComfyUI | Size | |---|---|---:| | `Mage-Flow-Turbo-XPO3-NVFP4.safetensors` | `models/diffusion_models/` | 4.87 GB | | `qwen3vl_4b_fp8_scaled.safetensors` | `models/text_encoders/` | 5.24 GB | | `Mage-Flow-VAE.safetensors` | `models/vae/` | 345 MB | The diffusion model is one file rather than four Hugging Face transformer shards. The complete three-weight download is 10.45 GB. Install [`ComfyUI-XPO3`](https://github.com/AJH-Code/ComfyUI-XPO3), restart ComfyUI, and use: 1. **Load Mage-Flow Turbo XPO3** 2. **Mage-Flow Turbo XPO3 Generate** The custom-node repository includes an API-format starter workflow that defaults to 1280×1280. Recommended settings are four steps, CFG 1, and static shift 6. The Generate node also exposes independent runtime toggles for fused GELU-up, the FP4 image-MLP bridge, bridge-block selection, Sparge/Sage2 attention, direct-HND, and selected attention steps and blocks. ### V1 → V2 No model-weight redownload is required. V2 is a runtime and integration upgrade: fused GELU-up, the calibrated FP4 bridge, the validated accelerated attention/direct-HND route, explicit per-feature fallbacks, a 1280-default ComfyUI workflow, and a reproducible benchmark runner in ComfyUI-XPO3. ## Speed: 0.59 s at 1024, 0.96 s at 1280 on RTX 5080 Measured on 16 GB RTX 5080 and RTX 5060 Ti cards under Linux, Python 3.11, and PyTorch `2.13.0+cu130`, with one resident model load per matrix arm. The table distinguishes repeated interactive generation from the first image of a new prompt: - **Hot**: model remains resident; prompt conditioning and its fixed screening prefix are reusable. - **Fresh prompt**: model remains resident, but the prompt-conditioning and screening caches begin empty. This is not a cold process start. Prompt conditioning is the Qwen3-VL embedding pass used by the diffusion model; it is independent of the random seed and can therefore be reused for re-rolls. The screening pass checks a prompt against a fixed content-policy prefix. Its immutable prefix KV state is also cacheable and does not change the generated image. | GPU | Resolution | Hot generation | Fresh prompt | |---|---|---:|---:| | **RTX 5080** | 1024×1024 | **0.5943 s median** (**0.5818 s** best) | **2.7974 s median** | | **RTX 5080** | 1280×1280 | **0.9570 s median** | **3.1302 s median** | | RTX 5060 Ti | 1024×1024 | **1.3012 s median** | — | | RTX 5060 Ti | 1280×1280 | **2.1308 s median** | — | The 1024 and 1280 hot figures are V2 resident runs. The `0.9570 s` 1280 figure is the more robust three-prompt × three-seed matrix median; the 1024 figure is a current V2 two-seed repeat. In the completed multi-prompt matrix, the promoted stack delivered a consistent **1.177x-1.212x** end-to-end gain over the previous packaged resident FA2 baseline. For process startup on the RTX 5080, model load plus resident placement takes about **9.46-10.14 s** before the first generation. That one-time cost is kept separate from the generation timings above. ### Reproduce the hot/fresh-prompt protocol The matching public runner ships with [`ComfyUI-XPO3`](https://github.com/AJH-Code/ComfyUI-XPO3). From the plugin directory, using the same ComfyUI Python environment and model filenames: ```bash python benchmark.py --width 1024 --height 1024 \ --output xpo3_1024_benchmark.json ``` It runs a fixed three-prompt/three-seed suite at four steps, CFG 1, and static shift 6; records a fresh-cache warm-up and repeated hot generations for each prompt; and writes timing, environment, and active-feature evidence to JSON. Your GPU, driver, and ComfyUI environment will produce different numbers. Peak allocated memory increased modestly versus that packaged baseline: - 1024×1024: **11.85 GiB** peak allocated (about **+51.0 MiB**) - 1280×1280: **12.33 GiB** peak allocated (about **+78.6 MiB**) The complete promoted stack still remained under 16 GB on both validated cards. Timings are hardware- and environment-dependent. ## Promoted runtime stack and fallback rules The current v0.3.0 custom-node runtime includes: - fused GELU-up on eligible NVFP4 up projections; - calibrated FP4 image-MLP bridge on image blocks `0-8`; - official Sparge/Sage2 meansim top-k `1.0`, smooth-K, all 12 blocks on steps `1-2`; - the validated CFG1, batch-one direct-HND processor on that routed path; - exact packaged fallback on steps `0` and `3`; - independent per-feature toggles exposed through the ComfyUI generate node. Fallback behavior is intentional rather than best-effort: - when fused GELU-up is disabled, the wrapper calls the exact original activation module; - when the bridge is disabled, a bridge block is deselected, or an unsupported shape/stream/device condition is seen, that block falls back before native launch; - when attention acceleration is disabled, CFG is not `1.0`, steps are not `4`, the pipe is unsupported, or the Sparge dependency is unavailable, the packaged attention path remains active; - direct-HND is only used on the supported CFG1, batch-one routed envelope. ## Quality XPO3 does not quantize every eligible projection merely to maximize the compression ratio. **This Turbo release ships one selected Balanced policy.** The Fast and Quality rows below are research controls used to choose it; they are not additional downloadable Turbo profiles. ### 1280×1280 BF16 versus XPO3 comparison set Each pair below uses the same prompt, seed, 1280×1280 resolution, four steps, CFG 1, and static shift 6. **Original Mage-Flow-Turbo BF16 is on the left; XPO3 NVFP4 is on the right.** The images are direct decodes with no upscaling or postprocessing. Click any comparison to inspect it at full size. | Alpine lake | Golden retriever | |---|---| | [![BF16 versus XPO3 alpine lake comparison](examples/bf16-vs-xpo3-1280/00-alpine-lake.jpg)](examples/bf16-vs-xpo3-1280/00-alpine-lake.jpg) | [![BF16 versus XPO3 golden retriever comparison](examples/bf16-vs-xpo3-1280/01-golden-retriever.jpg)](examples/bf16-vs-xpo3-1280/01-golden-retriever.jpg) | | Reading nook | Perfume product | | [![BF16 versus XPO3 reading nook comparison](examples/bf16-vs-xpo3-1280/02-reading-nook.jpg)](examples/bf16-vs-xpo3-1280/02-reading-nook.jpg) | [![BF16 versus XPO3 perfume product comparison](examples/bf16-vs-xpo3-1280/03-perfume-product.jpg)](examples/bf16-vs-xpo3-1280/03-perfume-product.jpg) | | Vintage car | Fisherman portrait | | [![BF16 versus XPO3 vintage car comparison](examples/bf16-vs-xpo3-1280/04-vintage-car.jpg)](examples/bf16-vs-xpo3-1280/04-vintage-car.jpg) | [![BF16 versus XPO3 fisherman portrait comparison](examples/bf16-vs-xpo3-1280/05-fisherman-portrait.jpg)](examples/bf16-vs-xpo3-1280/05-fisherman-portrait.jpg) | | Ramen | Butterfly macro | | [![BF16 versus XPO3 ramen comparison](examples/bf16-vs-xpo3-1280/06-ramen.jpg)](examples/bf16-vs-xpo3-1280/06-ramen.jpg) | [![BF16 versus XPO3 butterfly macro comparison](examples/bf16-vs-xpo3-1280/07-butterfly-macro.jpg)](examples/bf16-vs-xpo3-1280/07-butterfly-macro.jpg) | | Street portrait | Astronaut greenhouse | | [![BF16 versus XPO3 street portrait comparison](examples/bf16-vs-xpo3-1280/08-street-portrait.jpg)](examples/bf16-vs-xpo3-1280/08-street-portrait.jpg) | [![BF16 versus XPO3 astronaut greenhouse comparison](examples/bf16-vs-xpo3-1280/09-astronaut-greenhouse.jpg)](examples/bf16-vs-xpo3-1280/09-astronaut-greenhouse.jpg) | | Watercolor fox | Glass house | | [![BF16 versus XPO3 watercolor fox comparison](examples/bf16-vs-xpo3-1280/10-watercolor-fox.jpg)](examples/bf16-vs-xpo3-1280/10-watercolor-fox.jpg) | [![BF16 versus XPO3 glass house comparison](examples/bf16-vs-xpo3-1280/11-glass-house.jpg)](examples/bf16-vs-xpo3-1280/11-glass-house.jpg) | Across this general-prompt set, XPO3 closely follows BF16 subject, framing, lighting, palette, and overall quality. Small trajectory-dependent differences remain, so XPO3 should not be interpreted as pixel-identical or lossless. At 1024×1024 in the precision-policy study: | Transformer profile | Denoise | Peak allocated | PSNR / MAE versus BF16 | |---|---:|---:|---:| | Fast: 48 NVFP4 MLP projections | 3.44 s | 6.09 GiB | 18.27 / 0.0747 | | **Chosen Balanced profile** | **3.51 s** | **6.29 GiB** | **23.78 / 0.0283** | | Quality: image MLPs in BF16 | 4.20 s | 7.31 GiB | 23.38 / 0.0312 | | BF16 transformer | 4.89 s | 8.52 GiB | reference | That study isolated transformer precision before the later resident, single-branch, QKV-fusion, cache, and fused-GELU speed work, so its latency numbers should not be mixed with the final warm-runtime table above. ### Representative 1280×1280 XPO3 output ![Cyborg woman looking toward a star-filled sky, generated by XPO3](examples/cyborg_xpo3.png) This is a direct, unprocessed four-step XPO3 decode: 1280×1280, CFG 1, static shift 6, and seed `3334910515`. > A 4K resolution high detail photo realistic image of the top half of a > cyborg woman with dark black hair, striking blue eyes that have a very subtle > glow in the iris, standing side profile, head tilted up towards the sky with > a questioning expression, she has subtle gaps in her skin that hint at a > robotic nature, outdoor forest night setting, sky filled with bright > brilliant stars that glow against the dark setting, nebula visible This intentionally difficult prompt combines photorealistic anatomy, exposed machinery, damaged skin, glowing eyes, and complex night lighting. It is shown as one representative current result, not as evidence that every prompt is pixel-identical to BF16. ## What XPO3 changed ### 1. Architecture-aware precision selection Forty-four Mage transformer MLP projections run in native NVFP4. Four late image-stream projections remain BF16: - `transformer_blocks.9.img_mlp.net.2` - `transformer_blocks.10.img_mlp.net.2` - `transformer_blocks.11.img_mlp.net.0.proj` - `transformer_blocks.11.img_mlp.net.2` Keeping these four projections in BF16 substantially improved the 1024 portrait result while adding far less size and latency than returning the complete image stream to BF16. ### 2. Quality-selected activation scaling BF16 activations are dynamically quantized to block-16 NVFP4 immediately before native W4A4 execution. Scaling is selected by projection role: the NVFP4 MLP down paths use a `0.75` activation-scale multiplier, while the other native projections retain their validated scaling policy. Alternative block-scale searches, weight `4-over-6`, and bias-correction candidates were tested and rejected because lower local reconstruction error did not consistently produce a better final diffusion trajectory. ### 3. Offline-fused NVFP4 attention Each image/text Q, K, and V triplet is concatenated into one native NVFP4 projection. All 24 fused QKV groups are packed in the downloadable checkpoint: - 72 original BF16 Q/K/V projections replaced; - approximately 0.91 GiB lower peak allocation in the experimental matched attention test; - no 6.29-second load-time attention packing step; - no full-weight BF16 expansion or BF16 GEMM fallback for these projections. ### 4. Fused MLP execution Twenty-three NVFP4 MLP up projections use a native cuBLASLt up-projection-plus-bias-plus-GELU path. The final quality-sensitive BF16 image up projection remains unfused, and the fused path remains independently toggleable from the ComfyUI node. ### 5. Calibrated image-MLP bridge The promoted runtime adds the calibrated FP4 image-MLP bridge on image blocks `0-8`, using the pinned per-block scales selected during the bounded bridge validation. The bridge is independently toggleable and retains the packaged module as its exact off path. ### 6. Step-gated Sparge/Sage2 attention The promoted runtime adds official Sparge/Sage2 routing on steps `1-2` across all 12 blocks, with the validated CFG1, batch-one direct-HND processor on that route. Steps `0` and `3` remain on the packaged exact path, and the attention route is independently toggleable. ### 7. Resident and cached generation The transformer, FP8 text encoder, and VAE fit together on the validated 16 GB GPU. CFG 1 uses a single conditional branch. Exact prompt conditioning is cached independently of the random seed, and the fixed policy-screen prefix uses a bounded KV cache. ### 8. Single-file deployment The XPO3 diffusion checkpoint embeds the transformer structure, precision policy, fused-attention mapping, native-runtime requirement, and base-model relationship in safetensors metadata. It contains 437 tensors and is approximately 977 MB smaller than the earlier four-shard Balanced transformer. ## What XPO3 is—and is not XPO3 is a checkpoint and runtime approach focused on speed and quality through architecture-aware cross-precision deployment and fused native execution. It uses NVIDIA's standard NVFP4 E2M1 block-16 representation; it does not claim to invent a new floating-point datatype. It is also not SVDQuant or a Nunchaku checkpoint. XPO3 does not use a high-precision low-rank outlier branch. Its quality recovery comes from measured precision placement and scale policy, while its speed comes from native NVFP4 execution, projection fusion, epilogue fusion, single-branch Turbo inference, and resident caching. ## Requirements and current limitations - Linux x86-64 - NVIDIA Blackwell SM120 - under 16 GB on the validated promoted stack; 16 GB class validated - Python 3.11 - PyTorch `2.13.0+cu130` - TorchVision `0.28.0+cu130` - TorchAudio `2.11.0+cu130` - CUDA 13.x runtime; bundled binaries built with CUDA 13.1 - `flash-attn==2.8.3` The first prebuilt runtime does not support Windows, WSL, pre-Blackwell GPUs, Python 3.12/3.13, or other PyTorch/CUDA ABIs. The custom-node plugin also bundles SpargeAttn runtime code and extensions under `runtime/spas_sage_attn/` under Apache-2.0 terms. Its included license text is at `third_party_licenses/SpargeAttn-LICENSE` in the plugin repository. The files use ComfyUI's normal diffusion-model, text-encoder, and VAE directories. Mage-Flow is not currently supported by ComfyUI's generic diffusion loader, so this release requires the XPO3 custom nodes and does not yet connect directly to stock KSampler, LoRA, ControlNet, or native Comfy conditioning nodes. ### Image-quality limitations Four-step Turbo is intentionally aggressive. Complex photorealistic prompts that combine organic anatomy with exposed mechanics—such as the cyborg stress prompt in this card—can produce seed-sensitive neck, shoulder, torso, or small-detail defects. A matched ten-seed comparison found the same failure class in the original all-BF16 Mage-Flow-Turbo model; it is not an XPO3-only defect. XPO3 can change the diffusion trajectory, so it is not pixel-identical or lossless. The published 1280 BF16/XPO3 gallery is intended to make that trade off inspectable rather than hidden. ## Sources and licenses - Mage-Flow-Turbo, the VAE, and the Mage inference source derive from [`microsoft/Mage-Flow-Turbo`](https://huggingface.co/microsoft/Mage-Flow-Turbo) under the upstream MIT terms. - The scaled-FP8 Qwen3-VL checkpoint comes from [`starsfriday/Mage-Flow-Edit-FP8`](https://huggingface.co/starsfriday/Mage-Flow-Edit-FP8) and derives from Apache-2.0 Qwen3-VL weights. - Bundled SpargeAttn runtime code and extensions under `runtime/spas_sage_attn/` are Apache-2.0; see [`thu-ml/SpargeAttn`](https://github.com/thu-ml/SpargeAttn) and the plugin's `third_party_licenses/SpargeAttn-LICENSE`. - XPO3 custom-node and packaging code is MIT licensed. See the [`ComfyUI-XPO3` third-party notices](https://github.com/AJH-Code/ComfyUI-XPO3/blob/main/THIRD_PARTY_NOTICES.md) and included license texts in the custom-node repository.