README.md

---
license: other
library_name: diffusers
pipeline_tag: text-to-image
base_model: microsoft/Mage-Flow-Turbo
base_model_relation: quantized
tags:
  - ajh
  - mage-flow
  - mage-flow-turbo
  - mage-flow-turbo-nvfp4-balanced-ajh
  - nvfp4
  - fp8
  - blackwell
  - qwen3-vl
  - text-to-image
  - quantization
  - balanced
---

# Mage-Flow-Turbo-NVFP4-Balanced-AJH

A standalone, four-step quantized derivative of
[`microsoft/Mage-Flow-Turbo`](https://huggingface.co/microsoft/Mage-Flow-Turbo).
It packages the complete transformer, scaled-FP8 text encoder, VAE,
scheduler, pinned Mage inference source, and native SM120 runtime.

## Policy

- Transformer: 44 native NVFP4 MLP projections + 4 BF16 projections.
- Text encoder: 252 scaled FP8 E4M3 projections.
- NVFP4 activation search: `amax`.
- Up activation multiplier: `1.0`.
- Down activation multiplier: `0.75`.
- Turbo defaults: 4 steps, CFG 1, static shift 6.

BF16 transformer projections:

- `transformer_blocks.9.img_mlp.net.2`
- `transformer_blocks.10.img_mlp.net.2`
- `transformer_blocks.11.img_mlp.net.0.proj`
- `transformer_blocks.11.img_mlp.net.2`

## Showcase

![Cyborg woman gazing toward a star-filled sky](examples/cyborg_stargaze_turbo_balanced.png)

Generated directly at 1024×1024 with this package, four steps, CFG 1,
static shift 6, and seed `3334072683`, without upscaling or
post-processing.

> A 4K resolution high detail photo realistic image of the top half of
> a cyborg woman with dark black hair, striking blue eyes that have a
> very subtle glow in the iris, standing side profile, head tilted up
> towards the sky with a questioning expression, she has subtle gaps
> in her skin that hint at a robotic nature, outdoor forest night
> setting, sky filled with bright brilliant stars that glow against
> the dark setting, nebula visible

## Matched BF16 comparison

Both images use the same Turbo weights, scaled-FP8 text encoder, prompt,
seed `3334072683`, 1024×1024 resolution, four steps, CFG 1, and static
shift 6. Only the transformer implementation changes.

| BF16 Turbo transformer | Balanced NVFP4 Turbo transformer |
|---|---|
| ![BF16 Turbo reference](examples/comparison_bf16_transformer.png) | ![Balanced NVFP4 Turbo](examples/cyborg_stargaze_turbo_balanced.png) |
| 4.89 s denoise, 8.52 GiB peak | 3.51 s denoise, 6.29 GiB peak |

## Matched transformer tradeoff

In the original matched two-branch harness on the project SM120 test GPU at
1024×1024, the Balanced transformer
denoised in 3.51 seconds with 6.29 GiB peak allocated, versus 4.89
seconds and 8.52 GiB for the BF16 Turbo transformer. This is about a
1.39× denoising speedup and 2.22 GiB lower transformer-stage VRAM for
this exact four-step case; results are hardware and prompt dependent.

The scaled-FP8 encoder used about 4.95 GiB standalone versus 8.36 GiB
for BF16. Direct prompt-conditioning cosine against BF16 ranged from
0.9932 to 0.9971 in the frozen four-case suite.

## CFG-1 single-branch execution

Turbo defaults to CFG 1. Following the upstream Mage pipeline, this runtime
does not encode a negative prompt or execute an unconditional transformer
branch unless CFG is greater than 1 and the negative prompt is nonempty.

At 1024×1024 in resident mode, removing that redundant branch reduced warm
denoising from 2.78 to 1.43 seconds and warm end-to-end latency from 4.90 to
3.56 seconds, a further 1.38× speedup. CFG values above 1 continue to use the
normal conditional and unconditional branches.

## Persistent server mode

For repeated generation on a 16 GiB GPU, the packaged runtime can retain the
text encoder, transformer, and VAE on CUDA instead of transferring each
component for every stage:

```python
pipe, load_report = runtime.load_pipeline(model=model_id, torch=torch)
placement = runtime.place_pipeline_resident(pipe, torch)
image, stages, screening = runtime.generate_staged(
    pipe=pipe,
    prompt=prompt,
    negative_prompt="",
    height=1024,
    width=1024,
    steps=4,
    cfg=1.0,
    seed=1,
    static_shift=6.0,
    torch=torch,
    keep_resident=True,
)
```

With the CFG-1 single-branch path active, warm 1024×1024 latency decreased
from 5.90 seconds staged to 3.56 seconds resident: a 1.66× end-to-end speedup.
Peak allocated VRAM increased from 5.89 GiB to 11.39 GiB. The staged and
resident decoded-pixel hashes were identical. Resident mode is intended for a
long-running WebUI, API server, or ComfyUI process; the default remains staged
for lower VRAM use.

Together, CFG-1 single-branch execution and resident placement reduced warm
latency from the original 7.21-second staged path to 3.56 seconds, a 2.02×
combined speedup on the tested GPU.

## Requirements

Tested on Linux x86-64, NVIDIA Blackwell SM120, CUDA 13.1, Python
3.11, PyTorch `2.13.0+cu130`, `comfy-kitchen==0.2.22`, and
`flash-attn==2.8.3`. Use a virtual environment.

```bash
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

The packaged binaries target the tested stack. Run `build_native.sh`
after changing PyTorch, CUDA, or the C++ ABI.

## Generate

```bash
CUDA_VISIBLE_DEVICES=0 .venv/bin/python generate.py \
  --model ajh-code/Mage-Flow-Turbo-NVFP4-Balanced-AJH \
  --prompt 'A detailed watercolor fox reading under an old oak tree' \
  --output fox.png --height 1024 --width 1024 \
  --steps 4 --cfg 1 --seed 1
```

Stock Diffusers does not understand the custom native NVFP4 modules;
use the packaged entry point or a compatible integration. SM120 is
required by this runtime.

## Validate

```bash
python validate_release.py
```

## Attribution

The scaled-FP8 Qwen3-VL artifact comes from
[`starsfriday/Mage-Flow-Edit-FP8`](https://huggingface.co/starsfriday/Mage-Flow-Edit-FP8)
at revision `5a64f75aa6a962cbc6c9f0bc42ed0f819b739890`.
See `THIRD_PARTY_NOTICES.md` and the included license files.