model-card.md

---
license: apache-2.0
base_model: black-forest-labs/FLUX.2-klein-4B
pipeline_tag: text-to-image
tags:
- text-to-image
- flux
- nvfp4
- svdquant
- quantization
- blackwell
- nunchaku
---

# FLUX.2 [klein] 4B — SVDQuant NVFP4 (W4A4, rank-128) — deployable on Blackwell

A **4-bit (NVFP4 W4A4)** quantization of [`black-forest-labs/FLUX.2-klein-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
(the 4-step, CFG-free distilled rectified-flow text-to-image model) that adds a **SVDQuant high-precision
low-rank branch** to stay closer to the bf16 teacher than plain NVFP4, and **runs on the real Nunchaku FP4
kernel on NVIDIA Blackwell** (~1.8× faster, ~29% less VRAM).

> **What this is:** an *ablation-backed, deployable* NVFP4 build of klein-4B. SVDQuant (Li et al., ICLR 2025),
> NVFP4, and the Nunchaku kernel are prior art; the contributions here are (1) a clean, reproducible
> measurement that the low-rank branch improves NVFP4 W4A4 **fidelity to the teacher** on this model,
> (2) a **DeepCompressor-free** bf16→Nunchaku-NVFP4 exporter (no public equivalent for klein), and
> (3) a verified deployable checkpoint on Blackwell.

## Method (calibration-free)

Per target Linear: `W ≈ L₁L₂ (bf16 low-rank, rank 128) + Q_nvfp4(W − L₁L₂)` (4-bit residual), activations
NVFP4 per-group-16. The low-rank branch is a **plain SVD of the weights** with 3 iterations of refinement to
absorb the 4-bit rounding error — **no activation calibration / no smoothing / no whitening** is needed
(SmoothQuant is harmful at this bit-width on this model; per-group NVFP4 activations need no migration).
The branch is free at inference (folded into the fused kernel). Exporter: `flux2distill/nunchaku_export.py`.

## Results (N=512, MJHQ-30k, 512px, vs the bf16 teacher)

Fidelity-to-teacher (LPIPS/PSNR/FID vs the teacher's own outputs); PickScore/CLIP stay at teacher level for
both 4-bit models, so quantization preserves semantics — the low-rank branch buys **fidelity**.

Measured N=512, MJHQ-30k, paired seeds. Our **rank-128** model vs **plain NVFP4 rank-0** (same recipe, no low-rank branch):

| metric (vs teacher) | plain NVFP4 r0 | **ours NVFP4 r128** | Δ |
|---|---|---|---|
| LPIPS↓ | 0.2076 | **0.1668** | **−19.7%** (closer) |
| PSNR↑ | 17.44 dB | **18.71 dB** | **+1.27 dB** |
| FID↓ (vs teacher) | 39.3 | **33.5** | **−14.7%** |
| PickScore↑ / CLIP↑ | 21.62 / 30.95 | 21.62 / 30.94 | ≈0 (no semantic loss; teacher = 21.64 / 30.95) |

The real Nunchaku FP4 kernel reproduces the fake-quant quality (LPIPS 0.167 vs 0.173), so the gain is real on
the deployed model. (For reference, BFL's official **FP8** klein-4B — 8-bit, ~1.2× speedup — sits closer to the
teacher at LPIPS 0.080 / PSNR 23.0; ours is the 4-bit/2.5×-kernel point with the low-rank branch closing most of the gap.)

## Speed / VRAM (RTX PRO 4500 Blackwell, batch 1, 4 steps, real Nunchaku FP4 kernel)

| | 512px | 1024px |
|---|---|---|
| end-to-end speedup vs bf16 | **1.76×** | **1.87×** |
| peak VRAM | **11.8 GB** (−29%) | **13.7 GB** (−26%) |
| per-step | 42 ms (vs 92) | 144 ms (vs 330) |

INT4 is **slower than bf16** on Blackwell (no INT4 tensor cores) — NVFP4 is the fast low-bit format here.

## Usage

Requires a Blackwell GPU (sm_120), `torch≥2.9+cu130`, and the Nunchaku FP4 runtime with the FLUX.2 loader.
Batch size must be **1** (packed rotary), and do **not** wrap generation in `torch.autocast` (the FP4 kernel
manages its own dtypes). See `docs/CUDA_SETUP_RUNBOOK.md` and `scripts/35_gen_real.py` for the exact path.

## License

Apache-2.0, inherited from the base model `black-forest-labs/FLUX.2-klein-4B`.

## Limitations

- Blackwell-only for the fast path (NVFP4 tensor cores). Batch=1 in the current fused build.
- Fidelity claim is **vs the bf16 teacher**, measured at 512px on MJHQ-30k (N=512), klein-4B only.
- Not benchmarked head-to-head against BFL's official NVFP4 checkpoint (its swizzled layout requires BFL's
  TensorRT runtime to unpack); see `report/HEADTOHEAD_klein4b_nvfp4.md` §5.