xocialize's picture
FLUX.2 klein 4B 1-step RDM (epfl-vita, Apache-2.0) as a diffusers tree for the Swift/MLX port
2037b5c verified
|
Raw History Blame Contribute Delete
3.63 kB
metadata
library_name: mlx
license: apache-2.0
pipeline_tag: text-to-image
base_model: epfl-vita/flux2-klein-1step-rdm
tags:
  - mlx
  - safetensors
  - apple-silicon
  - text-to-image
  - diffusion
  - mmdit
  - flux2
  - klein
  - one-step
  - distillation

mlx-community/FLUX.2-klein-4B-1step-RDM-bf16

epfl-vita/flux2-klein-1step-rdm (Apache-2.0) as a standard diffusers tree for the Swift/MLX FLUX.2 Klein port. The DiT is FLUX.2 [klein] 4B fine-tuned by EPFL VITA to draw an image in one step (Representation Distribution Matching, arXiv:2607.02375). The text encoder, VAE, tokenizer and scheduler are FLUX.2 [klein] 4B's own, unchanged, from mlx-community/FLUX.2-klein-4B-bf16.

Folder What
transformer/ RDM's DiT, bf16, in diffusers' layout (converted from its BFL-layout model.safetensors by make_rdm_snapshot.py)
text_encoder/, tokenizer/ Qwen3-4B conditioner, as in FLUX.2 [klein] 4B
vae/, scheduler/, model_index.json as in FLUX.2 [klein] 4B

Conversion

The source checkpoint holds the DiT in Black Forest Labs' native key layout under its training adapter's model. prefix. make_rdm_snapshot.py strips the prefix, maps BFL keys to diffusers' (qkv split into to_q/to_k/to_v and add_q/k/v_proj, the final adaLN's (shift, scale) halves swapped to diffusers' (scale, shift)), and checks the result against FLUX.2 [klein] 4B: all 169 keys and shapes match, and every tensor has cosine ≈ 1.000 with the base model's (RDM moves each by about 0.15 % relative). No weights were retrained or requantized.

Measured (Apple M5 Max, bf16, warm)

Against FLUX.2 [klein] 4B at its distilled 4 steps, same encoder and VAE:

512² 1024²
This model, 1 step 0.70–0.95 s 2.8–2.9 s
FLUX.2 [klein] 4B, 4 steps 1.97–2.46 s 6.5–6.6 s

On our own prompt sets, none of which appear in GenEval:

  • SigLIP2 image-quality scores were level with the 4-step model (0.903 vs 0.903; hard prompts 0.862 vs 0.866).
  • Keyword recall was level or better (1.00 vs 1.00; hard prompts 0.94 vs 0.89).
  • It spelled in-image text a little less reliably on our hard prompts.

comparison.jpg: the 4-step model (top) and this one (bottom), same prompts and seeds, 1024².

It was trained and evaluated by its authors at 512²; it holds up at 1024² in our runs. No CFG: a negative prompt does nothing. Text to image only.

The authors' GenEval figure (0.826) is partly in-distribution: about 17.6 % of GenEval's prompts are in its training pool (see the source model card).

Use (Swift / MLXEngine)

import MLXKlein
let config = KleinConfiguration(repo: "mlx-community/FLUX.2-klein-4B-1step-RDM-bf16", defaultSteps: 1, guidanceScale: 1.0)
let pkg = Klein4BT2IPackage(configuration: config)

Code: https://github.com/xocialize/flux2-klein-swift. In Gaea it is the klein-4b-rdm model of gaea-plugin-flux2-klein.

Licence and credit

Apache-2.0 (LICENSE, NOTICE). The RDM weights are © their authors (Lan Feng, Wuyang Li, Éloi Zablocki, Matthieu Cord, Alexandre Alahi; EPFL VITA), derived from FLUX.2 [klein] 4B by Black Forest Labs (Apache-2.0).

@article{feng2026rdm,
  title={Representation Distribution Matching for One-Step Visual Generation},
  author={Feng, Lan and Li, Wuyang and Zablocki, {\'E}loi and Cord, Matthieu and Alahi, Alexandre},
  journal={arXiv preprint arXiv:2607.02375},
  year={2026}
}