WaveCut/FLUX-3-Image-Klein-OrbitQuant-W4A8

FLUX.3 Image Klein OrbitQuant W4A8

OrbitQuant 4-bit weights for FLUX 3 Image Klein, the distilled 5.6B model for text-to-image generation and image editing with up to 10 reference images. The transformer runs 8-bit activations over 4-bit weights. The Qwen3-VL text encoder uses 6-bit weights and activations. The runtime uses the native flux3 pipeline of Black Forest Labs (16 steps, no guidance) and the ComfyUI-OrbitQuant nodes. The bundle is 7.9 GB. The original is 20.3 GB.

On one RTX 5090 a 1024×1024 image takes 2.11 s (BF16: 4.30 s) with 13.0 GiB of device memory at the peak (BF16: 29.0 GiB). With --offload the peak is 6.0 GiB and the image takes 5.25 s. The Performance section has the details.

Usage

Use Linux, Python 3.12 and an NVIDIA GPU with compute capability 8.0 or newer. The tested stack is Torch 2.12.1 with CUDA 13.0 on an RTX 5090 (driver 595.91.07).

python3.12 -m venv .venv && . .venv/bin/activate
pip install torch==2.12.1 torchvision==0.27.1 --index-url https://download.pytorch.org/whl/cu130
pip install huggingface_hub
hf download WaveCut/FLUX-3-Image-Klein-OrbitQuant-W4A8 --local-dir klein-orbitquant
cd klein-orbitquant
pip install -r requirements.txt
pip install --no-deps "flux3 @ git+https://github.com/black-forest-labs/flux-3-image-klein.git@45e57c9fb06f9751cdced29a65c5d2171003be05"
python -m orbitquant.cli.main kernels-install

Text to image:

python run_flux3.py --prompt "A wide-angle photograph of a lone American bison in a misty geothermal landscape, half hidden by white steam, a slate-blue stream and evergreens fading into the mist behind. Muted, cool palette, soft overcast light." --seed 7 --output bison.png

Editing. The output takes the size of the first reference unless you set --width and --height:

python run_flux3.py --prompt "The same lone American bison in the same misty landscape, now at night under a full moon: the steam lit from behind, cool blue light, deep shadows, stars above the evergreens." --image bison.png --seed 7 --output bison_night.png

Python, inside klein-orbitquant:

from flux3_orbitquant import Klein

model = Klein(".")
image = model.generate("A full scene caption ...", seed=7)
image.save("image.png")

Options of run_flux3.py (Klein takes the same settings as arguments):

• --offload: keep the transformer, the text encoder and the autoencoder in host memory. Each moves to the GPU for its own stage.
• --attention int8: INT8 Q·K attention. Text to image takes 1.93 s instead of 2.11 s, and the image changes slightly.

Write a full scene caption: framing, subject, background and lighting. A short prompt gives a generic image. Black Forest Labs publishes a prompt upsampler that writes such captions. This release does not include it.

ComfyUI

ComfyUI-OrbitQuant lists this release as WaveCut/FLUX-3-Image-Klein-OrbitQuant-W4A8. Connect OrbitQuant Model Loader to OrbitQuant Generate Image and save the result with a standard save node. Connect up to 10 images to reference_images for editing. The generator runs in the Python environment that python_executable selects. Install the packages of the Usage section there.

• profile: default, model-offload or int8-attention.
• keep_loaded (on by default): the worker keeps the model for the next request. The first text-to-image request takes 26.3 s because it loads the model. A later one takes 3.0 s. A request with another model or profile replaces the kept model. A video or audio run, a run with keep_loaded off, or an interrupt stops the worker and frees its GPU memory.

On the RTX 5090 the device total peaks at 12.6 GiB with the default profile. With model-offload it peaks at 6.2 GiB, and a request with keep_loaded off takes 27.2 s. These values include the 1.0 GiB that the idle ComfyUI server holds.

Quality

Mean LPIPS to BF16 over the 11 examples of the original repository, same seed (lower is closer):

Configuration	LPIPS
FP8 (variant of Black Forest Labs)	0.020
OrbitQuant W4A8	0.099
OrbitQuant W4A8, INT8 attention	0.098

The images differ from BF16 in details and sometimes in layout. Small lettering in BFL's typography examples, same seed:

Prompts of the OpenRouter Image Benchmarks, turned into captions by BFL's upsampler. Both columns use the same caption and seed. Edits show their reference images at the left.

Performance

One RTX 5090 (575 W, driver 595.91.07), Torch 2.12.1 with CUDA 13.0. One process per configuration in one session, batch size 1, 16 steps. Time is the warm median of 9 runs after 2 warm-up runs (5 runs with offload). It covers prompt encoding, sampling and VAE decoding, with CUDA synchronized at both ends. Device peak is the total from NVML at 0.1 s intervals, including the CUDA context. Tensor peak is the PyTorch allocator peak. Host RAM is the peak resident set of the process, including loading.

Configuration	Text to image	Edit, 1 reference	Device peak	Tensor peak	Host RAM peak	Checkpoint
BF16 (original)	4.30 s	9.33 s	29.0 GiB	23.0 GiB	11.2 GiB	20.3 GB
FP8 (variant of Black Forest Labs)	4.09 s	9.54 s	25.3 GiB	18.6 GiB	6.7 GiB	15.5 GB
OrbitQuant W4A8	2.11 s	4.73 s	13.0 GiB	10.4 GiB	5.6 GiB	7.9 GB
OrbitQuant W4A8, INT8 attention	1.93 s	3.95 s	12.4 GiB	10.4 GiB	5.6 GiB	7.9 GB
BF16 (original), offload	15.07 s	21.90 s	13.9 GiB	11.9 GiB	25.0 GiB	20.3 GB
OrbitQuant W4A8, offload	5.25 s	8.71 s	6.0 GiB	4.4 GiB	13.9 GiB	7.9 GB

The edit uses one 1024×1024 reference, which doubles the token count. Memory values are the larger of the two workloads. The loader drops the vision tower and the decoder layers after 32 from the text encoder. The pipeline encodes text only and reads hidden states 4 to 32. The loader also enables expandable allocator segments when PYTORCH_CUDA_ALLOC_CONF is empty. Other GPUs are untested.

Components

Component	Precision	Size
Transformer, 5.5B parameters	OrbitQuant 4-bit weights and 8-bit activations for the 258 block projections. INT4 for the 6 modulation projections. BF16 input, time and output layers.	2.94 GB
Text encoder, Qwen3-VL-4B-Instruct	OrbitQuant W6A6 for the language model. BF16 for layers 6 and 16 and for the vision tower, which the runtime does not load.	4.59 GB
Autoencoder	FP32, unchanged	0.34 GB

The text encoder keeps 6 bits because its residual stream carries values near 11,000 in the first token.

Files

• dit/: the packed transformer (model.safetensors), its codebooks and rotations, and the OrbitQuant configuration and manifest.
• text_encoder/: the quantized Qwen3-VL-4B-Instruct with its tokenizer and processor files.
• ae.safetensors: the autoencoder of the original model.
• flux3_orbitquant.py: the loader. run_flux3.py: the command-line entry point. The fused transformer blocks come from orbitquant 0.13.
• manifest.json: the components and the runtime pins. SHA256SUMS: the checksums of all files.

Limits and license

• The fused blocks run one sample at a time. The base model with guidance, which packs two samples, is not supported.
• Small text, counts and fine spatial relations can fail, as in the original model.

This release is a modified version of FLUX 3 Image Klein under the FLUX Kommunity License v.1.0. The license permits non-commercial use, and commercial output generation by Qualifying Users with less than US$5M annual revenue, under its conditions. Read the license before any commercial use, including its content-filter and disclosure requirements. Keep the attribution in NOTICE.md. Black Forest Labs did not create, endorse or validate this release. The Qwen3-VL-4B-Instruct text encoder is under the Apache License 2.0.