WaveCut/Krea-2-Turbo-OrbitQuant-W4A4

Krea 2 Turbo OrbitQuant W4A4

Deployment checkpoint for krea/Krea-2-Turbo. Both transformer-class components use OrbitQuant: the Qwen3-VL text encoder and the Krea 2 diffusion transformer. The Qwen linears are fully W4A4. The DiT uses W4A4 while retaining its 28 quality-sensitive main-block ff.down projections and latent input projection img_in in BF16.

The embedded preview links to the lossless 8192×10720 matrix. Every source tile in the original matrix remains at the model's native benchmark output size of 2048×2048; labels are added and tiles are concatenated without resizing. The twenty individual PNGs are in artifacts/generations/.

At a glance

Item	Value
Source revision	98e0fe118d17c9e3547fbb2e25acdbae2cadf7c7
Quantized components	text_encoder (Qwen3VLModel) and transformer (Krea2Transformer2DModel)
Qwen policy	356/356 linear modules OrbitQuant W4A4
DiT policy	232 OrbitQuant W4A4 linears; 32 BF16 linears, including 28 protected main-block ff.down projections and img_in
Quantization	RP-BH rotation, Lloyd-Max codebooks, no calibration dataset
Packed runtime	native_packed_matmul, direct INT8-surrogate activation preparation, adjacent-projection activation reuse, no decoded-weight cache
Tested Turbo settings	2048×2048, 8 steps, guidance 0, distilled schedule (mu=1.15)
Benchmark GPU	NVIDIA A40 48 GB, Torch 2.8.0+cu128
Learned-component storage	12.59 GiB vs 32.74 GiB BF16 (-61.6%)
Repository access	Public and ungated

Install and run

python -m venv .venv
source .venv/bin/activate
pip install -r https://huggingface.co/WaveCut/Krea-2-Turbo-OrbitQuant-W4A4/resolve/main/runtime-requirements.txt
hf download WaveCut/Krea-2-Turbo-OrbitQuant-W4A4 scripts/run_inference.py --local-dir .
orbitquant kernels-install --build

kernels-install uses a compatible prebuilt kernel when available. --build allows an exact local build for an unlisted Torch/CUDA ABI and requires the CUDA toolkit plus ninja.

The recommended runner stages Qwen and the DiT sequentially, enables tiled VAE decoding, and requires Flash SDPA plus packed OrbitQuant kernels:

python scripts/run_inference.py \
  --prompt "A rain-soaked Warsaw street seen through a tram window" \
  --width 2048 --height 2048 --steps 8 --seed 0 \
  --output krea2-orbitquant.png

This staging policy matters at 2048×2048. Qwen is used to produce prompt embeddings and released before the DiT and VAE move to CUDA. Keeping every component resident at once is simpler, but wastes VRAM without improving the result.

Direct Diffusers loading is also supported when the GPU can hold the whole pipeline:

import os
import torch

os.environ.setdefault("ORBITQUANT_STRICT_PACKED", "1")
import orbitquant  # registers the OrbitQuant Hugging Face integrations
from diffusers import Krea2Pipeline

pipe = Krea2Pipeline.from_pretrained(
    "WaveCut/Krea-2-Turbo-OrbitQuant-W4A4",
    torch_dtype=torch.bfloat16,
    is_distilled=True,
).to("cuda")

image = pipe(
    prompt="A clean technical poster with readable labels",
    width=1024,
    height=1024,
    num_inference_steps=8,
    guidance_scale=0.0,
    generator=torch.Generator(device="cuda").manual_seed(0),
).images[0]

What is quantized

Component	Source weights	Saved artifact	OrbitQuant linears	BF16 linears	Linear weight coverage
Krea DiT	24.48 GiB	9.97 GiB	232	32	77.70%
Qwen3-VL	8.27 GiB	2.62 GiB	356	0	100.00%
Total	32.74 GiB	12.59 GiB	588	32	83.05%

The DiT remains an OrbitQuant component. The sensitive output projection of each of its 28 main feed-forward blocks and the small img_in projection are protected; img_in adds only 0.76 MiB of BF16 weights. Attention projections, FFN gate/up projections, and the text-fusion stack remain packed W4A4. The universal policy also keeps two time-embedding projections and the final output projection in source precision. Embeddings, normalization parameters, convolutions, biases, the VAE, scheduler, and tokenizer are not 4-bit tensors.

Exact module names and parameter counts are in quantization_manifest.json.

Latency and VRAM

Measurements use the same A40, BF16 non-linear tensors, 2048×2048 output, 8 denoising steps, guidance 0, strict BF16 Flash SDPA, tiled VAE, lossless valid-token prompt compaction, and sequential Qwen→DiT component staging. Network downloads and PNG writes are excluded. No callback, latent copy, SageAttention, quantized attention, or full-model torch.compile is present in the timed path.

Metric	Original BF16	OrbitQuant W4A4	Change
First process generation, compiled-shape cache present	54.747 s	51.563 s	-5.8%
Hot end-to-end median	54.579 s	49.986 s	-8.4%
Hot prompt encode median	0.097 s	0.187 s	+93.0%
Hot DiT + tiled VAE median	54.495 s	49.793 s	-8.6%
Sequential peak, nvidia-smi	30.70 GiB	18.73 GiB	-39.0%
Sequential peak, Torch allocated	28.30 GiB	16.41 GiB	-42.0%
Component load time	16.226 s	5.344 s	-67.1%
Learned-component storage	32.74 GiB	12.59 GiB	-61.6%

An intentionally empty Triton cache took 68.875 s for the first OrbitQuant image: 7.057 s for Qwen encode and 61.818 s for DiT plus VAE. Triton specializes some kernels for new prompt lengths, so the first encounter with another uncached shape can also pay a one-time compilation cost. The hot table is a second ten-prompt sweep after all ten tested shapes were compiled; it is not presented as cold-start latency.

The runtime prepares RP-BH activations directly as INT8 centroid surrogates and reuses that exact prepared tensor across adjacent Q/K/V/gate and SwiGLU gate/up projections. On the controlled eight-step denoiser benchmark this reduced the previous packed runtime from 49.470 s to 47.878 s without changing its output hash; the BF16 denoiser took 52.626 s. Final denoiser peak allocation was 12.59 GiB, versus 26.40 GiB for BF16. The small BF16 img_in protection did not erase the runtime gain.

The previously surprising ~44 GiB behavior is avoided by the included runner: it releases Qwen before loading the denoiser, compacts padded prompt lanes before unmasked Flash attention, does not retain decoded INT8 weights, and does not keep every pipeline component resident simultaneously. The text encoder itself is slower than BF16, but contributes only about 0.19 s to a hot 2048² request; the DiT dominates the net speedup.

Raw records are in benchmark/. nvidia-smi includes the CUDA context and non-Torch allocations; Torch figures use torch.cuda.max_memory_allocated().

Quality-preservation policy

The protected DiT policy was selected with component isolation, per-step latent trajectories, and native-size image inspection:

Isolation case	Outcome on the product-detail stress prompt
OrbitQuant Qwen + BF16 DiT	Did not reproduce the dirty metal, broken reflection, or diagonal-line defect
BF16 Qwen + W4A4 DiT	Reproduced the defect
BF16 Qwen + W4 DiT with activation quantization disabled	Still reproduced the defect
BF16 Qwen + dequantized W4 DiT weights	Still reproduced the defect
W4A4 DiT with all 28 main ff.down projections in BF16	Restored clean reflective surfaces and substantially straighter grate detail
Protected-FFN DiT + BF16 img_in	Removed the remaining blotching in smooth sky, studio-gradient, and flat-fill stress cases

For the 34-token prompt, the quantized Qwen embeddings had cosine similarity 0.9674 to BF16 over valid tokens. This is a meaningful numerical difference, but it was not the source of the observed texture failure. The model therefore keeps Qwen fully W4A4.

The residual smooth-region problem was localized separately to DiT img_in. Mid-frequency residual ratios are normalized to BF16 (1.0 is closest): sky improved from 1.8497 to 1.0667, studio gradient from 1.3489 to 0.8760, and flat vector fills remained neutral (0.9853 → 1.0102). These are targeted diagnostic ratios, not a general image-quality score.

The diagnostic reports and causal matrices are retained under artifacts/diagnostics/.

Comparison protocol

#	Prompt ID	Stress category	Seed
01	studio-product	product detail	61000
02	human-portrait	portrait	61001
03	mucha-poster	public-domain artist style	61002
04	hokusai-wave	public-domain artist style	61003
05	bauhaus-risograph	popular image style	61004
06	technical-cutaway	technical diagram	61005
07	long-latin-text	long Latin text	61006
08	long-cyrillic-text	long Cyrillic text	61007
09	mixed-diagram	mixed Latin/Cyrillic diagram	61008
10	wide-city-scene	dense scene composition	61009

All pairs use the same prompt, seed, 2048×2048 output, 8-step distilled schedule, and guidance 0. The set probes product texture, portraits, line art, fine diagrams, Latin and Cyrillic text, mixed scripts, and dense reflections. It is a practical deployment check, not an FID, CLIP, or human-preference benchmark.

Repository contents

• text_encoder/: fully packed OrbitQuant W4A4 Qwen3-VL component.
• transformer/: OrbitQuant W4A4 Krea DiT with 28 protected BF16 FFN outputs plus BF16 img_in.
• vae/, scheduler/, and tokenizer/: pinned source components.
• artifacts/generations/original/ and artifacts/generations/orbitquant/: ten paired native-size PNG sets.
• assets/original_vs_orbitquant_w4a4.webp: full-resolution lossless comparison matrix.
• assets/original_vs_orbitquant_w4a4_preview.webp: reduced card preview linked to the original matrix.
• artifacts/diagnostics/: Qwen/DiT isolation and protection-ladder evidence.
• benchmark/: raw timings, memory measurements, prompts, environment, and kernel microbenchmarks.
• scripts/run_inference.py: recommended low-VRAM sequential runner.
• runtime/: the exact OrbitQuant wheel used by the runner and benchmarks.
• quantization_manifest.json, NOTICE, and MODIFICATIONS.md: provenance and modification details.

Limitations

• The first call pays a packed-kernel compilation cost; measure hot latency after warm-up.
• W4A4 may still change composition, contrast, fine typography, object counts, and small labels. Inspect the paired PNGs for your workload.
• Strict Flash SDPA requires a compatible CUDA/PyTorch build; the runner fails instead of silently selecting a slower high-memory attention path.
• Latency and memory depend on GPU architecture, drivers, Torch, Triton, resolution, and allocator state.
• This derivative inherits the source model's intended-use, safety, and license restrictions.

License and attribution

This is a modified derivative of Krea 2 Turbo. The upstream Krea 2 Community License Agreement is copied as LICENSE.pdf. The required upstream notice and modification notice are in NOTICE, with a technical summary in MODIFICATIONS.md. Review the agreement itself before use or redistribution; no endorsement by Krea is implied.