--- library_name: diffusers license: other license_name: krea-2-community-license license_link: https://www.krea.ai/krea-2-licensing pipeline_tag: text-to-image base_model: - krea/Krea-2-Turbo base_model_relation: quantized tags: - krea - krea-2 - text-to-image - image-generation - vllm - vllm-omni - humming - byteshape --- # Krea-2-Turbo Humming (ShapeLearn Quantized, vLLM-Omni) This is a quantized version of **Krea-2-Turbo** for the **[vLLM-Omni](https://github.com/vllm-project/vllm-omni)** backend with optimized **Humming kernels**, the fastest way we ship this model. Quantization was produced with **ByteShape's ShapeLearn**, which learns the optimal datatype per tensor to maintain high quality even at very low bitlengths. > **Experimental.** We tested this flow with vLLM 0.26.0, vLLM-Omni 0.26.0 and `vllm-omni-humming` 0.3.0 on NVIDIA 30-, 40-, and 50-series and RTX Pro 6000 GPUs. **Linux and NVIDIA only** (SM75+). Five sizes are available, from **14.30 GB (8.92 bpw)** down to **6.14 GB (3.83 bpw)**. All variants generate high-quality images. Each package bundles the text encoder, VAE, scheduler, and tokenizer, so there are no separate component downloads: you pick a size and serve. Krea-2-Turbo is **step-distilled**: 8 steps, no classifier-free guidance. A 1024 x 1024 image takes about **4 seconds** on an RTX 5090. See [Generate](#4-generate) for the request shape. > Need to run on something other than Linux and NVIDIA? We also publish GGUF builds of the same model: [byteshape/Krea-2-Turbo-GGUF](https://huggingface.co/byteshape/Krea-2-Turbo-GGUF). This repo drives ComfyUI too, see [Driving the Server from ComfyUI](#driving-the-server-from-comfyui). If you have questions or want to share feedback, reach us on [Reddit](https://www.reddit.com/r/ByteShape/). ## A Note on Speed Unlike the decoding stage of an LLM, diffusion inference is **not** heavily constrained by memory bandwidth. Compressing the model therefore does not necessarily make image generation faster. Quantization here buys you **VRAM headroom**, not throughput. Backend optimization is what moves the needle. This backend uses optimized Humming kernels and is about **1.6x faster per step** than the GGUF path. Measured on an NVIDIA RTX 5090 at 1024 x 1024, 8 steps, everything resident on the GPU: | Backend | Time per step (RTX 5090) | Time for 8 steps | End-to-end | |---|---|---|---| | **vLLM-Omni** (this repo) | \~0.41 s | \~3.3 s | \~4.0 s | | [GGUF](https://huggingface.co/byteshape/Krea-2-Turbo-GGUF) | \~0.67 s | \~5.4 s | \~5.9 s | Both rows are the same ByteShape recipe (the 5.06 bpw Humming transformer and the 4.93 bpw GGUF) with a BF16 text encoder. Speed is nearly flat across bit-widths; pick a size by VRAM. ## What Do the Sizes Look Like? ![One prompt rendered by BF16 and all five Humming sizes, with a matching close-up crop of the step badge and its caption under each variant](img/collage.jpg) Judge for yourself: the [blog post](https://byteshape.com/blogs/Krea-2-Turbo/) walks through the full set, and the [interactive comparison explorer](https://byteshape.com/blogs/Krea-2-Turbo/comparison/) lets you A/B any two variants across 24 curated prompts at full resolution, with a slider, zoom, and a curator note for each prompt. If you read the outputs differently, tell us on [Reddit](https://www.reddit.com/r/ByteShape/). ## Repository Layout The package is a shared **shell** (the pipeline) plus a pool of **transformers**, one per size: ``` Krea-2-Turbo/ # the shell: text encoder, VAE, scheduler, tokenizer quantized_models/ 8.92bpw/transformer/ 7.26bpw/transformer/ 5.06bpw/transformer/ 4.27bpw/transformer/ 3.83bpw/transformer/ ``` You download the shell once, then drop the transformer of your choice into it. ## Available Models | Model ID | Bits/Weight | Transformer Size | Transformer + shell (on disk) | Folder | |---|---|---|---|---| | VLLM-1 | 3.83 | 6.14 GB | 15.5 GB | `quantized_models/3.83bpw` | | VLLM-2 | 4.27 | 6.85 GB | 16.2 GB | `quantized_models/4.27bpw` | | VLLM-3 | 5.06 | 8.12 GB | 17.5 GB | `quantized_models/5.06bpw` | | VLLM-4 | 7.26 | 11.63 GB | 21.0 GB | `quantized_models/7.26bpw` | | VLLM-5 | 8.92 | 14.30 GB | 23.7 GB | `quantized_models/8.92bpw` | **Selection rule:** take the largest model that comfortably fits your card. Since compression does not speed up diffusion, there is little reason to go smaller than your VRAM allows. ## Quick Start ### 1. Set up the environment Python 3.12, one conda env, two pip lines: ```bash conda create -y -n krea2-turbo-humming python=3.12 conda activate krea2-turbo-humming pip install vllm==0.26.0 vllm-omni==0.26.0 pip install --no-deps vllm-omni-humming==0.3.0 ``` A few things that matter here: - **`--no-deps` is required.** `vllm-omni-humming` pins `vllm-omni==0.24.*` in its metadata, and pip would otherwise downgrade the whole stack. Its patches apply unchanged on 0.26.0. - **Do not install `torch` or `humming-kernels` separately.** `torch` (2.11.0, cu13), `humming-kernels` (0.1.10), `ninja` and `nvcc` all come pinned by the `vllm` wheel. `vllm-omni-humming` names `humming-kernels>=0.1.11` in an optional `kernels` extra; that extra is not installed on this path, and the 0.1.10 that vLLM pins is the version this flow was verified against. - **No system CUDA toolkit is needed.** Humming JIT-compiles its GEMM kernels at first use against the CUDA it finds in the environment. Set `CUDA_HOME` only to override that choice. Kernels cache to `~/.humming/cache`, or to `HUMMING_CACHE_DIR`. - **Pin vLLM-Omni 0.26.0.** It ships the Krea 2 pipeline in tree. There is no stable `0.25.0` on PyPI (only `0.25.0rc1`), so do not try to pin 0.25. `vllm-omni-humming` is our own plugin, and it is the one required extra piece for this path: stock vLLM-Omni cannot read a pre-quantized Humming checkpoint on its own. The plugin registers itself and detects our models from their `transformer/config.json`. ### 2. Download the model Grab the shell and the size you want: ```bash pip install -U huggingface_hub # the shell + one transformer (here the 5.06 bpw size) hf download byteshape/Krea-2-Turbo-Humming \ --include "Krea-2-Turbo/*" --include "quantized_models/5.06bpw/*" \ --local-dir Krea-2-Turbo-Humming cd Krea-2-Turbo-Humming ``` Then drop your chosen transformer into the shell. The folder is already named `transformer`, so it lands with the correct name: ```bash # symlink it in (instant, no extra disk) ... ln -sfn ../quantized_models/5.06bpw/transformer Krea-2-Turbo/transformer # ... or move it in (a real, self-contained model dir) mv quantized_models/5.06bpw/transformer Krea-2-Turbo/transformer ``` To try a different size later, swap the symlink, or move one transformer out and another in. The folders are named by bit-width (`8.92bpw`, `5.06bpw`, `3.83bpw`, and so on). ### 3. Serve From the folder root: ```bash vllm-omni serve Krea-2-Turbo \ --omni --served-model-name krea/Krea-2-Turbo \ --enforce-eager --port 8126 ``` - Quantization is read from `transformer/config.json` (`quant_method: humming`). Pass **no** `--quantization` flag and set **no** `VLLM_HUMMING_*` variables. - Leave `VLLM_PLUGINS` unset. It is an allowlist: if it is set at all, it must name `vllm_omni_humming`, or the checkpoint fails to load. - Add `--enable-cpu-offload` only if a variant does not fit your VRAM. On a 32 GB card none of these need it. Ready when the log prints `Application startup complete`; check with `curl -s http://127.0.0.1:8126/v1/models`. **The first start is the slow one**, because the kernel cache is cold. Expect a few minutes while Humming compiles for your card, then it caches. ### 4. Generate The server exposes the OpenAI images API. This one is a quick 1024 x 1024 smoke test. Once it works, switch to `"size": "2048x2048"`, which is where this model looks best ([why](#run-it-at-2k)): ```bash curl -s http://127.0.0.1:8126/v1/images/generations \ -H 'Content-Type: application/json' \ -d '{ "model": "krea/Krea-2-Turbo", "prompt": "a red panda on a mossy log, soft morning light", "size": "1024x1024", "num_inference_steps": 8, "seed": 42, "n": 1, "response_format": "b64_json" }' \ | python3 -c "import sys,json,base64; d=json.load(sys.stdin); open('out.png','wb').write(base64.b64decode(d['data'][0]['b64_json']))" ``` Krea-2-Turbo is CFG-distilled. The pipeline keeps guidance off by default via `is_distilled: true` in `model_index.json`, and the request above gets that default by simply not mentioning guidance. There is nothing for a `negative_prompt` to steer either, so leave that field out too. **Do not send `guidance_scale`.** Supplying it at all with a value above 0 overrides the distilled default and switches classifier-free guidance back on, which **doubles the work per step** and gives a harder, over-saturated image. Omit the field, or send `0`. ### Run it at 2K **Send `"size": "2048x2048"` if you can.** Krea-2-Turbo resolves noticeably more real detail at 2K than at 1K: fur and hair separate into individual strands, bark and foliage keep their texture instead of dissolving into soft blobs, and fine features like whiskers survive. The same prompt and seed at 1024 x 1024 give a good but distinctly softer image. It is not free. 2K is 4x the pixels and costs substantially more time per step, so iterate on prompts at 1024 x 1024, then re-run the keeper at 2K. ## Driving the Server from ComfyUI If you prefer a GUI over `curl`, the `ComfyUI-vLLM-Omni` custom node sends generation requests to a running vLLM-Omni server, local or remote. ComfyUI is only the front end; vLLM still does the work. Install the node into your ComfyUI installation: ```bash cd /path/to/ComfyUI/custom_nodes git clone https://github.com/vllm-project/vllm-omni.git cp -r vllm-omni/apps/ComfyUI-vLLM-Omni ComfyUI-vLLM-Omni rm -rf vllm-omni ``` Restart ComfyUI (`python main.py`), then open a browser and go to **http://127.0.0.1:8188**. A new **vLLM-Omni** folder appears in the **Node Library** sidebar. No extra Python packages are needed beyond ComfyUI's own. Keep the vLLM-Omni server from step 3 running. ComfyUI reaches it at `http://127.0.0.1:8126/v1`. If ComfyUI runs on a different machine, use the server's address instead of `127.0.0.1`. **Use the workflow shipped in this repo: [`krea2-turbo-humming-comfyui-workflow.json`](krea2-turbo-humming-comfyui-workflow.json).** Drag it onto the ComfyUI canvas and hit Run. It is already set up for Krea-2-Turbo at 2048 x 2048, 8 steps, guidance off. Adjust **url** if your server is not on `127.0.0.1:8126`. > **Set `guidance_scale` to 0.** This is the one setting that will bite you. The **Diffusion Sampling Params** node defaults to **7.5**, and this pipeline treats *any* supplied `guidance_scale > 0` as "turn classifier-free guidance on", even for a distilled model, where `is_distilled: true` would otherwise keep it off. Leaving the default doubles the time per step and gives a harder, over-saturated image. The shipped workflow sets it to 0. If you would rather build it by hand, start from the node's own template at **sidebar → Templates → ComfyUI-vLLM-Omni → vLLM-Omni Image Generation** (it is wired for image *editing*, so delete the `Load Image` node), and in the **Generate Image** node set: - **url** → `http://127.0.0.1:8126/v1` - **model** → `krea/Krea-2-Turbo` - **prompt** → your text; leave **negative_prompt** empty - **width** / **height** → 2048 x 2048 for best quality, 1024 x 1024 while iterating (2048 is the node's maximum) In the attached **Diffusion Sampling Params** node set `num_inference_steps` to **8** and `guidance_scale` to **0**, then run the workflow. ## Troubleshooting | Symptom | Cause and fix | |---|---| | `HummingConfig ... unexpected keyword argument`, or a shape mismatch at load | The plugin is not installed in the active environment. Check with `pip show vllm-omni-humming`. | | Checkpoint fails to load, plugin never registers | `VLLM_PLUGINS` is set but does not include `vllm_omni_humming`. Unset it. | | pip downgraded vLLM to 0.24 | `vllm-omni-humming` was installed without `--no-deps`. Reinstall the stack. | | Unknown pipeline / Krea 2 not recognised | vLLM-Omni predates the in-tree Krea 2 pipeline. Install `vllm-omni==0.26.0`. | | Half the expected speed, and harder / over-saturated images | A `guidance_scale` above 0 was sent, which turns CFG back on for this distilled model. Omit the field or set it to `0`. In ComfyUI the **Diffusion Sampling Params** node defaults to 7.5. | | Blurry, unfinished-looking images | Too few or too many steps for a distilled model. Use 8. | | First serve hangs for minutes | Cold Humming kernel cache. Expected once per GPU; it caches to `~/.humming/cache`. | | Out of memory on serve | Add `--enable-cpu-offload`, or pick a smaller bit-width from the table above. | ## License Krea 2 Community License. Read it at [krea.ai/krea-2-licensing](https://www.krea.ai/krea-2-licensing) ([PDF](https://cdn.jsdelivr.net/gh/krea-ai/krea-2@db3984fbc6e13b34c0064990fc2d95ac64d00058/assets/hf_samples/LICENSE.pdf)); the same PDF is bundled as `LICENSE.pdf` in the shell folder. Use is additionally subject to Krea's [Acceptable Use Policy](https://www.krea.ai/krea-2-use-policy). Quantized weights inherit the base model's terms. These weights are a **quantized** version of [krea/Krea-2-Turbo](https://huggingface.co/krea/Krea-2-Turbo), modified by quantizing the transformer with ByteShape's ShapeLearn. They are not an official Krea product and are not endorsed by Krea. > Krea 2 is licensed under the Krea 2 Community License Agreement. For more information, visit https://krea.ai/krea-2-licensing.