README.md

---
license: other
license_name: see-individual-model-repos
pipeline_tag: text-to-image
tags:
  - w4a8
  - quantized
  - int4
  - comfyui
  - comfy-kitchen
  - convrot
  - low-vram
  - safetensors
---

# Rebels W4A8 Collection

 
<table align="center">
 <tr>
    <td><img src="https://cdn-uploads.huggingface.co/production/uploads/68761990332d15464ccc8dee/SNhCJ0PXAiQ9eAD18Ywtt.png" width="300"/></td>
    <td><img src="https://cdn-uploads.huggingface.co/production/uploads/68761990332d15464ccc8dee/89-J55SmjsVOtPH4gaUz-.png" width="300"/></td>
    <td><img src="https://cdn-uploads.huggingface.co/production/uploads/68761990332d15464ccc8dee/UkEPjN6KJUsomhZiD8oc7.png" width="300"/></td>
  </tr>
</table>



## Models

| Model | Params | Size | Notes |
|---|---|---|---|
| **Flux2-Klein-4B-w4a8** | 3.9 B | **2.46 GB** | Apache-2.0. Smallest here — runs on 6 GB cards |
| **Z-Image-Turbo-w4a8** | 6.2 B | **3.67 GB** | Distilled — 8 steps, CFG 1.0 |
| **Flux2-Klein-9B-w4a8** | 9.1 B | **5.62 GB** | Flux License |
| **Krea-2-Turbo-w4a8** | 12.8 B | **7.23 GB** | Turbo — CFG 1.0, not 4.0 |
| **SCAIL-2-14B-w4a8** | ~14 B | **_.__ GB** | Wan2.1-based video, NEEDS WORK |
| **Wan-Animate-2-TURBO-w4a8** | 16.4 B | **9.59 GB** | Video. **Reference image must match the driving video's opening pose** |
| **Qwen-Image-2512-w4a8** | ~20 B | **14.5 GB** | Full model — normal steps and CFG, not the Flash recipe, NEEDS WORK |
| **MiniMax-H3-REF2VA-w4a8** | 33.1 B | **24.5 GB** | Reference-to-video with audio. Mixed: `adaln_proj` at int8 — its 2688-wide rows can't use the w4a8 kernel |

All load with the stock **Load Diffusion Model** node. Base licenses carry over — check each source model before commercial use.

All load with the stock **Load Diffusion Model** node. Base licenses carry over — check each source model before commercial use.

**Four-bit weights. Eight-bit math. Native ComfyUI kernels. No custom nodes.**

W4A8 conversions of current diffusion and video models, quantized to run on ComfyUI's native `asym_w4a8_int8` path — where int8 tensor cores do the work instead of a dequantize-then-fp16 fallback.

Roughly **0.56 bytes per parameter**, and it *runs* at that size rather than merely storing at it.

---
<table align="center">
  <tr>
    <td><img src="https://cdn-uploads.huggingface.co/production/uploads/68761990332d15464ccc8dee/khVfxQbfqNsXy7ceyD8Y9.png" alt="INT8" width="150"/></td>
    <td><img src="https://cdn-uploads.huggingface.co/production/uploads/68761990332d15464ccc8dee/hVnkIGDU8p8hh90cVhowJ.png" alt="w4a8" width="150"/></td>
  </tr>
  <tr>
    <td align="center"><b>INT8</b></td>
    <td align="center"><b>w4a8</b></td>
  </tr>
</table>


## Why W4A8 instead of GGUF

GGUF is excellent and I ship plenty of it. But every GGUF forward pass unpacks weights back to fp16 before the matmul — the file is small, the math is not. W4A8 keeps compute in int8 end to end.

| | GGUF Q4_K_M | W4A8 |
|---|---|---|
| Storage | ~0.60 B/elem | ~0.56 B/elem |
| Compute path | dequant → fp16 GEMM | int8 GEMM |
| Loader | ComfyUI-GGUF node | **stock Load Diffusion Model** |
| Weight error (measured) | varies by tensor | ~7% relL2 |

The format comes from Kijai's `AsymW4A8Int8Layout` work in comfy-kitchen. This collection is about applying it correctly to models nobody has converted yet, and being explicit about what was verified.

---




## What's inside a file

Each quantized Linear stores five pieces:

| Tensor | Purpose |
|---|---|
| `weight` | int4 codes, two per byte |
| `weight_s_rel` | fp8 scale, one per group of 16 |
| `weight_s_channel` | one scale per output channel |
| `weight_codebook` | 16 Lloyd-Max levels, fit to the tensor |
| `comfy_quant` | layout config the loader reads |

Three ideas stacked: a **ConvRot Hadamard rotation** that flattens outliers so four bits go further, a **codebook** of non-uniform levels fit to the actual weight distribution instead of an even grid, and **per-group fp8 scales** preserving local dynamic range. Calibration-free — no activation dataset, so nothing in the conversion biases the model toward one kind of prompt.

---

## What I do differently

**Sensitive layers are never quantized.** Timestep embeddings, conditioning projections, patch projections, final output layers and rotary tables stay high precision. On a few-step model the timestep embedder has only a handful of sigma values to distinguish — crushing it to four bits corrupts every step of the schedule. Every file is checked after conversion to confirm those layers really are stored at F16/F32, because quantizers do not preserve them automatically.

**Mixed formats where the kernel demands it.** The fused W4A8 kernel accepts a ConvRot group of exactly 256, so any layer whose input dimension isn't divisible by 256 cannot use it. Rather than silently shipping a file that errors on load, those layers are written as `int8_tensorwise` — also native, no group constraint, ~1% error. Each model card states which layers took that path.

**Every file is measured.** Conversion reports per-layer reconstruction error against the original bf16 weights. Anything that doesn't land where it should doesn't get uploaded.

---

## Requirements

- **ComfyUI 0.30.0+** with `asym_w4a8_int8` in its native quant registry
- **comfy-kitchen** installed (ships the kernels)
- An NVIDIA GPU or AMD GPU.

On startup ComfyUI prints its available formats. You want `asym_w4a8_int8` in the **Native ops** list — under *emulated* it still runs, without the int8 speed advantage.

---

## Usage

1. Drop the `.safetensors` in `ComfyUI/models/diffusion_models`
2. Load it with **Load Diffusion Model** — the stock node, no custom loader
3. Text encoder, VAE and sampler settings are unchanged from the base model

Per-model notes (step counts, CFG, resolution) live in each model's card. Distilled models have fixed schedules that must be respected — base-model settings on them produce poor results regardless of quantization.

---

## Models

*Collection in progress. Each conversion has its own repo with exact sizes, measured error, and the list of layers kept at high precision.*

---

## Licensing

These are quantized derivatives. **Every original license and usage restriction carries over unchanged**, and each model repo states the license of its base model. Check the specific model's card before commercial use — several bases in this collection are not permissive.

---

## Credits

- **Kijai** — the W4A8 int8-codebook layout and kernels
- **Comfy-Org / comfyanonymous** — comfy-kitchen and the native quantization registry
- **city96** — ComfyUI-GGUF, which taught most of us how quantized loading works in ComfyUI
- Original model authors — all base licenses apply

Quantized by [RealRebelAI](https://huggingface.co/realrebelai) · [GitHub](https://github.com/RealRebelAI) · [X](https://x.com/realrebelai)