Rebels W4A8 Collection


INT8	w4a8

Four-bit weights. Eight-bit math. Native ComfyUI kernels. No custom nodes.

W4A8 conversions of current diffusion and video models, quantized to run on ComfyUI's native asym_w4a8_int8 path — where int8 tensor cores do the work instead of a dequantize-then-fp16 fallback.

Roughly 0.56 bytes per parameter, and it runs at that size rather than merely storing at it.

Why W4A8 instead of GGUF

GGUF is excellent and I ship plenty of it. But every GGUF forward pass unpacks weights back to fp16 before the matmul — the file is small, the math is not. W4A8 keeps compute in int8 end to end.

	GGUF Q4_K_M	W4A8
Storage	~0.60 B/elem	~0.56 B/elem
Compute path	dequant → fp16 GEMM	int8 GEMM
Loader	ComfyUI-GGUF node	stock Load Diffusion Model
Weight error (measured)	varies by tensor	~7% relL2

The format comes from Kijai's AsymW4A8Int8Layout work in comfy-kitchen. This collection is about applying it correctly to models nobody has converted yet, and being explicit about what was verified.

What's inside a file

Each quantized Linear stores five pieces:

Tensor	Purpose
`weight`	int4 codes, two per byte
`weight_s_rel`	fp8 scale, one per group of 16
`weight_s_channel`	one scale per output channel
`weight_codebook`	16 Lloyd-Max levels, fit to the tensor
`comfy_quant`	layout config the loader reads

Three ideas stacked: a ConvRot Hadamard rotation that flattens outliers so four bits go further, a codebook of non-uniform levels fit to the actual weight distribution instead of an even grid, and per-group fp8 scales preserving local dynamic range. Calibration-free — no activation dataset, so nothing in the conversion biases the model toward one kind of prompt.

What I do differently

Sensitive layers are never quantized. Timestep embeddings, conditioning projections, patch projections, final output layers and rotary tables stay high precision. On a few-step model the timestep embedder has only a handful of sigma values to distinguish — crushing it to four bits corrupts every step of the schedule. Every file is checked after conversion to confirm those layers really are stored at F16/F32, because quantizers do not preserve them automatically.

Mixed formats where the kernel demands it. The fused W4A8 kernel accepts a ConvRot group of exactly 256, so any layer whose input dimension isn't divisible by 256 cannot use it. Rather than silently shipping a file that errors on load, those layers are written as int8_tensorwise — also native, no group constraint, ~1% error. Each model card states which layers took that path.

Every file is measured. Conversion reports per-layer reconstruction error against the original bf16 weights. Anything that doesn't land where it should doesn't get uploaded.

Requirements

ComfyUI 0.30.0+ with asym_w4a8_int8 in its native quant registry
comfy-kitchen installed (ships the kernels)
An Ampere or newer NVIDIA GPU (int8 tensor cores, sm_80+)

On startup ComfyUI prints its available formats. You want asym_w4a8_int8 in the Native ops list — under emulated it still runs, without the int8 speed advantage.

Usage

Drop the .safetensors in ComfyUI/models/diffusion_models
Load it with Load Diffusion Model — the stock node, no custom loader
Text encoder, VAE and sampler settings are unchanged from the base model

Per-model notes (step counts, CFG, resolution) live in each model's card. Distilled models have fixed schedules that must be respected — base-model settings on them produce poor results regardless of quantization.

Models

Collection in progress. Each conversion has its own repo with exact sizes, measured error, and the list of layers kept at high precision.

Licensing

These are quantized derivatives. Every original license and usage restriction carries over unchanged, and each model repo states the license of its base model. Check the specific model's card before commercial use — several bases in this collection are not permissive.

Credits

Kijai — the W4A8 int8-codebook layout and kernels
Comfy-Org / comfyanonymous — comfy-kitchen and the native quantization registry
city96 — ComfyUI-GGUF, which taught most of us how quantized loading works in ComfyUI
Original model authors — all base licenses apply

Quantized by RealRebelAI · GitHub · X

Downloads last month: -