Hugging Face's logo

ussoewwin
/
ControlNet-models-ConvRot-INT8

Image-to-Image
controlnet
text-to-image
inpainting
sdxl
stable-diffusion-xl
illustrious-xl
z-image
z-image-turbo
qwen-image
qwen-image-2512
flux
flux.1-dev
diffsynth
diffsynth-studio
quantized
int8
convrot
comfyui
Model card Files
xet
 Community

ControlNet Models (ConvRot INT8)

High-fidelity ConvRot INT8 quantized weights for multi-condition and dedicated ControlNet models across diverse generative architectures (Illustrious-XL / SDXL, SDXL 1.0, Z-Image-Turbo, Qwen-Image / Qwen-Image-2512, and FLUX.1-dev).

⚠️ Where each pack goes, and how it loads

This repository mixes two runtime object types. They use different loaders, and getting this wrong means the file silently fails to apply.

group the runtime object the loader the directory
ControlNet (SDXL anytest A/B, SDXL union-pro-max, Qwen-Image-2512 Union 2602, Qwen-Image Inpainting, FLUX.1-dev Union Pro 2.0) a ControlNet ControlNetLoader (stock) or the HSWQ ControlNet loader ComfyUI/models/controlnet/
Model patch (Z-Image-Turbo Fun Controlnet Union/Tile, Qwen-Image DiffSynth canny/depth/inpaint) a model patch, despite the "ControlNet" name HSWQ Model Patch Loader (HSWQModelPatchLoaderCustom). The stock ModelPatchLoader cannot read the INT8 layout ComfyUI/models/model_patches/

The model patches are applied by the dedicated apply nodes (ZImageFunControlnet, QwenImageDiffsynthControlnet), not by Apply ControlNet.

🌟 Model Overview

This repository hosts high-quality ConvRot INT8 quantized weights for all-in-one ControlNet Union and dedicated condition-specific ControlNet models. Orthogonal Hadamard rotation is applied to each 2D weight before per-channel INT8 quantization, which suppresses activation outlier distortion and cuts the disk footprint roughly in half while preserving structural control fidelity:

  • CN-anytest4_illustrious2 (Variants A & B): Multi-purpose all-in-one Anytest v4 ControlNet models fine-tuned for Illustrious-XL / SDXL by 2vXpSwA7, offering high-precision anime and illustration structure guiding.
  • controlnet-union-pro-max-sdxl-1.0: The comprehensive all-in-one ControlNet Union model for SDXL 1.0 by xinsir, supporting 10+ control conditions in a single compact file.
  • Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps: Multi-condition ControlNet Union model (lite architecture, 8-step distilled) for the Z-Image-Turbo architecture by alibaba-pai, providing lightweight and fast structure control (Canny, Depth, Pose, Color, etc.). Model patch.
  • Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps: Dedicated high-resolution Tile and upscaling ControlNet model (lite architecture, 8-step distilled) for the Z-Image-Turbo architecture by alibaba-pai, optimized for super-resolution detail enhancement. Model patch.
  • Qwen-Image-2512-Fun-Controlnet-Union-2602: Multi-condition ControlNet Union model (5 layer blocks) for the Qwen-Image-2512 architecture.
  • Qwen-Image-ControlNet-Inpainting: Dedicated inpainting and editing ControlNet model for the Qwen-Image architecture.
  • Qwen-Image DiffSynth ControlNet Suite (canny / depth / inpaint): Dedicated single-condition ControlNet models for the Qwen-Image architecture developed by DiffSynth-Studio. Model patches.
  • FLUX.1-dev-ControlNet-Union-Pro-2.0: Next-generation unified 7-in-1 ControlNet for the FLUX.1-dev architecture by Shakker Labs.

📦 Available Models

Sizes are decimal (1 GB = 10^9 bytes) and exact; the SHA256 column is the pack's own hash, so a download can be verified against it.

Filename Base Architecture Base Model / Author Supported Conditions Quantization File Size SHA256 License
CN-anytest4_illustrious2_A_hswq_convrot_int8.safetensors Illustrious-XL / SDXL 2vXpSwA7/iroiro-lora (Anytest v4 Variant A) Multi-condition (Canny, Lineart, Depth, Pose, Structure) ConvRot INT8 1425.1 MB be9ff80b2f4895684c7114afff12220394f838179b5c82eadd8ebc46f5d82c25 Fair AI / OpenRAIL++-M
CN-anytest4_illustrious2_B_hswq_convrot_int8.safetensors Illustrious-XL / SDXL 2vXpSwA7/iroiro-lora (Anytest v4 Variant B) Multi-condition (Canny, Lineart, Depth, Pose, Structure) ConvRot INT8 1423.5 MB 7e34dc3cc3334afa2b9fe5461131a1f61e289a83b1a11413d6c687e94b9dde6e Fair AI / OpenRAIL++-M
controlnet-union-pro-max-sdxl-1.0_hswq_convrot_int8.safetensors SDXL 1.0 xinsir/controlnet-union-sdxl-1.0 OpenPose, Depth, Canny, Lineart, Anime Lineart, Scribble, Soft Edge, Normal, Segment, Tile, Inpaint ConvRot INT8 1435.3 MB dcdf6f2037205c4a40b1f8863a91827ca3a5b55ec3261e07bb13d2dc855b47fb OpenRAIL++-M / Apache-2.0
FLUX.1-dev-ControlNet-Union-Pro-2.0_hswq_convrot_int8.safetensors FLUX.1-dev Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro-2.0 Canny, Depth, Pose, Blur, Gray, Soft Edge, Low Quality ConvRot INT8 2710.6 MB fbcba3c76a221fba461c83680f682d3ff7aa813ef554c42e3a5449b981090c46 Other / Non-Commercial
Qwen-Image-2512-Fun-Controlnet-Union-2602_hswq_convrot_int8.safetensors Qwen-Image-2512 alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union Canny, HED, Depth, Pose, MLSD, Scribble, Gray, Inpaint ConvRot INT8 1758.7 MB 5aec39a53f583bade3b7db18f38ed5a01905a0610c2ad2f9f792802178596bc4 Apache-2.0
Qwen-Image-ControlNet-Inpainting_hswq_convrot_int8.safetensors Qwen-Image alibaba-pai/Qwen-Image-ControlNet-Inpainting Inpainting, Image Editing ConvRot INT8 2120.3 MB 38f0ab7495029517d4b0bb4187cedcbdfa090a29f0bb50dfa5733f5076a0c451 Apache-2.0
qwen_image_canny_diffsynth_controlnet_hswq_convrot_int8.safetensors Qwen-Image modelscope/DiffSynth-Studio Canny Edge Detection & Structural Guidance ConvRot INT8 1135.9 MB c2899255252b81fd64ef2fae0fbfd86e1496ba04cf10cc95fe45679120f1c996 Apache-2.0
qwen_image_depth_diffsynth_controlnet_hswq_convrot_int8.safetensors Qwen-Image modelscope/DiffSynth-Studio Geometric Depth Map Guidance ConvRot INT8 1135.9 MB 7b82a9360169fa99524c03bcf913ce63713930174ad1fc332e553e5646bad44c Apache-2.0
qwen_image_inpaint_diffsynth_controlnet_hswq_convrot_int8.safetensors Qwen-Image modelscope/DiffSynth-Studio Masked Inpainting & Localized Editing ConvRot INT8 1135.9 MB 471b6affc7fcedafb9940fafd6354dd6eb5fac5705e647029c90598c407c82d3 Apache-2.0
Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors Z-Image-Turbo alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1 Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) ConvRot INT8 1009.7 MB 1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4 Apache-2.0
Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors Z-Image-Turbo alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1 Tile, Super-Resolution, Detail Enhancing ConvRot INT8 1009.7 MB 2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8 Apache-2.0

📐 Measured fidelity

Each pack's protection set was chosen by a measured aggregate contribution on the model's own output, over 8 seeds, and the acceptance was measured separately on a fixed 256-item set (seed 42) through the same loader the runtime uses. "Gate" is cos.mean >= 0.99 and cos.min >= 0.98.

Pack the score (mean / min) the gate the note
CN-anytest4_illustrious2_A 0.999626 (forward, relMSE 1.11e-03) PASS trajectory-impact selection, 36 protected
CN-anytest4_illustrious2_B 0.992958 / 0.948279 (trajectory) PASS trajectory gate, 35 protected
controlnet-union-pro-max-sdxl-1.0 0.999961 PASS curve flat, smallest satisfying pack taken (41 protected)
Qwen-Image-2512-Fun-Controlnet-Union-2602 0.999158 / 0.998215 PASS floor case: all 77 layers INT8, no protection
Qwen-Image-ControlNet-Inpainting 0.999545 / 0.998887 PASS floor case: all 95 layers INT8, no protection
qwen_image_canny_diffsynth 0.9999924905 / 0.9999924096 PASS one protected layer (img_in), 598x above the runner-up
qwen_image_depth_diffsynth 0.9999924109 / 0.9999923262 PASS one protected layer (img_in), 376x above the runner-up
qwen_image_inpaint_diffsynth 0.9999921236 / 0.9999919542 PASS one protected layer (img_in), 147x above the runner-up
Z-Image-Turbo-Fun-Controlnet-Union 0.999862 / 0.999491 (trajectory) PASS all 38 layers INT8, no protection
Z-Image-Turbo-Fun-Controlnet-Tile 0.999956 / 0.999813 (trajectory) PASS all 38 layers INT8, no protection
FLUX.1-dev-ControlNet-Union-Pro-2.0 0.999771 / 0.999488 PASS 12 protected; the all-INT8 state measured 0.999622 / 0.999020

The per-checkpoint reasoning and the full measured curves are in the HSWQ repository's md/ guides.

🛠️ Key Features

  • All-in-One & Dedicated Condition Control: Unified multi-condition architectures alongside specialized single-condition models (Canny, Depth, Tile, Inpainting) for fine-grained workflow flexibility.
  • ConvRot INT8 Precision: Orthogonal Hadamard rotations redistribute channel-wise outlier spikes across the input dimension before per-channel INT8 quantization, preventing quantization error buildup in deep control layers.
  • Measured, not assumed, protection: Where a layer's measured contribution is large it stays FP16 (kept tensors are stored FP16, not FP32). Several packs here need no protection at all; the measured curves say so.
  • Footprint: roughly half the unquantized checkpoint, with the INT8 execution path (Tensor Core INT8 matmul) as the additional runtime gain.

🚀 Usage in ComfyUI

Install the loader extension:

cd ComfyUI/custom_nodes
git clone https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools.git

Then place each file in the directory its type requires (see the table at the top):

  • ControlNet files go in ComfyUI/models/controlnet/ and load with ControlNetLoader.
  • Model patch files go in ComfyUI/models/model_patches/ and load with HSWQ Model Patch Loader (the stock ModelPatchLoader reads the INT8 tensors as plain weights), then apply with ZImageFunControlnet / QwenImageDiffsynthControlnet.

📜 Credits & License

Base Models & Research

Disclaimer: These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image-to-Image
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support