Text Encoder & CLIP Models (ConvRot INT8)

High-fidelity Native ConvRot INT8 quantized weights for primary Text Encoders and Vision-Language CLIP models across state-of-the-art generative diffusion architectures (SDXL, FLUX.1, SD 1.5 / SD 2.1, SD3 / SD3.5, and Next-Gen Diffusion pipelines).

🌟 Model Overview

This repository hosts high-quality Native ConvRot INT8 quantized weights for essential text encoders and CLIP vision-language backbones. By applying orthogonal Hadamard rotation ($W_{rot} = W \cdot H^T$) prior to per-channel symmetric INT8 quantization, activation outlier spikes in transformer feed-forward and projection layers are redistributed uniformly across channels. This eliminates quantization-induced semantic drift and cuts VRAM and storage footprint by ~50% while preserving precise prompt conditioning fidelity:

CLIP-SAE-ViT-L-14-FP32: Primary ViT-L/14 text encoder with high-precision FP32/SAE baseline representation, quantized to native ConvRot INT8 for ultra-fast, lightweight prompt encoding across SD 1.5, SDXL, and FLUX workflows.
CLIP-ViT-bigG-14-laion2B-39B-b160k: The massive OpenCLIP ViT-bigG/14 text encoder (laion2B) for SDXL and multi-encoder generative pipelines, reducing disk and VRAM demands from 3.69 GB down to 1.85 GB.
flan_t5_xxl: The 11B-parameter dense Flan-T5 XXL text encoder for next-generation text-to-image and multimodal architectures, compressed from ~22.4 GB down to ~11.28 GB to allow inference on consumer GPUs.

📦 Available Models

Filename	Base Architecture	Base Model / Upstream	Quantization	File Size	Baseline Size	VRAM Savings
`CLIP-SAE-ViT-L-14-FP32_native_convrot_int8.safetensors`	OpenAI CLIP ViT-L/14	openai/clip-vit-large-patch14	ConvRot INT8 (`int8_tensorwise`)	~0.43 GB (412 MB)	~0.93 GB	~54% Reduction
`CLIP-ViT-bigG-14-laion2B-39B-b160k_convrot_int8.safetensors`	OpenCLIP ViT-bigG/14	laion/CLIP-ViT-bigG-14-laion2B-39B-b160k	ConvRot INT8 (`int8_tensorwise`)	~1.85 GB (1,765 MB)	~3.69 GB	~50% Reduction
`flan_t5_xxl_convrot_int8.safetensors`	Google Flan-T5 XXL (11B)	google/flan-t5-xxl	ConvRot INT8 (`int8_tensorwise`)	~11.28 GB (10.50 GB)	~22.40 GB	~50% Reduction

🛠️ Key Features

Orthogonal Hadamard Pre-Rotation (ConvRot): Weight matrices are pre-rotated group-wise along the input feature dimension ($W_{rot} = W \cdot H^T$) using power-of-4 normalized Hadamard blocks (groupsize=256), eliminating catastrophic dynamic range compression caused by outlier channels.
Native ComfyUI Compatibility: Encodes standard comfy_quant header metadata ({"format": "int8_tensorwise", "convrot": true, "convrot_groupsize": 256}) and per-channel scale tensors, loading automatically in ComfyUI with standard CLIPLoader, DualCLIPLoader, or TripleCLIPLoader without custom nodes.
Full Precision Preservation for Critical Weights: Non-2D weights (token embeddings, position embeddings, layer normalizations, biases) are maintained in native floating-point precision to guarantee prompt parsing integrity and boundary consistency.
Hardware Acceleration: Fully compatible with Tensor Core INT8 matrix multiplication across modern GPUs (NVIDIA Turing, Ampere, Ada Lovelace, Blackwell).

🚀 Usage in ComfyUI

Model Placement

Download the desired .safetensors files and place them into your ComfyUI text encoder directory:

ComfyUI/models/clip/

Loading

Load directly via standard ComfyUI CLIP loader nodes:

Single CLIP: Load CLIP / CLIPLoader $\rightarrow$ select <model_name>_convrot_int8.safetensors.
Dual CLIP (SDXL): DualCLIPLoader $\rightarrow$ set clip_name1 to clip_l (or CLIP-SAE-ViT-L-14-FP32_native_convrot_int8.safetensors) and clip_name2 to CLIP-ViT-bigG-14-laion2B-39B-b160k_convrot_int8.safetensors (type sdxl).
Triple CLIP (SD3 / FLUX): TripleCLIPLoader $\rightarrow$ assign CLIP-L, CLIP-G, and T5-XXL ConvRot INT8 weights accordingly.

ComfyUI automatically recognizes the comfy_quant stamp and dispatches optimized INT8 execution kernels seamlessly.

📜 Credits & License

OpenAI CLIP ViT-L/14: openai/clip-vit-large-patch14 (MIT License)
OpenCLIP ViT-bigG/14: LAION / OpenCLIP (MIT / Apache-2.0 License)
Google Flan-T5 XXL: google/flan-t5-xxl (Apache-2.0 License)
Quantization Pipeline: HSWQ / ConvRot INT8 Suite
Base Licenses: Please adhere to the individual upstream licenses of the respective foundation models.

Downloads last month: -; Downloads are not tracked for this model. How to track