Stick9190/qwen3vl_4b_int8_convrot

Qwen3-VL 4B - INT8 ConvRot

INT8 ConvRot quantization of Qwen3-VL 4B for ComfyUI text-encoder and Krea 2 conditioning use. The 8.88 GB BF16 checkpoint is reduced to approximately 5.37 GB.

Conversion

• Tool: silveroxides/convert_to_quant
• Format: INT8 row-wise with embedded ConvRot metadata
• ConvRot group size: 256
• Method: learned rounding (AdaRound) with low-memory streaming conversion
• Quantized: language-transformer blocks 3-34 and vision-transformer blocks 1-22 (312 matrices; ~3.51B parameters, or 79.1% of matrix parameters)

The token embedding, language blocks 0-2 and 35, vision boundary blocks, vision embeddings, and vision merger components remain BF16.

Layer selection

Qwen3-VL injects three DeepStack vision representations into language blocks 0-2, so those multimodal integration blocks remain BF16. Language block 35 is also retained as the final block for general Qwen3-VL use.

Vision blocks 0 and 23 are retained because they form the input and output boundaries of the 24-block vision tower; the final block feeds directly into the vision mergers. Keeping these boundary layers in BF16 is a conservative quality-preservation heuristic rather than a model-author-specified ConvRot requirement.

Command

ctq -i <input-model>.safetensors -o <output-model>.safetensors `
  --int8 --scaling_mode row `
  --convrot --convrot-group-size 256 `
  --comfy_quant --save-quant-metadata `
  --low-memory --device cuda `
  --exclude-layers '(^model\.language_model\.embed_tokens\.weight$|^model\.language_model\.layers\.(0|1|2|35)\.|^model\.visual\.blocks\.(0|23)\.|^model\.visual\.(pos_embed|patch_embed|merger|deepstack_merger_list)(\.|$))' `
  --verbose NORMAL

Quantization is lossy, so outputs are not bit-identical to the original BF16 checkpoint.