BAGEL-7B-MoT ConvRot (INT8, INT6, INT4 & INT2 QAT) for ComfyUI

Quantized single-file checkpoints of ByteDance-Seed/BAGEL-7B-MoT in four clean bit widths: Full INT8, Full INT6 (w6a8_int8), Full INT4 (asym_w4a8_int8 + Lloyd-Max codebook), and Full INT2 (w2a8_int8 QAT).

All tiers use ConvRot (block Hadamard rotation) across all 392 coupled MoT Qwen2 + diffusion generation linears, while preserving SigLIP vision encoder, connectors, VAE heads, and embeddings in BF16.

  • Full INT8: ConvRot + Act-Order GPTQ.
  • Full INT6 (w6a8_int8) & Full INT4 (asym_w4a8_int8): Split-MoT SmoothQuant + ConvRot + GPTQ Hessian compensation (INT6 adds damped coordinate descent; INT4 adds sequential H_qq GPTQ, 2-pass Lloyd-Max refinement, and CUDA damped coordinate descent).
  • Full INT2 (w2a8_int8 QAT, WIP β€” Not Final Version): Current work-in-progress two-phase Block-AP + Split-MoT End-to-End Flow-Matching QAT checkpoint (7.20 GB, -75.3% vs BF16) with grouped FP8 scales (gs=16), ConvRot-256 dynamic INT8 activations, and 4-entry symmetric Lloyd-Max codebooks. Preserves fluent <think> reasoning, VLM understanding, and image editing (16.95 dB PSNR / 0.89627 cos), while text-to-image fidelity is still being actively improved and this is not the final version.

Includes the updated ComfyUI-BAGEL custom node (ComfyUI-BAGEL/ and ComfyUI-BAGEL-93811c5c5931.tar.gz) with native w6a8_int8, asym_w4a8_int8, and w2a8_int8 loader support.


Available Checkpoints

Precision Tier File (ComfyUI/models/bagel/) Size (vs 29.21 GB BF16) Quantized MoT Linears
Full INT8 ConvRot bagel-7b-mot-int8-convrot.safetensors 16.18 GB (-44.6%) 392 / 392 MoT INT8 (Act-Order GPTQ)
Full INT6 ConvRot bagel-7b-mot-int6-convrot.safetensors 13.73 GB (-53.0%) 392 / 392 MoT INT6 (w6a8_int8 Split-MoT SmoothQuant + CD)
Full INT4 ConvRot bagel-7b-mot-int4-convrot.safetensors 10.47 GB (-64.2%) 392 / 392 MoT INT4 (asym_w4a8_int8 Split-MoT SmoothQuant + Sequential GPTQ + CUDA CD)
Full INT2 ConvRot QAT (WIP) bagel-7b-mot-int2-convrot.safetensors 7.20 GB (-75.3%) 392 / 392 MoT INT2 (w2a8_int8 Block-AP + Split-MoT E2E QAT)

All files load directly via ComfyUI-BAGEL using native comfy-kitchen & Triton kernels (ComfyUI >= v0.45.0).


Visual Comparisons (1024x1024, 50 steps, cfg_text_scale=4.0, timestep_shift=3.0)

Text-to-Image

T2I Cabin 5-way T2I Pocket Watch 5-way

Text-to-Image with <think> Chain-of-Thought

T2I Think Robot 5-way

Image Editing (Using Matched BF16 Input Image)

Image Edit Winter Cabin 5-way


Evaluation Metrics (G4 GPU, ComfyUI v0.45.3)

Measured via scripts/eval_bagel_g4.py directly against unquantized BF16 (bagel-7b-mot.safetensors):

Precision Tier Peak VRAM (T2I / Edit) t2i_cabin_s42 (Time / Cos / SSIM / PSNR) t2i_watch_s1337 (Time / Cos / SSIM / PSNR) t2i_think_s42 (Time / Cos / SSIM / PSNR) edit_cabin_winter_s42 (Time / Cos / SSIM / PSNR) understand_cabin Report
BF16 Baseline 30.30 / 31.32 GB 36.60s / ref 31.58s / ref 34.72s / ref 40.48s / ref 1.59s g4_bagel_bf16_stats.json
Full INT8 ConvRot 17.17 / 18.19 GB 28.53s / 0.99687 / 0.96509 / 32.21 dB 25.96s / 0.99041 / 0.95838 / 26.71 dB 29.93s / 0.86022 / 0.65112 / 15.18 dB 33.97s / 0.99611 / 0.95728 / 32.38 dB 1.96s g4_bagel_int8_stats.json
Full INT6 ConvRot 14.74 / 15.76 GB 29.35s / 0.99627 / 0.96011 / 31.38 dB 27.20s / 0.97862 / 0.91515 / 23.32 dB 30.86s / 0.94435 / 0.80724 / 19.53 dB 35.56s / 0.99584 / 0.95983 / 32.46 dB 1.73s g4_bagel_int6_stats.json
Full INT4 ConvRot 11.47 / 12.48 GB 28.71s / 0.98822 / 0.92394 / 28.02 dB 26.78s / 0.95403 / 0.84440 / 20.30 dB 35.53s / 0.71399 / 0.56152 / 12.12 dB 35.08s / 0.99254 / 0.94893 / 28.92 dB 1.71s g4_bagel_int4_stats.json
Full INT2 ConvRot QAT (WIP) 8.24 / 9.25 GB 29.42s / 0.39500 / 0.29581 / 9.32 dB 27.69s / 0.58873 / 0.31820 / 9.96 dB 34.71s / 0.37770 / 0.39292 / 7.95 dB 36.00s / 0.89627 / 0.66833 / 16.95 dB 1.13s g4_bagel_int2_stats.json

ComfyUI Setup & Usage

  1. Install the updated ComfyUI-BAGEL custom node: Extract ComfyUI-BAGEL-93811c5c5931.tar.gz (or copy ComfyUI-BAGEL/) into ComfyUI/custom_nodes/ComfyUI-BAGEL/.
  2. Download INT8, INT6, INT4, or INT2 .safetensors into ComfyUI/models/bagel/.
  3. Ensure BAGEL's VAE (ae.safetensors) is placed in ComfyUI/models/vae/.
  4. Hardware note: Requires an Ampere / Ada / Hopper / Blackwell CUDA GPU with ComfyUI >= v0.45.0 and comfy-kitchen.

License & Attribution

Licensed under Apache License 2.0, matching the upstream ByteDance-Seed/BAGEL-7B-MoT repository and ComfyUI-BAGEL. See NOTICE for full upstream attribution and a summary of modifications made to the weights and custom node.

Downloads last month
126
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for SUSSYEATING/BAGEL-7B-MoT-ConvRot

Base model

Qwen/Qwen2.5-7B
Quantized
(6)
this model