Colibro/Qwen3.6-35B-INT4-MTP8-MM-F16

Qwen3.6-35B Colibro INT4 MTP8 MM-F16

Colibro-format INT4 sparse-MoE checkpoint converted from Qwen/Qwen3.6-35B-A3B.

Intended for the Colibro / qwen.exe runtime (OpenAI- and Anthropic-compatible APIs, web UI, CUDA expert tiering). This is not a drop-in replacement for vanilla Hugging Face Transformers inference.

Default API / UI model id when serving with Colibro:

qwen3.6-35B-COLIBRO-INT4-MTP8-MM-F16

License

Apache 2.0 (inherited from the base Qwen3.6-35B-A3B weights). See the base model LICENSE.

Format

• Layout: Colibro MoE shards named colibro-NNNNN.safetensors (not model-*.safetensors)
• Size: ~18.4 GB total (26 weight shards + tokenizer / config sidecars)
• Architecture: Qwen3.6 sparse MoE — 40 layers, 256 experts, 8 experts/token, hidden size 2048

Precision policy

Component	Precision
Dense / routed expert matrices	INT4
MTP matrices (native Qwen3.6 head)	INT8
Embedding / output matrices	INT8
Vision tower	FP16 (colibro_vision_dtype: f16)
Norms, routing, other small tensors	FP32

Conversion retained MTP (776 keys) and vision (333 keys) from the source tree.

Key features

• INT4 MoE main trunk for compact disk / RAM footprint
• Native MTP speculative drafting (enable with COLI_MTP=1; drafts via COLI_MTP_DRAFTS=1..4 on the single-request path)
• Multimodal image input (JPEG / PNG / WebP data URIs) via the resident FP16 vision tower
• Sidecars: config.json, tokenizer files, chat_template.jinja.txt, experts.json (Atlas), preprocessor configs

Files (ballpark)

Pattern	Notes
colibro-00000.safetensors … colibro-00025.safetensors	Weight shards (~18.4 GB combined)
config.json	Model + vision config (mtp_num_hidden_layers, vision_config, …)
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt	Tokenizer
chat_template.jinja.txt	Chat template
experts.json	Expert layout / Atlas asset
preprocessor_config.json, video_preprocessor_config.json	Vision preprocessing
generation_config.json	Sampling defaults

How to run (Colibro)

Clone / build Colibro, then point the Qwen runtime at a local copy of this repo. Project docs: README-QWEN.md and QUICKSTART-QWEN.md in the Colibro tree.

Download

huggingface-cli download Colibro/Qwen3.6-35B-INT4-MTP8-MM-F16 `
  --local-dir models\Qwen3.6-35B-A3B-colibro-int4-mtp8-mm-f16

Doctor / validate

python c\coli doctor `
  --runtime qwen `
  --model models\Qwen3.6-35B-A3B-colibro-int4-mtp8-mm-f16 `
  --auto-tier

Web UI + API

python c\coli web `
  --runtime qwen `
  --model models\Qwen3.6-35B-A3B-colibro-int4-mtp8-mm-f16 `
  --auto-tier `
  --gpu auto `
  --kv-slots 4 `
  --port 8000

Or on Windows: .\run_qwen_backend.bat (default model id qwen3.6-35B-COLIBRO-INT4-MTP8-MM-F16).

Optional MTP

$env:COLI_MTP = "1"
$env:COLI_MTP_DRAFTS = "1"

Intended use / limitations

• Use with Colibro (coli run / coli web / coli serve / qwen.exe). Loading these shards in stock Transformers AutoModel is unsupported.
• Multimodal requests use data-URI images; remote http(s) image URLs are rejected by the Colibro API.
• Quantization trades some quality vs the original BF16 checkpoint for much smaller storage and faster load under Colibro’s tiered expert placement.

Attribution

Converted from Qwen/Qwen3.6-35B-A3B (Alibaba Qwen team) using Colibro’s convert_qwen_moe.py / initialize.py pipeline (dense INT4, experts INT4, MTP INT8, I/O INT8, vision FP16).

Please cite and respect the license of the base model when redistributing or publishing derivatives.