Kreuzzelg
/

kat-coder-v2.5-dev-colibri-i4-gs64

qwen3_5_moe_text

qwen3.6-35b-a3b

Mixture of Experts

Model card Files Files and versions

Kreuzzelg commited on 18 days ago

Commit

f2ccba2

·

verified ·

1 Parent(s): 233dd5b

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -19,7 +19,7 @@ library_name: colibri
 a coding model that shares the Qwen3.6-35B-A3B architecture (`qwen3_5_moe`,
 35B total / 3B active, 40 layers, 256 experts top-8, hybrid Gated Attention +
 Gated DeltaNet). Experts are quantized to **group-scaled int4** (`expert_gs=64`,
-one f32 scale per 64 input elements per row, ~22 GB), the higher-accuracy
 variant of colibri's int4 format: on the sibling base model the same pipeline
 cut first-token logit error by **~44 %** vs per-row scaling (see
 [`qwen36-35b-a3b-colibri-i4-gs64`](https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64)).
@@ -55,7 +55,7 @@ separate open release. Reported numbers differ substantially:
 ## Which commits you need
-The qwen36 engine is not merged upstream yet; pick the branch matching your
 setup (all on [github.com/kreuzzelg/colibri](https://github.com/kreuzzelg/colibri)):
 | Setup | Branch / commit | Upstream status |

 a coding model that shares the Qwen3.6-35B-A3B architecture (`qwen3_5_moe`,
 35B total / 3B active, 40 layers, 256 experts top-8, hybrid Gated Attention +
 Gated DeltaNet). Experts are quantized to **group-scaled int4** (`expert_gs=64`,
+one f32 scale per 64 input elements per row, about 22 GB), the higher-accuracy
 variant of colibri's int4 format: on the sibling base model the same pipeline
 cut first-token logit error by **~44 %** vs per-row scaling (see
 [`qwen36-35b-a3b-colibri-i4-gs64`](https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64)).
 ## Which commits you need
+The qwen3.6 engine is not merged upstream yet; pick the branch matching your
 setup (all on [github.com/kreuzzelg/colibri](https://github.com/kreuzzelg/colibri)):
 | Setup | Branch / commit | Upstream status |