EXL3 models
Collection
50 items • Updated
• 40
EXL3 quants of Qwen3.5-35B-A3B
⚠️ Requires ExLlamaV3 v0.0.23 (or v0.0.22 dev branch)
Base bitrates:
2.00 bits per weight
3.00 bits per weight
4.00 bits per weight
5.00 bits per weight
6.00 bits per weight
Optimized:
2.13 bits per weight
2.54 bits per weight
3.09 bits per weight
3.54 bits per weight
4.09 bits per weight
Benchmarks
SVG Catbench
2.00 bpw
|
2.13 bpw
|
2.54 bpw
|
3.00 bpw
|
3.09 bpw
|
3.54 bpw
|
4.00 bpw
|
4.09 bpw
|
5.00 bpw
|
6.00 bpw
|
FP16
|
(more coming soon)
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support