File size: 1,041 Bytes
8240279 ea2efc5 789a087 3ee84d7 868d323 789a087 9e0361f 868d323 9e0361f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | ---
license: apache-2.0
base_model: stepfun-ai/Step-3.5-Flash
base_model_relation: quantized
quantized_by: turboderp
tags:
- exl3
---
EXL3 quants of [Step-3.5-Flash](https://huggingface.co/stepfun-ai/Step-3.5-Flash)
⚠️ Requires ExLlamaV3 v0.0.23 (or v0.0.22 `dev` branch)
Base bitrates:
[2.00 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/2.00bpw)
[3.00 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/3.00bpw)
[4.00 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/4.00bpw)
Optimized:
[2.08 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/2.08bpw)
[3.05 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/3.05bpw)
*(more coming soon)*
. | Ppl¹ | KL-div
---------|--------|---------
2.00 bpw | 2.629 | 0.653
2.08 bpw | 2.154 | 0.466
3.00 bpw | 1.521 | 0.142
3.05 bpw | 1.478 | 0.118
4.00 bpw | 1.379 | 0.053
Original | 1.336 |
¹ (10 rows of wikitext2) |