| | --- |
| | license: apache-2.0 |
| | base_model: stepfun-ai/Step-3.5-Flash |
| | base_model_relation: quantized |
| | quantized_by: turboderp |
| | tags: |
| | - exl3 |
| | --- |
| | |
| | EXL3 quants of [Step-3.5-Flash](https://huggingface.co/stepfun-ai/Step-3.5-Flash) |
| |
|
| | ⚠️ Requires ExLlamaV3 v0.0.23 (or v0.0.22 `dev` branch) |
| |
|
| | Base bitrates: |
| |
|
| | [2.00 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/2.00bpw) |
| | [3.00 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/3.00bpw) |
| | [4.00 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/4.00bpw) |
| |
|
| | Optimized: |
| |
|
| | [2.08 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/2.08bpw) |
| | [3.05 bits per weight](https://huggingface.co/turboderp/Step-3.5-Flash-exl3/tree/3.05bpw) |
| | *(more coming soon)* |
| |
|
| |
|
| | . | Ppl¹ | KL-div |
| | ---------|--------|--------- |
| | 2.00 bpw | 2.629 | 0.653 |
| | 2.08 bpw | 2.154 | 0.466 |
| | 3.00 bpw | 1.521 | 0.142 |
| | 3.05 bpw | 1.478 | 0.118 |
| | 4.00 bpw | 1.379 | 0.053 |
| | Original | 1.336 | |
| |
|
| | ¹ (10 rows of wikitext2) |
