Update README.md
Browse files
README.md
CHANGED
|
@@ -10,10 +10,15 @@ tags:
|
|
| 10 |
- exl3
|
| 11 |
---
|
| 12 |
|
| 13 |
-
[
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
[
|
| 18 |
-
[
|
| 19 |
-
[
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
- exl3
|
| 11 |
---
|
| 12 |
|
| 13 |
+
Quantization was performed using [exllama3 v0.0.22](https://github.com/turboderp-org/exllamav3).
|
| 14 |
+
|
| 15 |
+
| Quant | Size (GB) | KL-div (quant, orig) | KL-div (orig, quant) | Perplexity | Top-K K=1 | Top-K K=2 | Top-K K=3 | Top-K K=4 | Top-K K=5 |
|
| 16 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 17 |
+
| [2.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/2.0bpw) | 55 | 0.36735150 | 0.42469226 | 9.46492433 | 0.7699 | 0.4340 | 0.2006 | 0.0796 | 0.0289 |
|
| 18 |
+
| [3.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/3.0bpw) | 82 | 0.14842009 | 0.15566614 | 8.74921130 | 0.8640 | 0.6125 | 0.3773 | 0.2072 | 0.1040 |
|
| 19 |
+
| [4.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/4.0bpw) | 108 | 0.07256054 | 0.07650418 | 8.43832064 | 0.9118 | 0.7281 | 0.5222 | 0.3439 | 0.2105 |
|
| 20 |
+
| [5.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/5.0bpw) | 135 | 0.04801990 | 0.04921814 | 8.35222293 | 0.9344 | 0.7901 | 0.6154 | 0.4472 | 0.3056 |
|
| 21 |
+
| [6.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/6.0bpw) | 161 | 0.04015230 | 0.04071388 | 8.35782554 | 0.9449 | 0.8209 | 0.6651 | 0.5071 | 0.3670 |
|
| 22 |
+
| [7.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/7.0bpw) | 188 | 0.03484128 | 0.03757493 | 8.35427106 | 0.9510 | 0.8380 | 0.6922 | 0.5420 | 0.4046 |
|
| 23 |
+
| [8.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.5-exl3/tree/8.0bpw) | 214 | 0.03227931 | 0.03371121 | 8.33833098 | 0.9533 | 0.8440 | 0.7042 | 0.5587 | 0.4226 |
|
| 24 |
+
| original | 214 | - | - | 8.34981264 | - | - | - | - | - |
|