Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -42,6 +42,7 @@ Comparison across all tested arms for this model:
|
|
| 42 |
|---|---:|---:|---:|---:|---:|
|
| 43 |
| fp16 (reference) | 16.0 | 0.9024 | 0.8598 | 0.8647 | 0.7268 |
|
| 44 |
| Unsloth bnb-4bit (competitor) | 4.6919 | 0.9024 | 0.8720 | 0.8596 | 0.7293 |
|
|
|
|
| 45 |
| naive_rowwise_4bit | 4.7594 | 0.8902 | 0.8415 | 0.8521 | 0.7268 |
|
| 46 |
| outlier_4bit | 4.7594 | 0.8780 | 0.8293 | 0.8571 | 0.7193 |
|
| 47 |
| rotation_outlier_4bit | 4.7784 | 0.9024 | 0.8537 | 0.8521 | 0.7168 |
|
|
@@ -50,10 +51,10 @@ Comparison across all tested arms for this model:
|
|
| 50 |
**Honest caveat**: mixed_allocator_4bit does not top every column in this table —
|
| 51 |
it matches Unsloth's bnb-4bit exactly on MBPP+Extra (0.7293) and on MBPP Base
|
| 52 |
(0.8571 vs 0.8596, essentially tied), but trails on HumanEval relative to
|
| 53 |
-
rotation_outlier_4bit
|
| 54 |
-
|
| 55 |
-
pass@1, which has higher run-to-run
|
| 56 |
-
task counts.
|
| 57 |
|
| 58 |
## Provenance
|
| 59 |
|
|
|
|
| 42 |
|---|---:|---:|---:|---:|---:|
|
| 43 |
| fp16 (reference) | 16.0 | 0.9024 | 0.8598 | 0.8647 | 0.7268 |
|
| 44 |
| Unsloth bnb-4bit (competitor) | 4.6919 | 0.9024 | 0.8720 | 0.8596 | 0.7293 |
|
| 45 |
+
| GGUF Q4_K_M (Unsloth, competitor) | 4.8471 | 0.8902 | 0.8293 | 0.8546 | 0.7293 |
|
| 46 |
| naive_rowwise_4bit | 4.7594 | 0.8902 | 0.8415 | 0.8521 | 0.7268 |
|
| 47 |
| outlier_4bit | 4.7594 | 0.8780 | 0.8293 | 0.8571 | 0.7193 |
|
| 48 |
| rotation_outlier_4bit | 4.7784 | 0.9024 | 0.8537 | 0.8521 | 0.7168 |
|
|
|
|
| 51 |
**Honest caveat**: mixed_allocator_4bit does not top every column in this table —
|
| 52 |
it matches Unsloth's bnb-4bit exactly on MBPP+Extra (0.7293) and on MBPP Base
|
| 53 |
(0.8571 vs 0.8596, essentially tied), but trails on HumanEval relative to
|
| 54 |
+
rotation_outlier_4bit, Unsloth, and the real Unsloth GGUF Q4_K_M quant. Its
|
| 55 |
+
advantage over the other TAQ strategies shows more clearly in the fidelity
|
| 56 |
+
(PPL/KLD) table above than in code-eval pass@1, which has higher run-to-run
|
| 57 |
+
variance at n=164 (HumanEval) / n=399 (MBPP) task counts.
|
| 58 |
|
| 59 |
## Provenance
|
| 60 |
|