rshoemake commited on
Commit
ec753c4
·
verified ·
1 Parent(s): 83a6290

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -42,6 +42,7 @@ Comparison across all tested arms for this model:
42
  |---|---:|---:|---:|---:|---:|
43
  | fp16 (reference) | 16.0 | 0.9024 | 0.8598 | 0.8647 | 0.7268 |
44
  | Unsloth bnb-4bit (competitor) | 4.6919 | 0.9024 | 0.8720 | 0.8596 | 0.7293 |
 
45
  | naive_rowwise_4bit | 4.7594 | 0.8902 | 0.8415 | 0.8521 | 0.7268 |
46
  | outlier_4bit | 4.7594 | 0.8780 | 0.8293 | 0.8571 | 0.7193 |
47
  | rotation_outlier_4bit | 4.7784 | 0.9024 | 0.8537 | 0.8521 | 0.7168 |
@@ -50,10 +51,10 @@ Comparison across all tested arms for this model:
50
  **Honest caveat**: mixed_allocator_4bit does not top every column in this table —
51
  it matches Unsloth's bnb-4bit exactly on MBPP+Extra (0.7293) and on MBPP Base
52
  (0.8571 vs 0.8596, essentially tied), but trails on HumanEval relative to
53
- rotation_outlier_4bit and Unsloth. Its advantage over the other TAQ strategies
54
- shows more clearly in the fidelity (PPL/KLD) table above than in code-eval
55
- pass@1, which has higher run-to-run variance at n=164 (HumanEval) / n=399 (MBPP)
56
- task counts.
57
 
58
  ## Provenance
59
 
 
42
  |---|---:|---:|---:|---:|---:|
43
  | fp16 (reference) | 16.0 | 0.9024 | 0.8598 | 0.8647 | 0.7268 |
44
  | Unsloth bnb-4bit (competitor) | 4.6919 | 0.9024 | 0.8720 | 0.8596 | 0.7293 |
45
+ | GGUF Q4_K_M (Unsloth, competitor) | 4.8471 | 0.8902 | 0.8293 | 0.8546 | 0.7293 |
46
  | naive_rowwise_4bit | 4.7594 | 0.8902 | 0.8415 | 0.8521 | 0.7268 |
47
  | outlier_4bit | 4.7594 | 0.8780 | 0.8293 | 0.8571 | 0.7193 |
48
  | rotation_outlier_4bit | 4.7784 | 0.9024 | 0.8537 | 0.8521 | 0.7168 |
 
51
  **Honest caveat**: mixed_allocator_4bit does not top every column in this table —
52
  it matches Unsloth's bnb-4bit exactly on MBPP+Extra (0.7293) and on MBPP Base
53
  (0.8571 vs 0.8596, essentially tied), but trails on HumanEval relative to
54
+ rotation_outlier_4bit, Unsloth, and the real Unsloth GGUF Q4_K_M quant. Its
55
+ advantage over the other TAQ strategies shows more clearly in the fidelity
56
+ (PPL/KLD) table above than in code-eval pass@1, which has higher run-to-run
57
+ variance at n=164 (HumanEval) / n=399 (MBPP) task counts.
58
 
59
  ## Provenance
60