experiment #19706 (Q6_K interleave): no measurable gain, not merged — data recorded
Browse files
README.md
CHANGED
|
@@ -258,6 +258,7 @@ their data and are **not** merged (❌ 沒幫助).
|
|
| 258 |
|---|---|---:|---:|---:|
|
| 259 |
| upstream PR [#27024](https://github.com/ggml-org/llama.cpp/pull/27024) by [@jinzihao](https://github.com/jinzihao) | ggml-cpu/amx: fix the (dead) `block_q8_K` VNNI quantization and enable the VNNI path | ✅ 有幫助 — merged | 57.25 → 59.20 (**+3.4 %**) | 10.94 → 11.24 (**+2.7 %**) |
|
| 260 |
| upstream PR [#22525](https://github.com/ggml-org/llama.cpp/pull/22525) by [@HyeongiJeon](https://github.com/HyeongiJeon) | ggml-cpu: optimize `ggml_gemm_q4_K_8x8_q8_K` interleaving/staging for AVX-512 (and AVX2) | ✅ 有幫助(prefill)— merged | 55.78 → 57.42 (**+2.9 %**) | 10.70 → 10.87 (+1.5 %, neutral) |
|
|
|
|
| 261 |
|
| 262 |
Round-by-round for #27024 (candidate vs baseline, t/s):
|
| 263 |
|
|
@@ -294,6 +295,25 @@ The candidate won 6 of 8 pairwise rounds on prefill (1 loss, 1 tie within
|
|
| 294 |
interleaving/staging of the Q4_K GEMM (the prefill path), so the measured
|
| 295 |
profile matches the expectation. Merged. Measured 2026-10-16.
|
| 296 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 297 |
## License
|
| 298 |
|
| 299 |
This project (the patch set, `patch.sh` and documentation) is licensed
|
|
|
|
| 258 |
|---|---|---:|---:|---:|
|
| 259 |
| upstream PR [#27024](https://github.com/ggml-org/llama.cpp/pull/27024) by [@jinzihao](https://github.com/jinzihao) | ggml-cpu/amx: fix the (dead) `block_q8_K` VNNI quantization and enable the VNNI path | ✅ 有幫助 — merged | 57.25 → 59.20 (**+3.4 %**) | 10.94 → 11.24 (**+2.7 %**) |
|
| 260 |
| upstream PR [#22525](https://github.com/ggml-org/llama.cpp/pull/22525) by [@HyeongiJeon](https://github.com/HyeongiJeon) | ggml-cpu: optimize `ggml_gemm_q4_K_8x8_q8_K` interleaving/staging for AVX-512 (and AVX2) | ✅ 有幫助(prefill)— merged | 55.78 → 57.42 (**+2.9 %**) | 10.70 → 10.87 (+1.5 %, neutral) |
|
| 261 |
+
| upstream PR [#19706](https://github.com/ggml-org/llama.cpp/pull/19706) by [@Manogna-Sree](https://github.com/Manogna-Sree) | Q6_K block interleaving for x86 SIMD (AVX-512/AVX2) | ❌ 沒幫助 — not merged | 58.16 → 58.34 (+0.3 %) | 11.06 → 11.07 (+0.1 %) |
|
| 262 |
|
| 263 |
Round-by-round for #27024 (candidate vs baseline, t/s):
|
| 264 |
|
|
|
|
| 295 |
interleaving/staging of the Q4_K GEMM (the prefill path), so the measured
|
| 296 |
profile matches the expectation. Merged. Measured 2026-10-16.
|
| 297 |
|
| 298 |
+
Round-by-round for #19706 (candidate vs baseline, t/s), default settings:
|
| 299 |
+
|
| 300 |
+
| round | pp512 | tg128 |
|
| 301 |
+
|---|---:|---:|
|
| 302 |
+
| 1 | 58.42 vs 53.67 | 11.67 vs 11.45 |
|
| 303 |
+
| 2 | 59.46 vs 58.68 | 12.04 vs 10.84 |
|
| 304 |
+
| 3 | 54.57 vs 60.04 | 10.54 vs 11.82 |
|
| 305 |
+
| 4 | 62.32 vs 57.06 | 10.95 vs 11.16 |
|
| 306 |
+
| 5 | 57.66 vs 58.59 | 11.10 vs 10.86 |
|
| 307 |
+
| 6 | 57.26 vs 57.92 | 11.05 vs 11.17 |
|
| 308 |
+
| 7 | 59.71 vs 56.95 | 10.98 vs 10.96 |
|
| 309 |
+
| 8 | 58.25 vs 58.41 | 11.11 vs 10.83 |
|
| 310 |
+
|
| 311 |
+
**Verdict ❌ 沒幫助 (no measurable gain).** Medians differ by +0.3 %
|
| 312 |
+
(prefill) and +0.1 % (decode), with a 4 W / 3 L / 1 T split on both — pure
|
| 313 |
+
noise. The base `Q3_K_M` model contains only **one** single `Q6_K` tensor,
|
| 314 |
+
so a faster Q6_K kernel has almost no weight to act on. The PR is *not*
|
| 315 |
+
merged; the data is recorded here for the record. Measured 2026-10-16.
|
| 316 |
+
|
| 317 |
## License
|
| 318 |
|
| 319 |
This project (the patch set, `patch.sh` and documentation) is licensed
|