HelloSun commited on
Commit
20c42e1
·
1 Parent(s): 5d3e027

experiment #19706 (Q6_K interleave): no measurable gain, not merged — data recorded

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -258,6 +258,7 @@ their data and are **not** merged (❌ 沒幫助).
258
  |---|---|---:|---:|---:|
259
  | upstream PR [#27024](https://github.com/ggml-org/llama.cpp/pull/27024) by [@jinzihao](https://github.com/jinzihao) | ggml-cpu/amx: fix the (dead) `block_q8_K` VNNI quantization and enable the VNNI path | ✅ 有幫助 — merged | 57.25 → 59.20 (**+3.4 %**) | 10.94 → 11.24 (**+2.7 %**) |
260
  | upstream PR [#22525](https://github.com/ggml-org/llama.cpp/pull/22525) by [@HyeongiJeon](https://github.com/HyeongiJeon) | ggml-cpu: optimize `ggml_gemm_q4_K_8x8_q8_K` interleaving/staging for AVX-512 (and AVX2) | ✅ 有幫助(prefill)— merged | 55.78 → 57.42 (**+2.9 %**) | 10.70 → 10.87 (+1.5 %, neutral) |
 
261
 
262
  Round-by-round for #27024 (candidate vs baseline, t/s):
263
 
@@ -294,6 +295,25 @@ The candidate won 6 of 8 pairwise rounds on prefill (1 loss, 1 tie within
294
  interleaving/staging of the Q4_K GEMM (the prefill path), so the measured
295
  profile matches the expectation. Merged. Measured 2026-10-16.
296
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
297
  ## License
298
 
299
  This project (the patch set, `patch.sh` and documentation) is licensed
 
258
  |---|---|---:|---:|---:|
259
  | upstream PR [#27024](https://github.com/ggml-org/llama.cpp/pull/27024) by [@jinzihao](https://github.com/jinzihao) | ggml-cpu/amx: fix the (dead) `block_q8_K` VNNI quantization and enable the VNNI path | ✅ 有幫助 — merged | 57.25 → 59.20 (**+3.4 %**) | 10.94 → 11.24 (**+2.7 %**) |
260
  | upstream PR [#22525](https://github.com/ggml-org/llama.cpp/pull/22525) by [@HyeongiJeon](https://github.com/HyeongiJeon) | ggml-cpu: optimize `ggml_gemm_q4_K_8x8_q8_K` interleaving/staging for AVX-512 (and AVX2) | ✅ 有幫助(prefill)— merged | 55.78 → 57.42 (**+2.9 %**) | 10.70 → 10.87 (+1.5 %, neutral) |
261
+ | upstream PR [#19706](https://github.com/ggml-org/llama.cpp/pull/19706) by [@Manogna-Sree](https://github.com/Manogna-Sree) | Q6_K block interleaving for x86 SIMD (AVX-512/AVX2) | ❌ 沒幫助 — not merged | 58.16 → 58.34 (+0.3 %) | 11.06 → 11.07 (+0.1 %) |
262
 
263
  Round-by-round for #27024 (candidate vs baseline, t/s):
264
 
 
295
  interleaving/staging of the Q4_K GEMM (the prefill path), so the measured
296
  profile matches the expectation. Merged. Measured 2026-10-16.
297
 
298
+ Round-by-round for #19706 (candidate vs baseline, t/s), default settings:
299
+
300
+ | round | pp512 | tg128 |
301
+ |---|---:|---:|
302
+ | 1 | 58.42 vs 53.67 | 11.67 vs 11.45 |
303
+ | 2 | 59.46 vs 58.68 | 12.04 vs 10.84 |
304
+ | 3 | 54.57 vs 60.04 | 10.54 vs 11.82 |
305
+ | 4 | 62.32 vs 57.06 | 10.95 vs 11.16 |
306
+ | 5 | 57.66 vs 58.59 | 11.10 vs 10.86 |
307
+ | 6 | 57.26 vs 57.92 | 11.05 vs 11.17 |
308
+ | 7 | 59.71 vs 56.95 | 10.98 vs 10.96 |
309
+ | 8 | 58.25 vs 58.41 | 11.11 vs 10.83 |
310
+
311
+ **Verdict ❌ 沒幫助 (no measurable gain).** Medians differ by +0.3 %
312
+ (prefill) and +0.1 % (decode), with a 4 W / 3 L / 1 T split on both — pure
313
+ noise. The base `Q3_K_M` model contains only **one** single `Q6_K` tensor,
314
+ so a faster Q6_K kernel has almost no weight to act on. The PR is *not*
315
+ merged; the data is recorded here for the record. Measured 2026-10-16.
316
+
317
  ## License
318
 
319
  This project (the patch set, `patch.sh` and documentation) is licensed