HelloSun commited on
Commit
c1ddb47
·
1 Parent(s): 176ef13

experiments: record draft PR #27396 (speculative draft prefill) - not applicable, not ported

Browse files
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -271,6 +271,7 @@ their data and are **not** merged (❌ 沒幫助).
271
  | upstream PR [#27590](https://github.com/ggml-org/llama.cpp/pull/27590) by [@SongXiaoXi](https://github.com/SongXiaoXi) | ggml-cpu: add AVX-512/VNNI paths for Q5_K/Q6_K dot products | ✅ 有幫助(`-nr`)— merged | `-nr`: 65.91 → 66.35 (+0.7 %) | `-nr`: 11.16 → 11.32 (**+1.4 %**) |
272
  | upstream PR [#18698](https://github.com/ggml-org/llama.cpp/pull/18698) by [@zzjianhui](https://github.com/zzjianhui) | ggml-cpu: repack weights in parallel (OpenMP, NUMA-local thread placement) | ❌ 沒幫助(推論退化)— not merged | 56.93 → 54.58 (**−4.1 %**) | 10.67 → 10.45 (**−2.1 %**) |
273
  | draft PR [#27032](https://github.com/ggml-org/llama.cpp/pull/27032) by [@lexasub](https://github.com/lexasub) | CPU optimizations (GET_ROWS+ADD fusion, `get_rows` fast paths, tokenizer, KV-cell rework) — only the ggml-kernel subset is portable onto this tree | ❌ 沒幫助(不適用於本模型)— not merged | 56.64 → 56.88 (+0.4 %) | 10.82 → 10.73 (−0.8 %) |
 
274
 
275
  Round-by-round for #27024 (candidate vs baseline, t/s):
276
 
@@ -398,6 +399,16 @@ optimized Q4_K dequant path is not used (`token_embd` is Q3_K); (3) the
398
  remaining `get_rows` cache-hit path only helps repeated 1-row gathers, which
399
  this workload barely has. Recorded for the record. Measured 2026-10-16.
400
 
 
 
 
 
 
 
 
 
 
 
401
  ### `-nr` vs default on this machine
402
 
403
  Side observation from the re-anchoring session: on this AMX machine the
 
271
  | upstream PR [#27590](https://github.com/ggml-org/llama.cpp/pull/27590) by [@SongXiaoXi](https://github.com/SongXiaoXi) | ggml-cpu: add AVX-512/VNNI paths for Q5_K/Q6_K dot products | ✅ 有幫助(`-nr`)— merged | `-nr`: 65.91 → 66.35 (+0.7 %) | `-nr`: 11.16 → 11.32 (**+1.4 %**) |
272
  | upstream PR [#18698](https://github.com/ggml-org/llama.cpp/pull/18698) by [@zzjianhui](https://github.com/zzjianhui) | ggml-cpu: repack weights in parallel (OpenMP, NUMA-local thread placement) | ❌ 沒幫助(推論退化)— not merged | 56.93 → 54.58 (**−4.1 %**) | 10.67 → 10.45 (**−2.1 %**) |
273
  | draft PR [#27032](https://github.com/ggml-org/llama.cpp/pull/27032) by [@lexasub](https://github.com/lexasub) | CPU optimizations (GET_ROWS+ADD fusion, `get_rows` fast paths, tokenizer, KV-cell rework) — only the ggml-kernel subset is portable onto this tree | ❌ 沒幫助(不適用於本模型)— not merged | 56.64 → 56.88 (+0.4 %) | 10.82 → 10.73 (−0.8 %) |
274
+ | draft PR [#27396](https://github.com/ggml-org/llama.cpp/pull/27396) by [@lexasub](https://github.com/lexasub) | speculative draft-model prefill (zero-copy embedding, event-based sync) for EAGLE3/DFlash/GLM4-style draft heads | ❌ 沒幫助(不適用;未移植)— not merged | — | — |
275
 
276
  Round-by-round for #27024 (candidate vs baseline, t/s):
277
 
 
399
  remaining `get_rows` cache-hit path only helps repeated 1-row gathers, which
400
  this workload barely has. Recorded for the record. Measured 2026-10-16.
401
 
402
+ Recorded verdict for draft #27396 (not ported, not measured): the draft
403
+ optimizes the *speculative draft-model prefill* path (zero-copy embedding,
404
+ event-based sync) for EAGLE3 / DFlash / GLM4-style draft heads, and it
405
+ touches core files (`llama-batch`, `llama-context`, `llama-graph`, ...) that
406
+ have diverged from its base — 7 of 17 files conflict and a faithful port
407
+ would require redesigning the speculative prefill plumbing. This CPU setup
408
+ does not use speculative decoding at all — the bundled MTP head measured
409
+ net-negative on CPU (see the inference section above) — so the draft has no
410
+ applicable surface here. Recorded 2026-10-16.
411
+
412
  ### `-nr` vs default on this machine
413
 
414
  Side observation from the re-anchoring session: on this AMX machine the