HelloSun commited on
Commit ·
c1ddb47
1
Parent(s): 176ef13
experiments: record draft PR #27396 (speculative draft prefill) - not applicable, not ported
Browse files
README.md
CHANGED
|
@@ -271,6 +271,7 @@ their data and are **not** merged (❌ 沒幫助).
|
|
| 271 |
| upstream PR [#27590](https://github.com/ggml-org/llama.cpp/pull/27590) by [@SongXiaoXi](https://github.com/SongXiaoXi) | ggml-cpu: add AVX-512/VNNI paths for Q5_K/Q6_K dot products | ✅ 有幫助(`-nr`)— merged | `-nr`: 65.91 → 66.35 (+0.7 %) | `-nr`: 11.16 → 11.32 (**+1.4 %**) |
|
| 272 |
| upstream PR [#18698](https://github.com/ggml-org/llama.cpp/pull/18698) by [@zzjianhui](https://github.com/zzjianhui) | ggml-cpu: repack weights in parallel (OpenMP, NUMA-local thread placement) | ❌ 沒幫助(推論退化)— not merged | 56.93 → 54.58 (**−4.1 %**) | 10.67 → 10.45 (**−2.1 %**) |
|
| 273 |
| draft PR [#27032](https://github.com/ggml-org/llama.cpp/pull/27032) by [@lexasub](https://github.com/lexasub) | CPU optimizations (GET_ROWS+ADD fusion, `get_rows` fast paths, tokenizer, KV-cell rework) — only the ggml-kernel subset is portable onto this tree | ❌ 沒幫助(不適用於本模型)— not merged | 56.64 → 56.88 (+0.4 %) | 10.82 → 10.73 (−0.8 %) |
|
|
|
|
| 274 |
|
| 275 |
Round-by-round for #27024 (candidate vs baseline, t/s):
|
| 276 |
|
|
@@ -398,6 +399,16 @@ optimized Q4_K dequant path is not used (`token_embd` is Q3_K); (3) the
|
|
| 398 |
remaining `get_rows` cache-hit path only helps repeated 1-row gathers, which
|
| 399 |
this workload barely has. Recorded for the record. Measured 2026-10-16.
|
| 400 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 401 |
### `-nr` vs default on this machine
|
| 402 |
|
| 403 |
Side observation from the re-anchoring session: on this AMX machine the
|
|
|
|
| 271 |
| upstream PR [#27590](https://github.com/ggml-org/llama.cpp/pull/27590) by [@SongXiaoXi](https://github.com/SongXiaoXi) | ggml-cpu: add AVX-512/VNNI paths for Q5_K/Q6_K dot products | ✅ 有幫助(`-nr`)— merged | `-nr`: 65.91 → 66.35 (+0.7 %) | `-nr`: 11.16 → 11.32 (**+1.4 %**) |
|
| 272 |
| upstream PR [#18698](https://github.com/ggml-org/llama.cpp/pull/18698) by [@zzjianhui](https://github.com/zzjianhui) | ggml-cpu: repack weights in parallel (OpenMP, NUMA-local thread placement) | ❌ 沒幫助(推論退化)— not merged | 56.93 → 54.58 (**−4.1 %**) | 10.67 → 10.45 (**−2.1 %**) |
|
| 273 |
| draft PR [#27032](https://github.com/ggml-org/llama.cpp/pull/27032) by [@lexasub](https://github.com/lexasub) | CPU optimizations (GET_ROWS+ADD fusion, `get_rows` fast paths, tokenizer, KV-cell rework) — only the ggml-kernel subset is portable onto this tree | ❌ 沒幫助(不適用於本模型)— not merged | 56.64 → 56.88 (+0.4 %) | 10.82 → 10.73 (−0.8 %) |
|
| 274 |
+
| draft PR [#27396](https://github.com/ggml-org/llama.cpp/pull/27396) by [@lexasub](https://github.com/lexasub) | speculative draft-model prefill (zero-copy embedding, event-based sync) for EAGLE3/DFlash/GLM4-style draft heads | ❌ 沒幫助(不適用;未移植)— not merged | — | — |
|
| 275 |
|
| 276 |
Round-by-round for #27024 (candidate vs baseline, t/s):
|
| 277 |
|
|
|
|
| 399 |
remaining `get_rows` cache-hit path only helps repeated 1-row gathers, which
|
| 400 |
this workload barely has. Recorded for the record. Measured 2026-10-16.
|
| 401 |
|
| 402 |
+
Recorded verdict for draft #27396 (not ported, not measured): the draft
|
| 403 |
+
optimizes the *speculative draft-model prefill* path (zero-copy embedding,
|
| 404 |
+
event-based sync) for EAGLE3 / DFlash / GLM4-style draft heads, and it
|
| 405 |
+
touches core files (`llama-batch`, `llama-context`, `llama-graph`, ...) that
|
| 406 |
+
have diverged from its base — 7 of 17 files conflict and a faithful port
|
| 407 |
+
would require redesigning the speculative prefill plumbing. This CPU setup
|
| 408 |
+
does not use speculative decoding at all — the bundled MTP head measured
|
| 409 |
+
net-negative on CPU (see the inference section above) — so the draft has no
|
| 410 |
+
applicable surface here. Recorded 2026-10-16.
|
| 411 |
+
|
| 412 |
### `-nr` vs default on this machine
|
| 413 |
|
| 414 |
Side observation from the re-anchoring session: on this AMX machine the
|