Upload report.md with huggingface_hub
Browse files
report.md
CHANGED
|
@@ -44,3 +44,14 @@ Memory: model buffer 4701 MiB (mmap), KV 17408 MiB @ 16k ctx (F16).
|
|
| 44 |
## Files
|
| 45 |
- results/amx_threads.jsonl: raw llama-bench samples
|
| 46 |
- build: llama-pinned/build-amx (binaries)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
## Files
|
| 45 |
- results/amx_threads.jsonl: raw llama-bench samples
|
| 46 |
- build: llama-pinned/build-amx (binaries)
|
| 47 |
+
|
| 48 |
+
## CLI measured (real generation, prompt ~19 tokens + 128 gen, 16 threads, F16 KV, FA on)
|
| 49 |
+
|
| 50 |
+
| Build | repack | Prompt t/s (median) | Gen t/s (median) |
|
| 51 |
+
|---|---|---:|---:|
|
| 52 |
+
| AMX-enabled | on (AMX VNNI) | 141.9 | 18.7 |
|
| 53 |
+
| AMX-enabled | off (AVX-512) | 147.4 | 19.4 |
|
| 54 |
+
| stock (AMX disabled) | on (CPU_REPACK) | 129.3 | 18.7 |
|
| 55 |
+
| stock (AMX disabled) | off | 134.3 | 19.3 |
|
| 56 |
+
|
| 57 |
+
Best: AMX-enabled build with repack off → 147.4 t/s prefill, 19.4 t/s decode.
|