Upload report.md with huggingface_hub
Browse files
report.md
CHANGED
|
@@ -55,3 +55,10 @@ Memory: model buffer 4701 MiB (mmap), KV 17408 MiB @ 16k ctx (F16).
|
|
| 55 |
| stock (AMX disabled) | off | 134.3 | 19.3 |
|
| 56 |
|
| 57 |
Best: AMX-enabled build with repack off → 147.4 t/s prefill, 19.4 t/s decode.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
| stock (AMX disabled) | off | 134.3 | 19.3 |
|
| 56 |
|
| 57 |
Best: AMX-enabled build with repack off → 147.4 t/s prefill, 19.4 t/s decode.
|
| 58 |
+
|
| 59 |
+
## Default vs optimized (CLI, same prompt, 128 gen)
|
| 60 |
+
| Config | Prompt t/s | Gen t/s |
|
| 61 |
+
|---|---:|---:|
|
| 62 |
+
| default (no flags, 96 threads) | 73.4 | 5.6 |
|
| 63 |
+
| 16 threads only | 147.7 | 18.3 |
|
| 64 |
+
| optimized (16 thr, no-repack, FA on, F16 KV) | 147.4 | 19.4 |
|