Upload report.md with huggingface_hub

report.md

@@ -1,64 +1,38 @@
-# Experiment_1: openPangu-VL-7B-Q4_K_M on Intel Xeon 8559C (CPU)
+# Experiment_1: openPangu-VL-7B-Q4_K_M, text only, CPU
 
-Model: HelloSun/openPangu-VL-7B-Q4KM-gguf (openPangu-VL-7B-Q4_K_M.gguf, 4.93 GB, 8.03B params)
-Device: Intel Xeon Platinum 8559C, 2 sockets x 48 cores (192 threads, HT), AVX-512 VNNI/BF16/FP16, AMX-INT8/TILE/BF16.
-Cgroup CPU quota: 16 (the host is shared; 16 threads is the sweet spot).
-RAM: ~2 TB host, ~2 GB usable in this container.
-Backend: llama.cpp 1268edac5 + openpangu-vl-llama.cpp.patch (pangu_vl RoPE), built with GGML_NATIVE=ON, AMX on.
-Build: /root/app/pangu/llama-pinned/build-amx (libggml-cpu with AMX).
+Model file: openPangu-VL-7B-Q4_K_M.gguf (unchanged, no re-quantization)
+CPU: Intel Xeon Platinum 8559C, cgroup quota 16 CPUs, shared host (run-to-run variance about +-15%)
+Runtime: llama.cpp 1268edac5 + openpangu-vl-llama.cpp.patch, CLI, 128 generated tokens, pinned to CPUs 0-15
 
-## Best config (text only)
-- 16 threads, pinned to CPUs 0-15 (single NUMA node0)
-- flash attention on
-- KV cache F16 (faster than Q8_0/Q4_0 on this CPU)
-- --no-repack (repack/AMX is slower here; stock AVX-512 VNNI path wins)
-- ubatch 512, batch 2048
+## Default vs optimized (llama-cli, real generation)
+| Config | Prefill t/s | Decode t/s |
+|---|---:|---:|
+| default (no flags, 96 threads) | 73.4 | 5.6 |
+| optimized (-t 16 -fa on, F16 KV, no-repack) | 147.4 | 19.4 |
+| speedup | ~2.0x | ~3.5x |
 
-## Results (median of 3-5 runs, llama-bench -o jsonl)
-| threads | pp512 t/s | tg128 t/s |
+## What actually caused the gain
+| Config | Prefill t/s | Decode t/s |
 |---|---:|---:|
+| -t 16 only | 147.7 | 18.3 |
+| -t 16 + FA + no-repack | 147.4 | 19.4 |
+
+The gain comes from using 16 threads. FA, repack on/off and the AMX build differ within noise
+(same config repeated: prefill 119-148 t/s). AMX gave no reliable speedup.
+
+## Thread scaling (llama-bench, repack on, AMX build)
+| threads | pp512 | tg128 |
+|---:|---:|---:|
 | 1 | 16.28 | 1.65 |
-| 2 | 31.82 | 3.23 |
 | 4 | 57.46 | 6.11 |
-| 6 | 95.77 | 8.44 |
 | 8 | 125.15 | 11.20 |
-| 10 | 157.58 | 13.00 |
 | 12 | 189.09 | 14.66 |
-| 14 | 210.57 | 16.58 |
 | 16 | 208.93 | 18.31 |
 | 20 | 156.98 | 9.95 |
 | 24 | 163.43 | 8.90 |
 
-Best (16 threads, no-repack): pp512 = 232.70 t/s, tg128 = 19.77 t/s.
-
-KV cache type (16 threads, repack on): F16 pp 208.93 / tg 18.31; Q8_0 pp 173.87 / tg 19.10; Q4_0 pp 170.14 / tg 18.59. F16 fastest prefill; Q8_0 slightly faster decode.
-
-Long context decode (16 threads, no-repack, F16 KV): tg64 @ 2048 ctx = 16.63, @ 4096 = 14.88, @ 8192 = 13.44 t/s.
-
-ubatch sweep (pp512): 64=189.00, 128=206.44, 256=201.25, 512=208.93, 1024=183.61.
+More than 16 threads degrades sharply (CPU quota throttling).
 
-AMX (repack on) vs stock (repack off): AMX pp 208.93 vs stock pp 232.70. Stock AVX-512 path wins on this CPU for Q4_K.
+Long context decode (tg64): 2048 ctx 16.63, 4096 ctx 14.88, 8192 ctx 13.44 t/s.
 
-Memory: model buffer 4701 MiB (mmap), KV 17408 MiB @ 16k ctx (F16).
-
-## Files
-- results/amx_threads.jsonl: raw llama-bench samples
-- build: llama-pinned/build-amx (binaries)
-
-## CLI measured (real generation, prompt ~19 tokens + 128 gen, 16 threads, F16 KV, FA on)
-
-| Build | repack | Prompt t/s (median) | Gen t/s (median) |
-|---|---|---:|---:|
-| AMX-enabled | on (AMX VNNI) | 141.9 | 18.7 |
-| AMX-enabled | off (AVX-512) | 147.4 | 19.4 |
-| stock (AMX disabled) | on (CPU_REPACK) | 129.3 | 18.7 |
-| stock (AMX disabled) | off | 134.3 | 19.3 |
-
-Best: AMX-enabled build with repack off → 147.4 t/s prefill, 19.4 t/s decode.
-
-## Default vs optimized (CLI, same prompt, 128 gen)
-| Config | Prompt t/s | Gen t/s |
-|---|---:|---:|
-| default (no flags, 96 threads) | 73.4 | 5.6 |
-| 16 threads only | 147.7 | 18.3 |
-| optimized (16 thr, no-repack, FA on, F16 KV) | 147.4 | 19.4 |
+Raw data: amx_threads.jsonl