Add measured Galaxy S26 GPU-vs-CPU rows for the .litertlm bundle

#2
by mlboydaisuke - opened
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -99,3 +99,14 @@ Note that all benchmark stats are from a Samsung S24 Ultra with
99
  * Benchmark is done assuming XNNPACK cache is enabled
100
  * Benchmark is run with cache enabled and initialized. During the first run, the time to first token may differ.
101
  * dynamic_int8: quantized model with int8 weights and float activations.
 
 
 
 
 
 
 
 
 
 
 
 
99
  * Benchmark is done assuming XNNPACK cache is enabled
100
  * Benchmark is run with cache enabled and initialized. During the first run, the time to first token may differ.
101
  * dynamic_int8: quantized model with int8 weights and float activations.
102
+
103
+ ### Galaxy S26 β€” `.litertlm` bundle, GPU vs CPU (litert-lm 0.16.0)
104
+
105
+ Community measurement of the `Phi-4-mini-instruct_multi-prefill-seq_q8_ekv4096.litertlm` file β€” a different runtime path and artifact than the S24 Ultra table above, so compare within one table, not across. Physical Samsung Galaxy S26 (SM-S942Q, Snapdragon 8 Elite Gen 5 / SM8850, Android 16), `litert_lm_advanced_main` from the litert-lm v0.16.0 release, GPU backend OpenCL (`LITERT_CL`). One fixed 205-token prompt text (201 tokens under this tokenizer), `--benchmark`. Two runs per backend taken back-to-back β€” cells show the range (the CPU's first run also pays cold file I/O on the 3.9 GB bundle). Peak RSS is the process VmHWM. Before quoting, the same file was run on each backend with a real prompt: both produced a correct text answer.
106
+
107
+ | Backend | Prefill (201 tok) | Decode | Time-to-first-token | Init | Peak RSS |
108
+ |---|---|---|---|---|---|
109
+ | **GPU (OpenCL)** | **195–196 tok/s** | **10.6 tok/s** | **1.12–1.13 s** | 5.9–7.9 s | **1447 MB** |
110
+ | CPU (XNNPACK) | 78–128 tok/s | 6.8–6.9 tok/s | 1.72–2.72 s | 4.8–5.0 s | 5661 MB |
111
+
112
+ The GPU takes the whole graph β€” decode 1648/1648 ops and all four prefill signatures 1677/1677 on `LITERT_CL`. It wins everywhere on this bundle: prefill 1.5–2.5Γ—, decode 1.5Γ—, and peak RSS 3.9Γ— lower (1447 against 5661 MB) β€” on a 4 GB-class model the memory difference is what decides whether the phone stays comfortable.