compunerd commited on
Commit
82bf4d8
·
1 Parent(s): 65d070e

Correct the latency claim: measure against llama.cpp, not our own worst case

Browse files

llama-bench on this machine, CPU-only, same model family: 3.02 tok/s
(0.33 s/token) Q4_K_M, against ~0.7 s/token here. We are 2.1x SLOWER than
the reference implementation. The previous '68 s -> 0.7 s' framing used an
unoptimised numpy path as the baseline, which is our own worst case and not
a referenceable measurement.

Files changed (1) hide show
  1. README.md +10 -4
README.md CHANGED
@@ -29,15 +29,21 @@ padding is never written to memory.
29
  | RoPE == free-running oscillator phase | **2.6e-08** rel-err |
30
  | Amplitude coupling reproduces `W·x` | **3.2e-08**; phase-only Kuramoto **9.75e-01** — the encoding is load-bearing |
31
  | int8 per-column fidelity | y cosine **0.999949**; tokens unchanged through all 28 layers |
32
- | Per-token latency | 68 s → **~0.7 s**, same hardware |
33
 
34
  ## Honest limitations
35
 
36
  - **It performs matmuls.** The wave form is an *equivalence proof* about the
37
  arithmetic, not a substitute for it: relaxation converges to `W·x`, which is a
38
- value you have already computed. There is no speedup from the physics and none
39
- is claimed. The speedup is ordinary systems work — OpenBLAS, int8 residency,
40
- KV cache, batched prefill, and not materialising bf16's sixteen zero bits.
 
 
 
 
 
 
41
  - **Ternary does not work.** Measured engine-free against real weights on both
42
  Llama-3.2-3B and Gemma-4-12B: y cosine **0.877** against int8's **0.99995**.
43
  Per-column scaling rescues int2 relative to a global scale (0.52 vs 0.036) but
 
29
  | RoPE == free-running oscillator phase | **2.6e-08** rel-err |
30
  | Amplitude coupling reproduces `W·x` | **3.2e-08**; phase-only Kuramoto **9.75e-01** — the encoding is load-bearing |
31
  | int8 per-column fidelity | y cosine **0.999949**; tokens unchanged through all 28 layers |
32
+ | Per-token latency | **~0.7 s** — see the honest note below; this is NOT a speedup |
33
 
34
  ## Honest limitations
35
 
36
  - **It performs matmuls.** The wave form is an *equivalence proof* about the
37
  arithmetic, not a substitute for it: relaxation converges to `W·x`, which is a
38
+ value you have already computed. There is no speedup from the physics.
39
+ - **And there is no speedup at all, against a real baseline.** On this same
40
+ machine, CPU-only, same model family, `llama-bench` measures **3.02 tok/s
41
+ (0.33 s/token)** for Q4_K_M against our **~0.7 s/token** — we are **2.1x
42
+ slower than llama.cpp**. Earlier drafts framed this as "68 s -> 0.7 s"; that
43
+ 68 s was an unoptimised numpy path re-reading 6.4 GB from disk per token, i.e.
44
+ our own worst case, and quoting it as a baseline was not honest. The work here
45
+ bought a *correct and resident* int8 CPU engine, not a fast one. The value is
46
+ in the equivalence proofs and the negative results, not the throughput.
47
  - **Ternary does not work.** Measured engine-free against real weights on both
48
  Llama-3.2-3B and Gemma-4-12B: y cosine **0.877** against int8's **0.99995**.
49
  Per-column scaling rescues int2 relative to a global scale (0.52 vs 0.036) but