Spaces:
Paused
Paused
Correct the latency claim: measure against llama.cpp, not our own worst case
Browse filesllama-bench on this machine, CPU-only, same model family: 3.02 tok/s
(0.33 s/token) Q4_K_M, against ~0.7 s/token here. We are 2.1x SLOWER than
the reference implementation. The previous '68 s -> 0.7 s' framing used an
unoptimised numpy path as the baseline, which is our own worst case and not
a referenceable measurement.
README.md
CHANGED
|
@@ -29,15 +29,21 @@ padding is never written to memory.
|
|
| 29 |
| RoPE == free-running oscillator phase | **2.6e-08** rel-err |
|
| 30 |
| Amplitude coupling reproduces `W·x` | **3.2e-08**; phase-only Kuramoto **9.75e-01** — the encoding is load-bearing |
|
| 31 |
| int8 per-column fidelity | y cosine **0.999949**; tokens unchanged through all 28 layers |
|
| 32 |
-
| Per-token latency |
|
| 33 |
|
| 34 |
## Honest limitations
|
| 35 |
|
| 36 |
- **It performs matmuls.** The wave form is an *equivalence proof* about the
|
| 37 |
arithmetic, not a substitute for it: relaxation converges to `W·x`, which is a
|
| 38 |
-
value you have already computed. There is no speedup from the physics
|
| 39 |
-
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
- **Ternary does not work.** Measured engine-free against real weights on both
|
| 42 |
Llama-3.2-3B and Gemma-4-12B: y cosine **0.877** against int8's **0.99995**.
|
| 43 |
Per-column scaling rescues int2 relative to a global scale (0.52 vs 0.036) but
|
|
|
|
| 29 |
| RoPE == free-running oscillator phase | **2.6e-08** rel-err |
|
| 30 |
| Amplitude coupling reproduces `W·x` | **3.2e-08**; phase-only Kuramoto **9.75e-01** — the encoding is load-bearing |
|
| 31 |
| int8 per-column fidelity | y cosine **0.999949**; tokens unchanged through all 28 layers |
|
| 32 |
+
| Per-token latency | **~0.7 s** — see the honest note below; this is NOT a speedup |
|
| 33 |
|
| 34 |
## Honest limitations
|
| 35 |
|
| 36 |
- **It performs matmuls.** The wave form is an *equivalence proof* about the
|
| 37 |
arithmetic, not a substitute for it: relaxation converges to `W·x`, which is a
|
| 38 |
+
value you have already computed. There is no speedup from the physics.
|
| 39 |
+
- **And there is no speedup at all, against a real baseline.** On this same
|
| 40 |
+
machine, CPU-only, same model family, `llama-bench` measures **3.02 tok/s
|
| 41 |
+
(0.33 s/token)** for Q4_K_M against our **~0.7 s/token** — we are **2.1x
|
| 42 |
+
slower than llama.cpp**. Earlier drafts framed this as "68 s -> 0.7 s"; that
|
| 43 |
+
68 s was an unoptimised numpy path re-reading 6.4 GB from disk per token, i.e.
|
| 44 |
+
our own worst case, and quoting it as a baseline was not honest. The work here
|
| 45 |
+
bought a *correct and resident* int8 CPU engine, not a fast one. The value is
|
| 46 |
+
in the equivalence proofs and the negative results, not the throughput.
|
| 47 |
- **Ternary does not work.** Measured engine-free against real weights on both
|
| 48 |
Llama-3.2-3B and Gemma-4-12B: y cosine **0.877** against int8's **0.99995**.
|
| 49 |
Per-column scaling rescues int2 relative to a global scale (0.52 vs 0.036) but
|