card: add the raw-MLX arm — Core AI ties MLX, both beat the ExecuTorch build by 14%
Browse files
README.md
CHANGED
|
@@ -80,6 +80,30 @@ code falls to 1.40× and free chat to **30.10 tok/s, below their number**; tool
|
|
| 80 |
matters more than acceptance does: verify cost here is a staircase (S ≤ 3 free, S = 4…8 ~1.47×,
|
| 81 |
S ≥ 9 ~2.3×), so K=8 makes free chat 5% *slower* while K=2 makes it 34% faster.
|
| 82 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
**Not claimed here:** Meta's published quality figure (1.0% degradation across 15 benchmarks
|
| 84 |
for the 17G quant) has no matched counterpart; this port has a token-exact gate and read
|
| 85 |
generations, not a benchmark suite.
|
|
|
|
| 80 |
matters more than acceptance does: verify cost here is a staircase (S ≤ 3 free, S = 4…8 ~1.47×,
|
| 81 |
S ≥ 9 ~2.3×), so K=8 makes free chat 5% *slower* while K=2 makes it 34% faster.
|
| 82 |
|
| 83 |
+
### Raw MLX is the third arm, and it changes what the ExecuTorch result means
|
| 84 |
+
|
| 85 |
+
ExecuTorch's `metal` backend is MLX-native, so beating it could mean beating MLX or beating
|
| 86 |
+
the wrapper around MLX. Only raw MLX separates the two. Same machine, same prompts, greedy,
|
| 87 |
+
192 tokens, interleaved CA/ET/MLX with a 45 s cooldown between every run:
|
| 88 |
+
|
| 89 |
+
| | p1 r1 | p1 r2 | p2 r1 | p2 r2 | mean |
|
| 90 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 91 |
+
| **Core AI** `int4hu`, 16.35 GB | 27.5 | 27.1 | 27.5 | 27.6 | **27.43** |
|
| 92 |
+
| **MLX** `mlx-community/…-4bit`, 18 GB | 27.18 | 27.38 | 27.50 | 27.37 | **27.36** |
|
| 93 |
+
| **ExecuTorch** `k-quant-17G…metal`, 17.9 GB | 24.0 | 23.8 | 24.1 | 24.1 | **24.00** |
|
| 94 |
+
|
| 95 |
+
**Core AI and raw MLX are indistinguishable (+0.3%). Both beat Meta's own build by ~14%.**
|
| 96 |
+
|
| 97 |
+
So the honest reading is not "Core AI is fast here" — it is that **Meta's shipped on-device
|
| 98 |
+
artifact leaves ~14% on the table against the runtime it is built on**. Core AI matching MLX
|
| 99 |
+
at 27.9 B dense is what [`coreai-vs-mlx-speed.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/coreai-vs-mlx-speed.md)
|
| 100 |
+
already predicts: Core AI ≥ MLX on small dense, converging to a tie as the model grows and
|
| 101 |
+
MLX's 4-bit byte advantage cashes in. This is the largest dense point on that curve so far,
|
| 102 |
+
and it lands on the tie.
|
| 103 |
+
|
| 104 |
+
Prompt processing is not matched here and no claim is made from it (Core AI 269 tok/s at 512
|
| 105 |
+
prompt tokens, MLX 128–136 at 77–81 — different lengths, different batching).
|
| 106 |
+
|
| 107 |
**Not claimed here:** Meta's published quality figure (1.0% degradation across 15 benchmarks
|
| 108 |
for the 17G quant) has no matched counterpart; this port has a token-exact gate and read
|
| 109 |
generations, not a benchmark suite.
|