card: same-day ANE vs GPU A/B on iPhone 17 Pro
Browse files
README.md
CHANGED
|
@@ -101,6 +101,8 @@ Gated on an iPhone 17 Pro (iOS 27.0) against the fp32 HF oracle: teacher-forced
|
|
| 101 |
method, 512-token prompt, 1024 generated, 5 trials, two back-to-back runs): decode **69.6 / 58.6 tok/s**, prefill 2710 / 2364 tok/s; footprint 1.3 GB (p128 / g256: decode 62.3). The Neural Engine is the power-efficient lane; these are
|
| 102 |
not a GPU comparison (different protocol from the `int8/` row above).
|
| 103 |
|
|
|
|
|
|
|
| 104 |
## Measured
|
| 105 |
|
| 106 |
| | decode | prefill | numerics | size |
|
|
|
|
| 101 |
method, 512-token prompt, 1024 generated, 5 trials, two back-to-back runs): decode **69.6 / 58.6 tok/s**, prefill 2710 / 2364 tok/s; footprint 1.3 GB (p128 / g256: decode 62.3). The Neural Engine is the power-efficient lane; these are
|
| 102 |
not a GPU comparison (different protocol from the `int8/` row above).
|
| 103 |
|
| 104 |
+
Same-day interleaved A/B on the same phone (one app embedding both bundles, 128-token prompt, 256 generated, 5 trials, ANE-GPU-ANE-GPU): Neural Engine 8-bit **76.8 / 76.8 tok/s** decode (3899 / 3865 prefill) vs this repo's `int8/` GPU bundle **67.1 / 64.4** (2359 / 2270) — equal weight bytes per token, so +17 % decode and +70 % prefill on the ANE; GPU cold specialization 3.6 s vs ANE AOT load 0.2 s.
|
| 105 |
+
|
| 106 |
## Measured
|
| 107 |
|
| 108 |
| | decode | prefill | numerics | size |
|