docs: performance section leads with the preferred mf64 bundle; superseded figures follow, explicitly labeled
Browse files
README.md
CHANGED
|
@@ -121,40 +121,9 @@ Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**, sequential en
|
|
| 121 |
`reset()` between samples. Load excluded from `s/row`; sample 1 excluded as a cache-warm
|
| 122 |
outlier.
|
| 123 |
|
| 124 |
-
### `
|
| 125 |
|
| 126 |
-
|
| 127 |
-
|---|---|
|
| 128 |
-
| Cold load | 21.3 s |
|
| 129 |
-
| Guided JSON parse | **10/10** |
|
| 130 |
-
| Enum-clean | **10/10** |
|
| 131 |
-
| s/row (long samples) | **17.88** |
|
| 132 |
-
| s/row (short samples) | **8.59** |
|
| 133 |
-
| Decode | **45.2–46.6 tok/s** |
|
| 134 |
-
| TTFT | **6.5–23.3 s** |
|
| 135 |
-
| Peak footprint | 0.16 GB |
|
| 136 |
-
| Max RSS | 10.61 GB |
|
| 137 |
-
|
| 138 |
-
Decode is healthy for the size — an MoE with ~1B active parameters decoding faster than a
|
| 139 |
-
dense 2.6B. **The per-sample cost is prefill, not decode**: at 17.88 s/row it is roughly
|
| 140 |
-
10× the LFM2.5-1.2B, entirely because of the missing prefill function.
|
| 141 |
-
|
| 142 |
-
Unguided, free-form generation from a fixed prompt, 900-token cap, with the chat template
|
| 143 |
-
patched to terminate the think block (unpatched, these runs return 0 characters).
|
| 144 |
-
|
| 145 |
-
| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint | max RSS |
|
| 146 |
-
|---|---|---|---|---|---|---|---|
|
| 147 |
-
| 3.4k | 3,249 | 70.05 s | 47.0 tok/s | 900 | 89.0 s | 0.21 GB | 10.58 GB |
|
| 148 |
-
| 8k | 7,673 | 165.11 s | 45.2 tok/s | 519 | 176.8 s | 0.27 GB | 11.03 GB |
|
| 149 |
-
|
| 150 |
-
**Needle-in-haystack recall at 8k: 3/3.** Three distinctive facts planted at 10/50/90% of
|
| 151 |
-
the filler, strict scoring; returned at 7,813 prompt tokens in a 27-token answer
|
| 152 |
-
(TTFT 168.09 s, 45.3 tok/s). 15k was not run: the manifest ceiling on `gpu-pipelined/` is
|
| 153 |
-
8192, and S=1 prefill at that depth would cost roughly five minutes.
|
| 154 |
-
|
| 155 |
-
### `mf64/`
|
| 156 |
-
|
| 157 |
-
Same machine, same harness, same prompts, same schema, chat template unmodified.
|
| 158 |
|
| 159 |
| | `gpu-pipelined/` | `mf64/` |
|
| 160 |
|---|---|---|
|
|
@@ -186,6 +155,37 @@ in depth.
|
|
| 186 |
includes the graph compile. Subsequent loads were **22.9–26.2 s**. Budget roughly 17 GB of
|
| 187 |
on-disk compile cache in addition to the 9.4 GB bundle.
|
| 188 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 189 |
## Status
|
| 190 |
|
| 191 |
| Artifact | Status |
|
|
|
|
| 121 |
`reset()` between samples. Load excluded from `s/row`; sample 1 excluded as a cache-warm
|
| 122 |
outlier.
|
| 123 |
|
| 124 |
+
### `mf64/` — preferred bundle
|
| 125 |
|
| 126 |
+
Chat template unmodified. Same harness and prompts as the superseded bundle (below); the comparison column restates its figures.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
|
| 128 |
| | `gpu-pipelined/` | `mf64/` |
|
| 129 |
|---|---|---|
|
|
|
|
| 155 |
includes the graph compile. Subsequent loads were **22.9–26.2 s**. Budget roughly 17 GB of
|
| 156 |
on-disk compile cache in addition to the 9.4 GB bundle.
|
| 157 |
|
| 158 |
+
### `gpu-pipelined/` — superseded
|
| 159 |
+
|
| 160 |
+
| | |
|
| 161 |
+
|---|---|
|
| 162 |
+
| Cold load | 21.3 s |
|
| 163 |
+
| Guided JSON parse | **10/10** |
|
| 164 |
+
| Enum-clean | **10/10** |
|
| 165 |
+
| s/row (long samples) | **17.88** |
|
| 166 |
+
| s/row (short samples) | **8.59** |
|
| 167 |
+
| Decode | **45.2–46.6 tok/s** |
|
| 168 |
+
| TTFT | **6.5–23.3 s** |
|
| 169 |
+
| Peak footprint | 0.16 GB |
|
| 170 |
+
| Max RSS | 10.61 GB |
|
| 171 |
+
|
| 172 |
+
Decode is healthy for the size — an MoE with ~1B active parameters decoding faster than a
|
| 173 |
+
dense 2.6B. **The per-sample cost is prefill, not decode**: at 17.88 s/row it is roughly
|
| 174 |
+
10× the LFM2.5-1.2B, entirely because of the missing prefill function.
|
| 175 |
+
|
| 176 |
+
Unguided, free-form generation from a fixed prompt, 900-token cap, with the chat template
|
| 177 |
+
patched to terminate the think block (unpatched, these runs return 0 characters).
|
| 178 |
+
|
| 179 |
+
| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint | max RSS |
|
| 180 |
+
|---|---|---|---|---|---|---|---|
|
| 181 |
+
| 3.4k | 3,249 | 70.05 s | 47.0 tok/s | 900 | 89.0 s | 0.21 GB | 10.58 GB |
|
| 182 |
+
| 8k | 7,673 | 165.11 s | 45.2 tok/s | 519 | 176.8 s | 0.27 GB | 11.03 GB |
|
| 183 |
+
|
| 184 |
+
**Needle-in-haystack recall at 8k: 3/3** (this superseded bundle). Three distinctive facts planted at 10/50/90% of
|
| 185 |
+
the filler, strict scoring; returned at 7,813 prompt tokens in a 27-token answer
|
| 186 |
+
(TTFT 168.09 s, 45.3 tok/s). 15k was not run on this superseded bundle (`mf64/` above passes 3/3 at 15k): its manifest ceiling is
|
| 187 |
+
8192, and S=1 prefill at that depth would cost roughly five minutes.
|
| 188 |
+
|
| 189 |
## Status
|
| 190 |
|
| 191 |
| Artifact | Status |
|