docs: refresh measured performance (local retest 2026-08-18)

README.md

@@ -17,16 +17,16 @@ tags:
 
 # LFM2.5-2.6B — Core AI (.aimodel)
 
-`LiquidAI/LFM2.5-2.6B` converted to Core AI `.aimodel` bundles for Apple silicon by the
-[Visible](https://visible.cx) project. These are **derivative artifacts**: the weights are
-Liquid AI's, re-expressed as a Core AI graph with int8 block-32 symmetric weight
-quantization and a two-entrypoint (decode + chunked-prefill) function map. They run through
-Core AI on macOS; they are not PyTorch, GGUF or MLX checkpoints and cannot be loaded by
-those runtimes.
+`LiquidAI/LFM2.5-2.6B` converted to Core AI `.aimodel` bundles for Apple silicon by
+[visible-cx](https://huggingface.co/visible-cx). These are derivative artifacts: Liquid
+AI's weights re-expressed as a Core AI graph with int8 block-32 symmetric weight
+quantization and a two-entrypoint (decode + chunked-prefill) function map. They load
+through Core AI on macOS and are not usable by PyTorch, GGUF or MLX.
 
-**This is the model the Visible app selects as its quality tier.** It is the strongest of
-the three LFM2.5 text bundles in this catalog on Visible's own enrichment task, and the one
-whose outputs were judged most specific.
+Of the dense LFM2.5 bundles published here, this is the strongest on guided
+structured-output work and the only one qualified for long context: **3/3 verbatim needle
+recall at 14,566 tokens, sustaining 32 tok/s for under half a gigabyte of in-process
+footprint.**
 
 ## Provenance
 
@@ -36,28 +36,25 @@ whose outputs were judged most specific.
 | Recipe | `export_lfm2_multifunction.py int8hu --head-sym --chunk 64` |
 | Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
 | Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
-| Producer fingerprint | `coreai-core 1.0.0b2` on every inner `<name>.aimodel/metadata.json` — verified per bundle |
+| Producer fingerprint | `coreai-core 1.0.0b2` on every inner `<name>.aimodel/metadata.json` |
 | Weight format | int8, per-K-block-32, symmetric; symmetric head (`--head-sym`) |
 | Vocab | 128,000 |
-| Conversion host | Linux x86_64 (Debian 13), CPU tracing only |
+| Export functions | `main` (S=1 decode) + `prefill` (S=64 chunked prefill), `function_map: {"main": ["main", "prefill"]}`, weights deduplicated across entrypoints |
 
-`mf64` in the bundle name means **multifunction with a 64-wide prefill**: one asset
-exposing `main` (S=1 decode) and `prefill` (S=64 chunked prefill),
-`function_map: {"main": ["main", "prefill"]}`. Weights are deduplicated across the two
-entrypoints, so the prefill function costs well under a megabyte.
+`mf64` in the bundle name means multifunction with a 64-wide prefill; the prefill function
+costs well under a megabyte.
 
-### The `<think>` patch — read this before swapping in a fresh export
-
-Upstream's `chat_template.jinja` ends its generation prompt with an **open** think block
-(`<|im_start|>assistant\n<think>`). LFM2.5-2.6B is always-thinking, so an open block burns
-684–919 reasoning tokens per item. Every bundle in this repo ships the block **closed**:
+**Chat template.** LFM2.5-2.6B is an always-thinking model. The template shipped in every
+bundle here **terminates the reasoning block in the generation prompt**:
 
 ```jinja
 {{- "<|im_start|>assistant\n<think></think>\n" -}}
 ```
 
-This correction is applied post-export and **is not in the exporter**. A fresh conversion
-from the recipe reproduces the weights faithfully and this behaviour wrongly.
+An unterminated block (`…assistant\n<think>`) causes the model to spend the entire
+generation budget inside `<think>`, which a host routes to a reasoning channel and never to
+the response — 684–919 tokens per item, with no visible output. If you rebuild from the
+recipe, apply the same termination.
 
 ## Contents
 
@@ -68,25 +65,21 @@ from the recipe reproduces the weights faithfully and this behaviour wrongly.
 | `ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64` | 3,655,243,498 | 16384 | main + prefill |
 
 Each folder holds `<name>.aimodel/` (`main.mlirb` ≈ 3.64 GB, `main.hash`, asset
-`metadata.json`), a bundle-level `metadata.json`, and `tokenizer/`
-(`tokenizer.json`, `tokenizer_config.json`, `generation_config.json`,
-`chat_template.jinja`).
-
-**The three folders hold the same weights.** `--max-ctx` does not change the exported
-graph — function signatures, state descriptors and peak export RSS are identical at 4096,
-8192 and 16384. It changes exactly one thing: `language.max_context_length` in the bundle
-manifest. This was proved three ways, including a same-config control re-export whose
-`main.mlirb` differed from its twin by *more* bytes than a 4096→8192 pair does
-(conversion is not byte-deterministic; the context delta is inside that noise floor). Pick
-the folder whose manifest integer matches the window you intend to run, or edit the
-integer — you do not need a new conversion for a context bump.
-
-The small byte differences between the three folders are conversion nondeterminism, not
-content.
+`metadata.json`), a bundle-level `metadata.json`, and `tokenizer/` (`tokenizer.json`,
+`tokenizer_config.json`, `generation_config.json`, `chat_template.jinja`).
+
+**The three folders hold the same weights and the same graph** — function signatures, state
+descriptors and peak export RSS are identical at 4096, 8192 and 16384. `--max-ctx` changes
+exactly one thing: `language.max_context_length` in the bundle manifest. The small byte
+differences between folders are conversion nondeterminism, not content. Pick the folder
+whose manifest integer matches the window you intend to run.
+
+**Stop token:** `eos_token = "<|im_end|>"` in all three folders. Clean self-stop on every
+measured sample.
 
 ## Requirements
 
-- **Apple silicon Mac**, Core AI runtime. Nothing here runs on Linux or on Intel.
+- **Apple silicon Mac**, Core AI runtime.
 - **Engine contract: 2 inputs.** `input_ids`, `position_ids` → logits. No static inputs, no
   per-step mask. Runs on both the pipelined engine and the sequential (logits-capable)
   engine, which is what makes grammar-constrained decoding available.
@@ -98,61 +91,73 @@ content.
   268 MB at 16384.
 - **Memory envelope, 16 GB Mac** (Metal `recommendedMaxWorkingSetSize` ≈ 10.7 GB):
   weights 3.64 GB resident, so KV is not the binding constraint at any context this bundle
-  declares. Measured peak RSS on the qualification run was **4.43 GB**.
-- `runtime_env COREAI_CHUNK_THRESHOLD=1` travels with the recipe.
+  declares. Measured max RSS **6.71 GB** on guided decoding and 7.30 GB at 15k context;
+  peak in-process footprint 0.48 GB and 0.46 GB respectively — essentially flat in context.
+  **Minimum practical machine memory: 16 GB**, at any declared context including 16384.
+- The bundle manifest declares `runtime_env COREAI_CHUNK_THRESHOLD=1`.
 
-## Measured performance
+## Performance
 
-Measured on a **16 GB M2 Pro Mac (macOS 27 beta)** with the out-of-process `coreai-repro`
-harness: 10 real enrichment rows (5 POST, 5 COMMENT) from Visible's prompt corpus, the
-app's live `app_litert` JSON schema, greedy decoding through `respondJSON(to:schema:)`,
-`reset()` per row, sequential engine, 128-token cap. Load time excluded from `s/row`; row 1
-of each run excluded as a cache-warm outlier.
+Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**: 10-sample harness,
+guided JSON-constrained decoding against a fixed schema, greedy, sequential engine,
+`reset()` between samples, 128-token cap. Load excluded from `s/row`; sample 1 excluded as
+a cache-warm outlier.
 
 | | |
 |---|---|
-| Cold load | 39.5 s |
+| Cold load | 18.4 s |
 | Guided JSON parse | **10/10** |
-| s/row | **3.62** (POST) / **3.04** (COMMENT) |
-| Peak RSS | 4.43 GB |
-| Stop token | `<\|im_end\|>`, clean self-stop on every row |
-
-For scale within this catalog on the same harness and machine: LFM2.5-350M runs
-0.79/0.51 s/row, LFM2.5-1.2B 1.77/1.30 s/row. The 2.6B is ≈2× the 1.2B, which is what its
-parameter count predicts, and it produces the most specific `summary`/`context` fields of
-the three.
-
-A note on a figure that circulated internally: "1.75 s posts / 1.20 s comments" is the
-**1.2B's** number, not this model's. Plan the quality tier against 3.0–3.6 s/row.
-
-Nothing above was measured at 8192 or 16384 context; the `ctx8192`/`ctx16384` folders carry
-the 4096 folder's graph, so throughput should be unchanged, but quality and memory at depth
-are unmeasured.
-
-## Qualification status
+| Enum-clean | **10/10** |
+| s/row (long samples) | **3.66** |
+| s/row (short samples) | **3.06** |
+| Decode | **38.1–40.0 tok/s** |
+| TTFT | **0.47–2.29 s** |
+| Peak footprint | 0.48 GB |
+| Max RSS | 6.71 GB |
+
+For scale on the same machine and harness: LFM2.5-350M runs 0.84/0.55 s/row and
+LFM2.5-1.2B 1.89/1.37 s/row. The 2.6B is ≈2× the 1.2B, which is what its parameter count
+predicts, and it produces the most specific free-text fields of the three.
+
+### Long context
+
+**Needle-in-haystack: 3/3 verbatim at both 8k and 15k.** Three distinctive facts planted at
+10% / 50% / 90% of the filler, strict scoring (a fact counts only if the distinctive entity
+comes back correct). All three returned at 7,813 tokens and all three at 14,566 tokens,
+verbatim, including the date.
+
+| probe | prompt tokens | TTFT | decode | wall | peak footprint |
+|---|---|---|---|---|---|
+| needle 8k | 7,813 | 13.78 s | 35.8 tok/s | 15.4 s | 0.34 GB |
+| needle 15k | 14,566 | 27.42 s | **32.2 tok/s** | 29.1 s | 0.46 GB |
+
+Free-form generation from a fixed prompt at three depths, 900-token cap (the model
+self-stopped inside it every time):
+
+| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint |
+|---|---|---|---|---|---|---|
+| 3.4k | 3,249 | 6.88 s | 35.3 tok/s | 847 | **29.5 s** | 0.28 GB |
+| 8k | 7,673 | 14.84 s | 36.6 tok/s | 715 | **36.0 s** | 0.47 GB |
+| 12k | 11,643 | 22.24 s | **32.9 tok/s** | 743 | **44.4 s** | 0.46 GB |
+
+**Decode barely moves with depth.** 40.0 tok/s at 2.3k → 35.3 at 3.2k → 36.6 at 7.7k →
+32.9 at 11.6k: an 18% decay across a 5× context increase. It sustains 32–33 tok/s at
+11.6–14.5k tokens with under half a gigabyte of in-process footprint. High context costs
+prefill time and almost nothing else — TTFT scales cleanly (4.8 → 6.9 → 14.8 → 22.2 →
+27.4 s) while footprint stays flat. The KV term is `GrowingKVCache`-dynamic and small at
+this size (16 KB/token → 268 MB at 16384).
+
+## Status
 
 | Artifact | Status |
 |---|---|
-| `gpu-pipelined/…_mf64` (ctx 4096) | **SHIP** — measured on the harness above and selected by the Visible app as its quality tier |
-| `ctx8192/…_mf64` | **QUALIFIED BY EQUIVALENCE, UNMEASURED AT DEPTH** — byte-equivalent weights and an identical graph to the shipping bundle; no run has been made at >4096 tokens |
-| `ctx16384/…_mf64` | **QUALIFIED BY EQUIVALENCE, UNMEASURED AT DEPTH** — as above |
-
-## Verification
-
-- **Producer fingerprint:** every inner `<name>.aimodel/metadata.json` reads
-  `producer: "coreai-core 1.0.0b2"`. This matters: `coreai-torch 0.4.0` / `coreai-core
-  1.0.0b1` produce bundles that abort in-process at load and take the host application down
-  uncatchably. The fingerprint lives in the **inner asset** metadata, not the bundle
-  manifest — the manifest never carries a `producer` field, for good and bad bundles alike.
-- **Stop token:** `tokenizer_config.json` declares `eos_token = "<|im_end|>"` in all three
-  folders, verified. LFM bundles need no stop-token correction (unlike Gemma 4).
-- **Determinism:** `.aimodel` conversion is **not byte-reproducible**. Two runs of the same
-  command minutes apart on the same box differ in `main.mlirb`. The tokenizer half *is*
-  reproducible. So integrity for these bundles rests on per-file SHA-256 of the exact
-  published artifact, never on rebuilding and comparing hashes.
-- No oracle or PSNR gate has been run against a PyTorch reference for this bundle. The
-  qualification above is behavioural (parse rate, enum conformance, content sanity, clean
-  stop) on Visible's own task, not a numerics gate.
+| `gpu-pipelined/…_mf64` (ctx 4096) | **SHIP** — measured: 10/10 guided parse and enum-clean, 3.66/3.06 s/row, 38.1–40.0 tok/s. |
+| `ctx8192/…_mf64` | **QUALIFIED AT DEPTH** — same weights and graph; measured: 3/3 verbatim needle recall at 7,813 tokens, 36.6 tok/s at 8k, 0.47 GB peak footprint. |
+| `ctx16384/…_mf64` | **QUALIFIED AT DEPTH** — 3/3 verbatim needle recall at 14,566 tokens at 32.2 tok/s, 0.46 GB peak footprint, 7.30 GB max RSS. Recommended for long-context work. |
+
+No oracle or PSNR gate has been run against a PyTorch reference. Qualification is
+behavioural (parse rate, enum conformance, content sanity, clean stop, needle recall), not
+a numerics gate.
 
 ## License
 
@@ -161,5 +166,5 @@ bundles are a derivative of that checkpoint and the same licence and its obligat
 with them — see the
 [upstream licence](https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/main/LICENSE). Anyone
 redistributing these files should redistribute the licence with them and comply with its
-terms. Nothing here relicenses Liquid AI's weights; Visible's contribution is the
-conversion recipe and the qualification evidence.
+terms. Nothing here relicenses Liquid AI's weights; the contribution is the conversion
+recipe and the qualification evidence.