model card: standard structure, measured facts, verified license declarations

README.md

@@ -3,6 +3,7 @@ license: other
 license_name: lfm1.0
 license_link: https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/main/LICENSE
 base_model: LiquidAI/LFM2.5-2.6B
+base_model_relation: quantized
 library_name: coreai
 pipeline_tag: text-generation
 tags:
@@ -10,6 +11,7 @@ tags:
   - aimodel
   - apple-silicon
   - on-device
+  - coreai-kit
   - quantized
   - int8
   - lfm2.5
@@ -18,43 +20,17 @@ tags:
 # LFM2.5-2.6B — Core AI (.aimodel)
 
 `LiquidAI/LFM2.5-2.6B` converted to Core AI `.aimodel` bundles for Apple silicon by
-[visible-cx](https://huggingface.co/visible-cx). These are derivative artifacts: Liquid
-AI's weights re-expressed as a Core AI graph with int8 block-32 symmetric weight
-quantization and a two-entrypoint (decode + chunked-prefill) function map. They load
-through Core AI on macOS and are not usable by PyTorch, GGUF or MLX.
+[visible-cx](https://huggingface.co/visible-cx). These are derivative artifacts: Liquid AI's
+weights re-expressed as a Core AI graph with int8 block-32 symmetric weight quantization and a
+two-entrypoint (decode + chunked-prefill) function map. They load through Core AI on macOS and
+are not usable by PyTorch, GGUF or MLX. This is the model the
+[Visible](https://visible.cx) app routes **enrichment** to — per-item labelling and tagging,
+where comprehension on argumentative text matters and a few seconds per item is acceptable.
 
-Of the dense LFM2.5 bundles published here, this is the strongest on guided
-structured-output work and the only one qualified for long context: **3/3 verbatim needle
-recall at 14,566 tokens, sustaining 32 tok/s for under half a gigabyte of in-process
-footprint.**
-
-## Provenance
-
-| | |
-|---|---|
-| Base checkpoint | `LiquidAI/LFM2.5-2.6B` |
-| Recipe | `export_lfm2_multifunction.py int8hu --head-sym --chunk 64` |
-| Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
-| Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
-| Producer fingerprint | `coreai-core 1.0.0b2` on every inner `<name>.aimodel/metadata.json` |
-| Weight format | int8, per-K-block-32, symmetric; symmetric head (`--head-sym`) |
-| Vocab | 128,000 |
-| Export functions | `main` (S=1 decode) + `prefill` (S=64 chunked prefill), `function_map: {"main": ["main", "prefill"]}`, weights deduplicated across entrypoints |
-
-`mf64` in the bundle name means multifunction with a 64-wide prefill; the prefill function
-costs well under a megabyte.
-
-**Chat template.** LFM2.5-2.6B is an always-thinking model. The template shipped in every
-bundle here **terminates the reasoning block in the generation prompt**:
-
-```jinja
-{{- "<|im_start|>assistant\n<think></think>\n" -}}
-```
-
-An unterminated block (`…assistant\n<think>`) causes the model to spend the entire
-generation budget inside `<think>`, which a host routes to a reasoning channel and never to
-the response — 684–919 tokens per item, with no visible output. If you rebuild from the
-recipe, apply the same termination.
+Of the dense LFM2.5 bundles published here, this is the strongest on guided structured-output
+work, the only one qualified for long context, and **the best-grounded local model this project
+has measured**: 3/3 verbatim needle recall at 14,566 tokens, and zero invented attributors on
+the real graph.
 
 ## Contents
 
@@ -71,37 +47,69 @@ Each folder holds `<name>.aimodel/` (`main.mlirb` ≈ 3.64 GB, `main.hash`, asse
 **The three folders hold the same weights and the same graph** — function signatures, state
 descriptors and peak export RSS are identical at 4096, 8192 and 16384. `--max-ctx` changes
 exactly one thing: `language.max_context_length` in the bundle manifest. The small byte
-differences between folders are conversion nondeterminism, not content. Pick the folder
-whose manifest integer matches the window you intend to run.
+differences between folders are conversion nondeterminism, not content. Pick the folder whose
+manifest integer matches the window you intend to run.
 
 **Stop token:** `eos_token = "<|im_end|>"` in all three folders. Clean self-stop on every
 measured sample.
 
+**Chat template.** LFM2.5-2.6B is an always-thinking model. The template shipped in every
+bundle here **terminates the reasoning block in the generation prompt**:
+
+```jinja
+{{- "<|im_start|>assistant\n<think></think>\n" -}}
+```
+
+An unterminated block (`…assistant\n<think>`) causes the model to spend the entire generation
+budget inside `<think>`, which a host routes to a reasoning channel and never to the response —
+684–919 tokens per item, with no visible output. If you rebuild from the recipe, apply the same
+termination.
+
+## Provenance
+
+| | |
+|---|---|
+| Base checkpoint | `LiquidAI/LFM2.5-2.6B` |
+| Recipe | `export_lfm2_multifunction.py int8hu --head-sym --chunk 64` |
+| Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
+| Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
+| Producer fingerprint | `coreai-core 1.0.0b2` on every inner `<name>.aimodel/metadata.json` |
+| Weight format | int8, per-K-block-32, symmetric; symmetric head (`--head-sym`) |
+| Vocab | 128,000 |
+| Export functions | `main` (S=1 decode) + `prefill` (S=64 chunked prefill), `function_map: {"main": ["main", "prefill"]}`, weights deduplicated across entrypoints |
+
+`mf64` in the bundle name means multifunction with a 64-wide prefill; the prefill function
+costs well under a megabyte.
+
+The symmetric head is not incidental. Measured on the sibling 8B bundle in this org, an
+**affine** head makes the compiler materialise two dequantised fp16 transposes of the whole
+`vocab × hidden` matrix — a gigabyte of graph constant that is never read. A symmetric
+dequantize is a scale multiply the GPU delegate folds into the matmul.
+
 ## Requirements
 
 - **Apple silicon Mac**, Core AI runtime.
 - **Engine contract: 2 inputs.** `input_ids`, `position_ids` → logits. No static inputs, no
-  per-step mask. Runs on both the pipelined engine and the sequential (logits-capable)
-  engine, which is what makes grammar-constrained decoding available.
+  per-step mask. Runs on both the pipelined engine and the sequential (logits-capable) engine,
+  which is what makes grammar-constrained decoding available.
 - **States:** `keyCache` / `valueCache` `Float16, 8 × 1 × 8 × ? × 64` plus
-  `convState Float16, 22 × 1 × 2048 × 2`. The sequence dim is dynamic, so the runtime
-  resolves a `GrowingKVCache` (initial capacity 256, doubling) rather than allocating the
-  manifest maximum up front. `convState` is fixed-size and does not scale with context.
+  `convState Float16, 22 × 1 × 2048 × 2`. The sequence dim is dynamic, so the runtime resolves
+  a `GrowingKVCache` (initial capacity 256, doubling) rather than allocating the manifest
+  maximum up front. `convState` is fixed-size and does not scale with context.
 - **KV cost: 16,384 bytes per token of context** (fp16) — 67 MB at 4096, 134 MB at 8192,
-  268 MB at 16384.
-- **Memory envelope, 16 GB Mac** (Metal `recommendedMaxWorkingSetSize` ≈ 10.7 GB):
-  weights 3.64 GB resident, so KV is not the binding constraint at any context this bundle
-  declares. Measured max RSS **6.71 GB** on guided decoding and 7.30 GB at 15k context;
-  peak in-process footprint 0.48 GB and 0.46 GB respectively — essentially flat in context.
-  **Minimum practical machine memory: 16 GB**, at any declared context including 16384.
+  268 MB at 16384. KV is not the binding constraint at any context this bundle declares.
+- **Minimum practical machine memory: 16 GB**, at any declared context including 16384.
 - The bundle manifest declares `runtime_env COREAI_CHUNK_THRESHOLD=1`.
 
-## Performance
+## Measurements
+
+Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**.
+
+### Guided structured output
 
-Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**: 10-sample harness,
-guided JSON-constrained decoding against a fixed schema, greedy, sequential engine,
-`reset()` between samples, 128-token cap. Load excluded from `s/row`; sample 1 excluded as
-a cache-warm outlier.
+10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, sequential
+engine, `reset()` between samples, 128-token cap. Load excluded from `s/row`; sample 1 excluded
+as a cache-warm outlier.
 
 | | |
 |---|---|
@@ -115,9 +123,29 @@ a cache-warm outlier.
 | Peak footprint | 0.48 GB |
 | Max RSS | 6.71 GB |
 
-For scale on the same machine and harness: LFM2.5-350M runs 0.84/0.55 s/row and
-LFM2.5-1.2B 1.89/1.37 s/row. The 2.6B is ≈2× the 1.2B, which is what its parameter count
-predicts, and it produces the most specific free-text fields of the three.
+For scale on the same machine and harness: LFM2.5-350M runs 0.84/0.55 s/row and LFM2.5-1.2B
+1.89/1.37 s/row. The 2.6B is ≈2× the 1.2B, which is what its parameter count predicts, and it
+produces the most specific free-text fields of the three.
+
+### Memory, measured rather than inferred
+
+`Max RSS` is a resident set, and a resident set counts clean mapped pages the kernel can drop
+for free — so it is not what the machine has to give up. Measured with an external watchdog
+sampling wired memory, on the bundle the app pins:
+
+| | GiB |
+|---|---:|
+| bundle on disk | 3.404 |
+| **compiled blob** | **4.185** |
+| graph constant | 0.797 |
+| blob ÷ bundle | 1.23× |
+| **wired, completed trace (3 legs)** | **4.300** |
+| forecast (`blob × 1.106`) | 4.628 |
+| **requirement** (peak + 1.25 GiB in-flight floor) | **6.03** |
+
+**This is the only completed trace this project has that checks the plateau law against a run
+that finished**, and the law reads **7.6% high** — i.e. conservative, on the safe side. Wired ÷
+blob for this bundle is **0.96**.
 
 ### Long context
 
@@ -131,8 +159,8 @@ verbatim, including the date.
 | needle 8k | 7,813 | 13.78 s | 35.8 tok/s | 15.4 s | 0.34 GB |
 | needle 15k | 14,566 | 27.42 s | **32.2 tok/s** | 29.1 s | 0.46 GB |
 
-Free-form generation from a fixed prompt at three depths, 900-token cap (the model
-self-stopped inside it every time):
+Free-form generation from a fixed prompt at three depths, 900-token cap (the model self-stopped
+inside it every time):
 
 | depth | prompt tokens | TTFT | decode | generated | wall | peak footprint |
 |---|---|---|---|---|---|---|
@@ -140,31 +168,91 @@ self-stopped inside it every time):
 | 8k | 7,673 | 14.84 s | 36.6 tok/s | 715 | **36.0 s** | 0.47 GB |
 | 12k | 11,643 | 22.24 s | **32.9 tok/s** | 743 | **44.4 s** | 0.46 GB |
 
-**Decode barely moves with depth.** 40.0 tok/s at 2.3k → 35.3 at 3.2k → 36.6 at 7.7k →
-32.9 at 11.6k: an 18% decay across a 5× context increase. It sustains 32–33 tok/s at
-11.6–14.5k tokens with under half a gigabyte of in-process footprint. High context costs
-prefill time and almost nothing else — TTFT scales cleanly (4.8 → 6.9 → 14.8 → 22.2 →
-27.4 s) while footprint stays flat. The KV term is `GrowingKVCache`-dynamic and small at
-this size (16 KB/token → 268 MB at 16384).
+**Decode barely moves with depth** — 40.0 tok/s at 2.3k → 32.9 at 11.6k, an 18% decay across a
+5× context increase — while peak footprint stays flat. High context costs prefill time and
+almost nothing else.
+
+### Grounding
+
+Three real report questions on a real knowledge graph, full scorer, watchdog attached:
+
+| question | prompt tokens | generated | tok/s | inversion | invented attributor | unsupported spans | narration | 3rd-person refs | JSON |
+|---|---:|---:|---:|---|---|---|---|---|---|
+| `reputationalRisks` | 5,306 | 228 | 38.4 | **0** | **0** | 0/0 | 0% | 0 | ok |
+| `headline` | 3,096 | 26 | 39.1 | **0** | **0** | 0/0 | 0% | 0 | ok |
+| `profileSummary` | 4,444 | 140 | 38.0 | **0** | **0** | 0/0 | 0% | 0 | ok |
+
+**Zero invented attributors on the real graph** — the defect the prompt recipe was built to
+kill, and the one a synthetic corpus could not produce. One scorer flag was hand-adjudicated
+and dismissed as a false positive: the "motive bait" list fired on the model *quoting* the
+subject's own words verbatim from the evidence window, and attributing a statement is not
+asserting an inner state.
+
+Across a wider comparison this bundle holds the **highest report quotation validity of any
+local model measured here — 67–83% real quotations**, against 0–29% for the smaller dense LFMs,
+and it is the only local model that declines to answer rather than inventing one.
+
+## Usage
+
+Swift Package Manager, via [CoreAIKit](https://github.com/john-rocky/coreai-kit) — a community
+package, not affiliated with Apple, requiring macOS 27 beta:
+
+```swift
+.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
+// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")
+```
+
+`ModelID` addresses a bundle as **repo + path + revision**, where `path` is the subtree in this
+repo holding one complete bundle (`metadata.json` + `*.aimodel/` + `tokenizer/`). It downloads
+from the Hub on first use and is cached afterwards:
+
+```swift
+import CoreAIKit
+
+let model = ModelID(
+    "visible-cx/LFM2.5-2.6B-CoreAI",
+    path: "ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64")
+
+var config = ChatSession.Configuration()
+config.engineVariant = .sequential   // required for guided / grammar-constrained decoding
+config.temperature = nil             // greedy
+
+let chat = try await ChatSession(model: model, configuration: config)
+for try await event in chat.streamResponse(to: "…") {
+    if case .response(let delta) = event { print(delta, terminator: "") }
+}
+```
+
+Pass `revision:` a Hub commit hash to pin an immutable bundle. `ChatSession(bundleAt:)` loads a
+bundle directory already on disk. Leave `COREAI_CHUNK_THRESHOLD` alone — the manifest sets it.
+
+## Integrity
+
+Core AI `.aimodel` bundles are **not byte-reproducible**: the exporter is not deterministic
+even against itself, and two runs of the same command on the same host differ by a few dozen
+bytes. Verify by digesting the exact published bytes rather than by rebuilding. Every bundle
+carries `main.hash`, the raw 32 bytes of `sha256(main.mlirb)`, so a downloaded bundle can be
+checked against itself; on the Hub the same value is recoverable from the LFS oid without
+fetching the file.
 
 ## Status
 
 | Artifact | Status |
 |---|---|
-| `gpu-pipelined/…_mf64` (ctx 4096) | **SHIP** — measured: 10/10 guided parse and enum-clean, 3.66/3.06 s/row, 38.1–40.0 tok/s. |
+| `gpu-pipelined/…_mf64` (ctx 4096) | **SHIP** — measured: 10/10 guided parse and enum-clean, 3.66/3.06 s/row, 38.1–40.0 tok/s, requirement 6.03 GiB from a completed wired trace. |
 | `ctx8192/…_mf64` | **QUALIFIED AT DEPTH** — same weights and graph; measured: 3/3 verbatim needle recall at 7,813 tokens, 36.6 tok/s at 8k, 0.47 GB peak footprint. |
-| `ctx16384/…_mf64` | **QUALIFIED AT DEPTH** — 3/3 verbatim needle recall at 14,566 tokens at 32.2 tok/s, 0.46 GB peak footprint, 7.30 GB max RSS. Recommended for long-context work. |
+| `ctx16384/…_mf64` | **QUALIFIED AT DEPTH** — 3/3 verbatim needle recall at 14,566 tokens at 32.2 tok/s, 0.46 GB peak footprint, plus the 3/3-clean real-graph grounding leg above. Recommended for long-context work. |
 
-No oracle or PSNR gate has been run against a PyTorch reference. Qualification is
-behavioural (parse rate, enum conformance, content sanity, clean stop, needle recall), not
-a numerics gate.
+No oracle or PSNR gate has been run against a PyTorch reference. Qualification is behavioural
+(parse rate, enum conformance, grounding scoring, clean stop, needle recall) plus the memory
+instrumentation, not a numerics gate.
 
 ## License
 
-`LiquidAI/LFM2.5-2.6B` is released under the **LFM Open License v1.0** (`lfm1.0`). These
-bundles are a derivative of that checkpoint and the same licence and its obligations travel
-with them — see the
+`LiquidAI/LFM2.5-2.6B` is released under the **LFM Open License v1.0** (`lfm1.0`), and upstream
+declares it as `license: other` + `license_name: lfm1.0`. These bundles are a derivative of that
+checkpoint and the same licence and its obligations travel with them — see the
 [upstream licence](https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/main/LICENSE). Anyone
-redistributing these files should redistribute the licence with them and comply with its
-terms. Nothing here relicenses Liquid AI's weights; the contribution is the conversion
-recipe and the qualification evidence.
+redistributing these files should redistribute the licence with them and comply with its terms.
+Nothing here relicenses Liquid AI's weights; the contribution is the conversion recipe and the
+qualification evidence.