README.md

---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-2.6B
base_model_relation: quantized
library_name: coreai
pipeline_tag: text-generation
tags:
  - core-ai
  - aimodel
  - apple-silicon
  - on-device
  - coreai-kit
  - quantized
  - int8
  - lfm2.5
---

# LFM2.5-2.6B — Core AI (.aimodel)

`LiquidAI/LFM2.5-2.6B` converted to Core AI `.aimodel` bundles for Apple silicon by
[visible-cx](https://huggingface.co/visible-cx). These are derivative artifacts: Liquid AI's
weights re-expressed as a Core AI graph with int8 block-32 symmetric weight quantization and a
two-entrypoint (decode + chunked-prefill) function map. They load through Core AI on macOS and
are not usable by PyTorch, GGUF or MLX. This is the model the
[Visible](https://visible.cx) app routes **enrichment** to — per-item labelling and tagging,
where comprehension on argumentative text matters and a few seconds per item is acceptable.

Of the dense LFM2.5 bundles published here, this is the strongest on guided structured-output
work, the only one qualified for long context, and **the best-grounded local model this project
has measured**: 3/3 verbatim needle recall at 14,566 tokens, and zero invented attributors on
the real graph.

## Contents

| Path | Bytes | Manifest context | Functions |
|---|---:|---|---|
| `gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64` | 3,655,243,493 | 4096 | main + prefill |
| `ctx8192/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64` | 3,655,243,501 | 8192 | main + prefill |
| `ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64` | 3,655,243,498 | 16384 | main + prefill |

Each folder holds `<name>.aimodel/` (`main.mlirb` ≈ 3.64 GB, `main.hash`, asset
`metadata.json`), a bundle-level `metadata.json`, and `tokenizer/` (`tokenizer.json`,
`tokenizer_config.json`, `generation_config.json`, `chat_template.jinja`).

**The three folders hold the same weights and the same graph** — function signatures, state
descriptors and peak export RSS are identical at 4096, 8192 and 16384. `--max-ctx` changes
exactly one thing: `language.max_context_length` in the bundle manifest. The small byte
differences between folders are conversion nondeterminism, not content. Pick the folder whose
manifest integer matches the window you intend to run.

**Stop token:** `eos_token = "<|im_end|>"` in all three folders. Clean self-stop on every
measured sample.

**Chat template.** LFM2.5-2.6B is an always-thinking model. The template shipped in every
bundle here **terminates the reasoning block in the generation prompt**:

```jinja
{{- "<|im_start|>assistant\n<think></think>\n" -}}
```

An unterminated block (`…assistant\n<think>`) causes the model to spend the entire generation
budget inside `<think>`, which a host routes to a reasoning channel and never to the response —
684–919 tokens per item, with no visible output. If you rebuild from the recipe, apply the same
termination.

## Provenance

| | |
|---|---|
| Base checkpoint | `LiquidAI/LFM2.5-2.6B` |
| Recipe | `export_lfm2_multifunction.py int8hu --head-sym --chunk 64` |
| Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
| Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
| Producer fingerprint | `coreai-core 1.0.0b2` on every inner `<name>.aimodel/metadata.json` |
| Weight format | int8, per-K-block-32, symmetric; symmetric head (`--head-sym`) |
| Vocab | 128,000 |
| Export functions | `main` (S=1 decode) + `prefill` (S=64 chunked prefill), `function_map: {"main": ["main", "prefill"]}`, weights deduplicated across entrypoints |

`mf64` in the bundle name means multifunction with a 64-wide prefill; the prefill function
costs well under a megabyte.

The symmetric head is not incidental. Measured on the sibling 8B bundle in this org, an
**affine** head makes the compiler materialise two dequantised fp16 transposes of the whole
`vocab × hidden` matrix — a gigabyte of graph constant that is never read. A symmetric
dequantize is a scale multiply the GPU delegate folds into the matmul.

## Requirements

- **Apple silicon Mac**, Core AI runtime.
- **Engine contract: 2 inputs.** `input_ids`, `position_ids` → logits. No static inputs, no
  per-step mask. Runs on both the pipelined engine and the sequential (logits-capable) engine,
  which is what makes grammar-constrained decoding available.
- **States:** `keyCache` / `valueCache` `Float16, 8 × 1 × 8 × ? × 64` plus
  `convState Float16, 22 × 1 × 2048 × 2`. The sequence dim is dynamic, so the runtime resolves
  a `GrowingKVCache` (initial capacity 256, doubling) rather than allocating the manifest
  maximum up front. `convState` is fixed-size and does not scale with context.
- **KV cost: 16,384 bytes per token of context** (fp16) — 67 MB at 4096, 134 MB at 8192,
  268 MB at 16384. KV is not the binding constraint at any context this bundle declares.
- **Minimum practical machine memory: 16 GB**, at any declared context including 16384.
- The bundle manifest declares `runtime_env COREAI_CHUNK_THRESHOLD=1`.

## Measurements

Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**.

### Guided structured output

10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, sequential
engine, `reset()` between samples, 128-token cap. Load excluded from `s/row`; sample 1 excluded
as a cache-warm outlier.

| | |
|---|---|
| Cold load | 18.4 s |
| Guided JSON parse | **10/10** |
| Enum-clean | **10/10** |
| s/row (long samples) | **3.66** |
| s/row (short samples) | **3.06** |
| Decode | **38.1–40.0 tok/s** |
| TTFT | **0.47–2.29 s** |
| Peak footprint | 0.48 GB |
| Max RSS | 6.71 GB |

For scale on the same machine and harness: LFM2.5-350M runs 0.84/0.55 s/row and LFM2.5-1.2B
1.89/1.37 s/row. The 2.6B is ≈2× the 1.2B, which is what its parameter count predicts, and it
produces the most specific free-text fields of the three.

### Memory, measured rather than inferred

`Max RSS` is a resident set, and a resident set counts clean mapped pages the kernel can drop
for free — so it is not what the machine has to give up. Measured with an external watchdog
sampling wired memory, on the bundle the app pins:

| | GiB |
|---|---:|
| bundle on disk | 3.404 |
| **compiled blob** | **4.185** |
| graph constant | 0.797 |
| blob ÷ bundle | 1.23× |
| **wired, completed trace (3 legs)** | **4.300** |
| forecast (`blob × 1.106`) | 4.628 |
| **requirement** (peak + 1.25 GiB in-flight floor) | **6.03** |

**This is the only completed trace this project has that checks the plateau law against a run
that finished**, and the law reads **7.6% high** — i.e. conservative, on the safe side. Wired ÷
blob for this bundle is **0.96**.

### Long context

**Needle-in-haystack: 3/3 verbatim at both 8k and 15k.** Three distinctive facts planted at
10% / 50% / 90% of the filler, strict scoring (a fact counts only if the distinctive entity
comes back correct). All three returned at 7,813 tokens and all three at 14,566 tokens,
verbatim, including the date.

| probe | prompt tokens | TTFT | decode | wall | peak footprint |
|---|---|---|---|---|---|
| needle 8k | 7,813 | 13.78 s | 35.8 tok/s | 15.4 s | 0.34 GB |
| needle 15k | 14,566 | 27.42 s | **32.2 tok/s** | 29.1 s | 0.46 GB |

Free-form generation from a fixed prompt at three depths, 900-token cap (the model self-stopped
inside it every time):

| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint |
|---|---|---|---|---|---|---|
| 3.4k | 3,249 | 6.88 s | 35.3 tok/s | 847 | **29.5 s** | 0.28 GB |
| 8k | 7,673 | 14.84 s | 36.6 tok/s | 715 | **36.0 s** | 0.47 GB |
| 12k | 11,643 | 22.24 s | **32.9 tok/s** | 743 | **44.4 s** | 0.46 GB |

**Decode barely moves with depth** — 40.0 tok/s at 2.3k → 32.9 at 11.6k, an 18% decay across a
5× context increase — while peak footprint stays flat. High context costs prefill time and
almost nothing else.

### Grounding

Three real report questions on a real knowledge graph, full scorer, watchdog attached:

| question | prompt tokens | generated | tok/s | inversion | invented attributor | unsupported spans | narration | 3rd-person refs | JSON |
|---|---:|---:|---:|---|---|---|---|---|---|
| `reputationalRisks` | 5,306 | 228 | 38.4 | **0** | **0** | 0/0 | 0% | 0 | ok |
| `headline` | 3,096 | 26 | 39.1 | **0** | **0** | 0/0 | 0% | 0 | ok |
| `profileSummary` | 4,444 | 140 | 38.0 | **0** | **0** | 0/0 | 0% | 0 | ok |

**Zero invented attributors on the real graph** — the defect the prompt recipe was built to
kill, and the one a synthetic corpus could not produce. One scorer flag was hand-adjudicated
and dismissed as a false positive: the "motive bait" list fired on the model *quoting* the
subject's own words verbatim from the evidence window, and attributing a statement is not
asserting an inner state.

Across a wider comparison this bundle holds the **highest report quotation validity of any
local model measured here — 67–83% real quotations**, against 0–29% for the smaller dense LFMs,
and it is the only local model that declines to answer rather than inventing one.

## Usage

Swift Package Manager, via [CoreAIKit](https://github.com/john-rocky/coreai-kit) — a community
package, not affiliated with Apple, requiring macOS 27 beta:

```swift
.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")
```

`ModelID` addresses a bundle as **repo + path + revision**, where `path` is the subtree in this
repo holding one complete bundle (`metadata.json` + `*.aimodel/` + `tokenizer/`). It downloads
from the Hub on first use and is cached afterwards:

```swift
import CoreAIKit

let model = ModelID(
    "visible-cx/LFM2.5-2.6B-CoreAI",
    path: "ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64")

var config = ChatSession.Configuration()
config.engineVariant = .sequential   // required for guided / grammar-constrained decoding
config.temperature = nil             // greedy

let chat = try await ChatSession(model: model, configuration: config)
for try await event in chat.streamResponse(to: "…") {
    if case .response(let delta) = event { print(delta, terminator: "") }
}
```

Pass `revision:` a Hub commit hash to pin an immutable bundle. `ChatSession(bundleAt:)` loads a
bundle directory already on disk. Leave `COREAI_CHUNK_THRESHOLD` alone — the manifest sets it.

## Integrity

Core AI `.aimodel` bundles are **not byte-reproducible**: the exporter is not deterministic
even against itself, and two runs of the same command on the same host differ by a few dozen
bytes. Verify by digesting the exact published bytes rather than by rebuilding. Every bundle
carries `main.hash`, the raw 32 bytes of `sha256(main.mlirb)`, so a downloaded bundle can be
checked against itself; on the Hub the same value is recoverable from the LFS oid without
fetching the file.

## Status

| Artifact | Status |
|---|---|
| `gpu-pipelined/…_mf64` (ctx 4096) | **SHIP** — measured: 10/10 guided parse and enum-clean, 3.66/3.06 s/row, 38.1–40.0 tok/s, requirement 6.03 GiB from a completed wired trace. |
| `ctx8192/…_mf64` | **QUALIFIED AT DEPTH** — same weights and graph; measured: 3/3 verbatim needle recall at 7,813 tokens, 36.6 tok/s at 8k, 0.47 GB peak footprint. |
| `ctx16384/…_mf64` | **QUALIFIED AT DEPTH** — 3/3 verbatim needle recall at 14,566 tokens at 32.2 tok/s, 0.46 GB peak footprint, plus the 3/3-clean real-graph grounding leg above. Recommended for long-context work. |

No oracle or PSNR gate has been run against a PyTorch reference. Qualification is behavioural
(parse rate, enum conformance, grounding scoring, clean stop, needle recall) plus the memory
instrumentation, not a numerics gate.

## License

`LiquidAI/LFM2.5-2.6B` is released under the **LFM Open License v1.0** (`lfm1.0`), and upstream
declares it as `license: other` + `license_name: lfm1.0`. These bundles are a derivative of that
checkpoint and the same licence and its obligations travel with them — see the
[upstream licence](https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/main/LICENSE). Anyone
redistributing these files should redistribute the licence with them and comply with its terms.
Nothing here relicenses Liquid AI's weights; the contribution is the conversion recipe and the
qualification evidence.