README.md

---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-1.2B-Instruct
base_model_relation: quantized
library_name: coreai
pipeline_tag: text-generation
tags:
  - core-ai
  - aimodel
  - apple-silicon
  - on-device
  - coreai-kit
  - quantized
  - int8
  - lfm2.5
---

# LFM2.5-1.2B-Instruct — Core AI (.aimodel)

`LiquidAI/LFM2.5-1.2B-Instruct` converted to Core AI `.aimodel` bundles for Apple silicon by
[visible-cx](https://huggingface.co/visible-cx). These are derivative artifacts: Liquid AI's
weights re-expressed as a Core AI graph with int8 block-32 symmetric weight quantization and a
two-entrypoint (decode + chunked-prefill) function map. They load through Core AI on macOS and
are not usable by PyTorch, GGUF or MLX.

It is the speed tier of the dense LFM2.5 bundles in this org: **1.89 s per guided
structured-output row at 71–75 tok/s in 3.42 GB of resident memory**, with 3/3 verbatim needle
recall at 15k tokens. It runs comfortably on an 8 GB Mac, which the 2.6B does not.

> **Use it for structured extraction and short cards, not for long prose over cited evidence.**
> On the same eleven-question report corpus that this org's 2.6B answered with 67–83% real
> quotations, the 1.2B's prose quotations measured **0% valid** — it produces fluent text with
> quotation marks around material that is not in the evidence. That is why the
> [Visible](https://visible.cx) app's routing table keeps it off report work regardless of how
> fast it is.

## Contents

| Path | Bytes | Manifest context | Functions |
|---|---:|---|---|
| `gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64` | 1,702,039,167 | 4096 | main + prefill |
| `ctx8192/gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64` | 1,702,039,171 | 8192 | main + prefill |
| `ctx16384/gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64` | 1,702,039,167 | 16384 | main + prefill |

Each folder holds `<name>.aimodel/` (`main.mlirb` ≈ 1.70 GB, `main.hash`, asset
`metadata.json`), a bundle-level `metadata.json`, and `tokenizer/` (`tokenizer.json`,
`tokenizer_config.json`, `special_tokens_map.json`, `chat_template.jinja`).

**The three folders hold the same weights and the same graph** — identical function signatures,
identical state descriptors, identical export peak RSS at 4096/8192/16384. `--max-ctx` sets one
manifest integer, `language.max_context_length`, and nothing else. The folders exist so each
declared window is a clean, fingerprinted artifact rather than a hand-edited manifest.

**Stop token:** `eos_token = "<|im_end|>"` in all three folders. Clean self-stop on every
measured sample.

This model is **not** a thinking model and needs no chat-template adjustment — its template ends
at `<|im_start|>assistant\n`.

## Provenance

| | |
|---|---|
| Base checkpoint | `LiquidAI/LFM2.5-1.2B-Instruct` |
| Recipe | `export_lfm2_multifunction.py int8hu --head-sym --chunk 64` |
| Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
| Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
| Producer fingerprint | `coreai-core 1.0.0b2` on every inner `<name>.aimodel/metadata.json` |
| Weight format | int8, per-K-block-32, symmetric; symmetric head (`--head-sym`) |
| Vocab | 65,536 |
| Export functions | `main` (S=1 decode) + `prefill` (S=64 chunked prefill), `function_map: {"main": ["main", "prefill"]}`, weights shared |

`mf64` in the bundle name means multifunction with a 64-wide prefill.

The symmetric head is not incidental. Measured on the sibling 8B bundle in this org, an
**affine** head makes the compiler materialise dequantised fp16 transposes of the whole
`vocab × hidden` matrix — graph constant that is never read. A symmetric dequantize is a scale
multiply the GPU delegate folds into the matmul.

## Requirements

- **Apple silicon Mac**, Core AI runtime.
- **Engine contract: 2 inputs** (`input_ids`, `position_ids`) → logits. No static inputs, no
  per-step mask. Runs on the pipelined engine and on the sequential engine, which is the only
  logits-capable one and therefore the only path for grammar-constrained decoding.
- **States:** `keyCache` / `valueCache` `Float16, 6 × 1 × 8 × ? × 64` plus
  `convState Float16, 10 × 1 × 2048 × 2`. Dynamic sequence dim → `GrowingKVCache` (initial 256,
  doubling), not a static allocation at the manifest maximum. `convState` does not scale with
  context.
- **KV cost: 12,288 bytes per token** (fp16) — 50 MB at 4096, 101 MB at 8192, 201 MB at 16384.
  KV is nowhere near binding at any context this bundle declares.
- **Minimum practical machine memory: 8 GB.**
- The bundle manifest declares `runtime_env COREAI_CHUNK_THRESHOLD=1`.

## Measurements

Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**.

### Guided structured output

10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, sequential
engine, `reset()` between samples, 128-token cap. Load excluded from `s/row`; sample 1 excluded
as a cache-warm outlier.

| | |
|---|---|
| Cold load | 5.4 s |
| Guided JSON parse | **10/10** |
| Enum-clean | **10/10** |
| s/row (long samples) | **1.89** |
| s/row (short samples) | **1.37** |
| Decode | **71.5–74.7 tok/s** |
| TTFT | **0.37–1.60 s** |
| Peak footprint | 0.14 GB |
| Max RSS | 3.42 GB |

**Output-identical to a stored reference baseline.** All ten guided outputs diff clean against
it — every character of every field, in order. That equality is behavioural and exact; it does
not mean byte-identical weights, since `.aimodel` conversion is not byte-deterministic.

### Memory, measured rather than inferred

`Max RSS` counts clean mapped pages the kernel can drop for free, so it is not what the machine
has to give up. Measured with an external watchdog sampling wired memory:

| | GiB |
|---|---:|
| bundle on disk | 1.585 |
| **compiled blob** | **2.128** |
| graph constant | 0.547 |
| blob ÷ bundle | 1.34× |
| wired floor (32-token leg — a lower bound, not a plateau) | 2.190 |
| forecast (`blob × 1.106`) | 2.353 |
| **requirement** (peak + 1.25 GiB in-flight floor) | **3.61** |

The measured floor and the forecast **agree to 0.3%** — 2.360 against 2.353 — from two
instruments with nothing in common. Wired ÷ blob for this bundle is **1.03**.

### Long context

**Needle-in-haystack: 3/3 verbatim at both 8k and 15k.** Three distinctive facts planted at
10% / 50% / 90% of the filler, strict scoring (a fact counts only if the distinctive entity
comes back correct). All three returned at 8,099 tokens and all three at 15,106 tokens,
verbatim, including the date.

| probe | prompt tokens | TTFT | decode | wall | peak footprint |
|---|---|---|---|---|---|
| needle 8k | 8,099 | 6.98 s | 64.5 tok/s | 8.3 s | 0.21 GB |
| needle 15k | 15,106 | 12.79 s | 59.7 tok/s | 14.4 s | 0.32 GB |

Free-form generation from a fixed prompt at three depths, 900-token cap (the model self-stopped
inside it every time):

| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint |
|---|---|---|---|---|---|---|
| 3.4k | 3,371 | 3.22 s | 71.0 tok/s | 361 | **8.5 s** | 0.16 GB |
| 8k | 7,953 | 6.69 s | 63.1 tok/s | 683 | **17.7 s** | 0.30 GB |
| 12k | 12,099 | 9.97 s | 59.4 tok/s | 514 | **19.1 s** | 0.29 GB |

**Decode barely moves with depth: 71.0 → 63.1 → 59.4 tok/s across a 3.6× context increase.**
High context costs prefill time and almost nothing else. TTFT scales cleanly and peak
in-process footprint stays under a third of a gigabyte at every depth measured.

### Where it is weak

On a per-task comparison against the 2.6B on the same machine, the 1.2B wins on speed at every
family — 1.89/1.37 s/row against 3.66/3.06 on enrichment, 8–11 s against 15–19 s on
perspective cards, at the same 10/10 parse and enum-clean, with 6/6 parseable cards and 9/9
real evidence references. It loses decisively on long prose over cited evidence: **0% valid
quotations on the report corpus, against the 2.6B's 67–83%.** Route accordingly.

## Usage

Swift Package Manager, via [CoreAIKit](https://github.com/john-rocky/coreai-kit) — a community
package, not affiliated with Apple, requiring macOS 27 beta:

```swift
.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")
```

`ModelID` addresses a bundle as **repo + path + revision**, where `path` is the subtree in this
repo holding one complete bundle (`metadata.json` + `*.aimodel/` + `tokenizer/`). It downloads
from the Hub on first use and is cached afterwards:

```swift
import CoreAIKit

let model = ModelID(
    "visible-cx/LFM2.5-1.2B-CoreAI",
    path: "ctx8192/gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64")

var config = ChatSession.Configuration()
config.engineVariant = .sequential   // required for guided / grammar-constrained decoding
config.temperature = nil             // greedy

let chat = try await ChatSession(model: model, configuration: config)
for try await event in chat.streamResponse(to: "…") {
    if case .response(let delta) = event { print(delta, terminator: "") }
}
```

Pass `revision:` a Hub commit hash to pin an immutable bundle. `ChatSession(bundleAt:)` loads a
bundle directory already on disk. Leave `COREAI_CHUNK_THRESHOLD` alone — the manifest sets it.

## Integrity

Core AI `.aimodel` bundles are **not byte-reproducible**: the exporter is not deterministic
even against itself. Verify by digesting the exact published bytes rather than by rebuilding.
Every bundle carries `main.hash`, the raw 32 bytes of `sha256(main.mlirb)`, so a downloaded
bundle can be checked against itself; on the Hub the same value is recoverable from the LFS oid
without fetching the file.

## Status

| Artifact | Status |
|---|---|
| `gpu-pipelined/…_mf64` (ctx 4096) | **SHIP** — measured, and output-identical to the stored reference baseline. Requirement 3.61 GiB. |
| `ctx8192/…_mf64` | **QUALIFIED AT DEPTH** — same weights and graph; measured: 3/3 verbatim needle recall at 8,099 tokens, 63.1 tok/s at 8k, 0.30 GB peak footprint. |
| `ctx16384/…_mf64` | **QUALIFIED AT DEPTH** — 3/3 verbatim needle recall at 15,106 tokens, 59.7 tok/s, 0.32 GB peak footprint. |

No PyTorch-reference oracle or PSNR gate has been run. Qualification is behavioural: an exact
diff against a stored baseline, the depth probes above, and the memory instrumentation.

## License

`LiquidAI/LFM2.5-1.2B-Instruct` is released under the **LFM Open License v1.0** (`lfm1.0`), and
upstream declares it as `license: other` + `license_name: lfm1.0`. These bundles are a
derivative and the same licence travels with them — see the
[upstream licence](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE).
Redistribute the licence with the files and comply with its terms. The contribution here is the
conversion and the qualification evidence, not the weights.