README.md

metadata

license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-1.2B-Instruct
base_model_relation: quantized
library_name: coreai
pipeline_tag: text-generation
tags:
  - core-ai
  - aimodel
  - apple-silicon
  - on-device
  - coreai-kit
  - quantized
  - int8
  - lfm2.5

LFM2.5-1.2B-Instruct — Core AI (.aimodel)

LiquidAI/LFM2.5-1.2B-Instruct converted to Core AI .aimodel bundles for Apple silicon by visible-cx. These are derivative artifacts: Liquid AI's weights re-expressed as a Core AI graph with int8 block-32 symmetric weight quantization and a two-entrypoint (decode + chunked-prefill) function map. They load through Core AI on macOS and are not usable by PyTorch, GGUF or MLX.

It is the speed tier of the dense LFM2.5 bundles in this org: 1.89 s per guided structured-output row at 71–75 tok/s in 3.42 GB of resident memory, with 3/3 verbatim needle recall at 15k tokens. It runs comfortably on an 8 GB Mac, which the 2.6B does not.

Use it for structured extraction and short cards, not for long prose over cited evidence. On the same eleven-question report corpus that this org's 2.6B answered with 67–83% real quotations, the 1.2B's prose quotations measured 0% valid — it produces fluent text with quotation marks around material that is not in the evidence. That is why the Visible app's routing table keeps it off report work regardless of how fast it is.

Contents

Path	Bytes	Manifest context	Functions
gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64	1,702,039,167	4096	main + prefill
ctx8192/gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64	1,702,039,171	8192	main + prefill
ctx16384/gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64	1,702,039,167	16384	main + prefill

Each folder holds <name>.aimodel/ (main.mlirb ≈ 1.70 GB, main.hash, asset metadata.json), a bundle-level metadata.json, and tokenizer/ (tokenizer.json, tokenizer_config.json, special_tokens_map.json, chat_template.jinja).

The three folders hold the same weights and the same graph — identical function signatures, identical state descriptors, identical export peak RSS at 4096/8192/16384. --max-ctx sets one manifest integer, language.max_context_length, and nothing else. The folders exist so each declared window is a clean, fingerprinted artifact rather than a hand-edited manifest.

Stop token: eos_token = "<|im_end|>" in all three folders. Clean self-stop on every measured sample.

This model is not a thinking model and needs no chat-template adjustment — its template ends at <|im_start|>assistant\n.

Provenance

Base checkpoint	LiquidAI/LFM2.5-1.2B-Instruct
Recipe	export_lfm2_multifunction.py int8hu --head-sym --chunk 64
Toolchain base	apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8 + the coreai-model-zoo python overlay
Toolchain	coreai-torch 0.4.1, coreai-core 1.0.0b2, coreai-opt 0.2.1, torch 2.9.0
Producer fingerprint	coreai-core 1.0.0b2 on every inner <name>.aimodel/metadata.json
Weight format	int8, per-K-block-32, symmetric; symmetric head (--head-sym)
Vocab	65,536
Export functions	main (S=1 decode) + prefill (S=64 chunked prefill), function_map: {"main": ["main", "prefill"]}, weights shared

mf64 in the bundle name means multifunction with a 64-wide prefill.

The symmetric head is not incidental. Measured on the sibling 8B bundle in this org, an affine head makes the compiler materialise dequantised fp16 transposes of the whole vocab × hidden matrix — graph constant that is never read. A symmetric dequantize is a scale multiply the GPU delegate folds into the matmul.

Requirements

• Apple silicon Mac, Core AI runtime.
• Engine contract: 2 inputs (input_ids, position_ids) → logits. No static inputs, no per-step mask. Runs on the pipelined engine and on the sequential engine, which is the only logits-capable one and therefore the only path for grammar-constrained decoding.
• States: keyCache / valueCache Float16, 6 × 1 × 8 × ? × 64 plus convState Float16, 10 × 1 × 2048 × 2. Dynamic sequence dim → GrowingKVCache (initial 256, doubling), not a static allocation at the manifest maximum. convState does not scale with context.
• KV cost: 12,288 bytes per token (fp16) — 50 MB at 4096, 101 MB at 8192, 201 MB at 16384. KV is nowhere near binding at any context this bundle declares.
• Minimum practical machine memory: 8 GB.
• The bundle manifest declares runtime_env COREAI_CHUNK_THRESHOLD=1.

Measurements

Measured on a 16 GB Apple silicon Mac (M2 Pro, macOS 27 beta).

Guided structured output

10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, sequential engine, reset() between samples, 128-token cap. Load excluded from s/row; sample 1 excluded as a cache-warm outlier.

Cold load	5.4 s
Guided JSON parse	10/10
Enum-clean	10/10
s/row (long samples)	1.89
s/row (short samples)	1.37
Decode	71.5–74.7 tok/s
TTFT	0.37–1.60 s
Peak footprint	0.14 GB
Max RSS	3.42 GB

Output-identical to a stored reference baseline. All ten guided outputs diff clean against it — every character of every field, in order. That equality is behavioural and exact; it does not mean byte-identical weights, since .aimodel conversion is not byte-deterministic.

Memory, measured rather than inferred

Max RSS counts clean mapped pages the kernel can drop for free, so it is not what the machine has to give up. Measured with an external watchdog sampling wired memory:

	GiB
bundle on disk	1.585
compiled blob	2.128
graph constant	0.547
blob ÷ bundle	1.34×
wired floor (32-token leg — a lower bound, not a plateau)	2.190
forecast (blob × 1.106)	2.353
requirement (peak + 1.25 GiB in-flight floor)	3.61

The measured floor and the forecast agree to 0.3% — 2.360 against 2.353 — from two instruments with nothing in common. Wired ÷ blob for this bundle is 1.03.

Long context

Needle-in-haystack: 3/3 verbatim at both 8k and 15k. Three distinctive facts planted at 10% / 50% / 90% of the filler, strict scoring (a fact counts only if the distinctive entity comes back correct). All three returned at 8,099 tokens and all three at 15,106 tokens, verbatim, including the date.

probe	prompt tokens	TTFT	decode	wall	peak footprint
needle 8k	8,099	6.98 s	64.5 tok/s	8.3 s	0.21 GB
needle 15k	15,106	12.79 s	59.7 tok/s	14.4 s	0.32 GB

Free-form generation from a fixed prompt at three depths, 900-token cap (the model self-stopped inside it every time):

depth	prompt tokens	TTFT	decode	generated	wall	peak footprint
3.4k	3,371	3.22 s	71.0 tok/s	361	8.5 s	0.16 GB
8k	7,953	6.69 s	63.1 tok/s	683	17.7 s	0.30 GB
12k	12,099	9.97 s	59.4 tok/s	514	19.1 s	0.29 GB

Decode barely moves with depth: 71.0 → 63.1 → 59.4 tok/s across a 3.6× context increase. High context costs prefill time and almost nothing else. TTFT scales cleanly and peak in-process footprint stays under a third of a gigabyte at every depth measured.

Where it is weak

On a per-task comparison against the 2.6B on the same machine, the 1.2B wins on speed at every family — 1.89/1.37 s/row against 3.66/3.06 on enrichment, 8–11 s against 15–19 s on perspective cards, at the same 10/10 parse and enum-clean, with 6/6 parseable cards and 9/9 real evidence references. It loses decisively on long prose over cited evidence: 0% valid quotations on the report corpus, against the 2.6B's 67–83%. Route accordingly.

Usage

Swift Package Manager, via CoreAIKit — a community package, not affiliated with Apple, requiring macOS 27 beta:

.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")

ModelID addresses a bundle as repo + path + revision, where path is the subtree in this repo holding one complete bundle (metadata.json + *.aimodel/ + tokenizer/). It downloads from the Hub on first use and is cached afterwards:

import CoreAIKit

let model = ModelID(
    "visible-cx/LFM2.5-1.2B-CoreAI",
    path: "ctx8192/gpu-pipelined/lfm2_5_1_2b_instruct_decode_int8hu_block32_sym_mf64")

var config = ChatSession.Configuration()
config.engineVariant = .sequential   // required for guided / grammar-constrained decoding
config.temperature = nil             // greedy

let chat = try await ChatSession(model: model, configuration: config)
for try await event in chat.streamResponse(to: "…") {
    if case .response(let delta) = event { print(delta, terminator: "") }
}

Pass revision: a Hub commit hash to pin an immutable bundle. ChatSession(bundleAt:) loads a bundle directory already on disk. Leave COREAI_CHUNK_THRESHOLD alone — the manifest sets it.

Integrity

Core AI .aimodel bundles are not byte-reproducible: the exporter is not deterministic even against itself. Verify by digesting the exact published bytes rather than by rebuilding. Every bundle carries main.hash, the raw 32 bytes of sha256(main.mlirb), so a downloaded bundle can be checked against itself; on the Hub the same value is recoverable from the LFS oid without fetching the file.

Status

Artifact	Status
gpu-pipelined/…_mf64 (ctx 4096)	SHIP — measured, and output-identical to the stored reference baseline. Requirement 3.61 GiB.
ctx8192/…_mf64	QUALIFIED AT DEPTH — same weights and graph; measured: 3/3 verbatim needle recall at 8,099 tokens, 63.1 tok/s at 8k, 0.30 GB peak footprint.
ctx16384/…_mf64	QUALIFIED AT DEPTH — 3/3 verbatim needle recall at 15,106 tokens, 59.7 tok/s, 0.32 GB peak footprint.

No PyTorch-reference oracle or PSNR gate has been run. Qualification is behavioural: an exact diff against a stored baseline, the depth probes above, and the memory instrumentation.

License

LiquidAI/LFM2.5-1.2B-Instruct is released under the LFM Open License v1.0 (lfm1.0), and upstream declares it as license: other + license_name: lfm1.0. These bundles are a derivative and the same licence travels with them — see the upstream licence. Redistribute the licence with the files and comply with its terms. The contribution here is the conversion and the qualification evidence, not the weights.