README.md

metadata

license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-2.6B
base_model_relation: quantized
library_name: coreai
pipeline_tag: text-generation
tags:
  - core-ai
  - aimodel
  - apple-silicon
  - on-device
  - coreai-kit
  - quantized
  - int8
  - lfm2.5

LFM2.5-2.6B — Core AI (.aimodel)

LiquidAI/LFM2.5-2.6B converted to Core AI .aimodel bundles for Apple silicon by visible-cx. These are derivative artifacts: Liquid AI's weights re-expressed as a Core AI graph with int8 block-32 symmetric weight quantization and a two-entrypoint (decode + chunked-prefill) function map. They load through Core AI on macOS and are not usable by PyTorch, GGUF or MLX. This is the model the Visible app routes enrichment to — per-item labelling and tagging, where comprehension on argumentative text matters and a few seconds per item is acceptable.

Of the dense LFM2.5 bundles published here, this is the strongest on guided structured-output work, the only one qualified for long context, and the best-grounded local model this project has measured: 3/3 verbatim needle recall at 14,566 tokens, and zero invented attributors on the real graph.

Contents

Path	Bytes	Manifest context	Functions
gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64	3,655,243,493	4096	main + prefill
ctx8192/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64	3,655,243,501	8192	main + prefill
ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64	3,655,243,498	16384	main + prefill

Each folder holds <name>.aimodel/ (main.mlirb ≈ 3.64 GB, main.hash, asset metadata.json), a bundle-level metadata.json, and tokenizer/ (tokenizer.json, tokenizer_config.json, generation_config.json, chat_template.jinja).

The three folders hold the same weights and the same graph — function signatures, state descriptors and peak export RSS are identical at 4096, 8192 and 16384. --max-ctx changes exactly one thing: language.max_context_length in the bundle manifest. The small byte differences between folders are conversion nondeterminism, not content. Pick the folder whose manifest integer matches the window you intend to run.

Stop token: eos_token = "<|im_end|>" in all three folders. Clean self-stop on every measured sample.

Chat template. LFM2.5-2.6B is an always-thinking model. The template shipped in every bundle here terminates the reasoning block in the generation prompt:

{{- "<|im_start|>assistant\n<think></think>\n" -}}

An unterminated block (…assistant\n<think>) causes the model to spend the entire generation budget inside <think>, which a host routes to a reasoning channel and never to the response — 684–919 tokens per item, with no visible output. If you rebuild from the recipe, apply the same termination.

Provenance

Base checkpoint	LiquidAI/LFM2.5-2.6B
Recipe	export_lfm2_multifunction.py int8hu --head-sym --chunk 64
Toolchain base	apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8 + the coreai-model-zoo python overlay
Toolchain	coreai-torch 0.4.1, coreai-core 1.0.0b2, coreai-opt 0.2.1, torch 2.9.0
Producer fingerprint	coreai-core 1.0.0b2 on every inner <name>.aimodel/metadata.json
Weight format	int8, per-K-block-32, symmetric; symmetric head (--head-sym)
Vocab	128,000
Export functions	main (S=1 decode) + prefill (S=64 chunked prefill), function_map: {"main": ["main", "prefill"]}, weights deduplicated across entrypoints

mf64 in the bundle name means multifunction with a 64-wide prefill; the prefill function costs well under a megabyte.

The symmetric head is not incidental. Measured on the sibling 8B bundle in this org, an affine head makes the compiler materialise two dequantised fp16 transposes of the whole vocab × hidden matrix — a gigabyte of graph constant that is never read. A symmetric dequantize is a scale multiply the GPU delegate folds into the matmul.

Requirements

• Apple silicon Mac, Core AI runtime.
• Engine contract: 2 inputs. input_ids, position_ids → logits. No static inputs, no per-step mask. Runs on both the pipelined engine and the sequential (logits-capable) engine, which is what makes grammar-constrained decoding available.
• States: keyCache / valueCache Float16, 8 × 1 × 8 × ? × 64 plus convState Float16, 22 × 1 × 2048 × 2. The sequence dim is dynamic, so the runtime resolves a GrowingKVCache (initial capacity 256, doubling) rather than allocating the manifest maximum up front. convState is fixed-size and does not scale with context.
• KV cost: 16,384 bytes per token of context (fp16) — 67 MB at 4096, 134 MB at 8192, 268 MB at 16384. KV is not the binding constraint at any context this bundle declares.
• Minimum practical machine memory: 16 GB, at any declared context including 16384.
• The bundle manifest declares runtime_env COREAI_CHUNK_THRESHOLD=1.

Measurements

Measured on a 16 GB Apple silicon Mac (M2 Pro, macOS 27 beta).

Guided structured output

10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, sequential engine, reset() between samples, 128-token cap. Load excluded from s/row; sample 1 excluded as a cache-warm outlier.

Cold load	18.4 s
Guided JSON parse	10/10
Enum-clean	10/10
s/row (long samples)	3.66
s/row (short samples)	3.06
Decode	38.1–40.0 tok/s
TTFT	0.47–2.29 s
Peak footprint	0.48 GB
Max RSS	6.71 GB

For scale on the same machine and harness: LFM2.5-350M runs 0.84/0.55 s/row and LFM2.5-1.2B 1.89/1.37 s/row. The 2.6B is ≈2× the 1.2B, which is what its parameter count predicts, and it produces the most specific free-text fields of the three.

Memory, measured rather than inferred

Max RSS is a resident set, and a resident set counts clean mapped pages the kernel can drop for free — so it is not what the machine has to give up. Measured with an external watchdog sampling wired memory, on the bundle the app pins:

	GiB
bundle on disk	3.404
compiled blob	4.185
graph constant	0.797
blob ÷ bundle	1.23×
wired, completed trace (3 legs)	4.300
forecast (blob × 1.106)	4.628
requirement (peak + 1.25 GiB in-flight floor)	6.03

This is the only completed trace this project has that checks the plateau law against a run that finished, and the law reads 7.6% high — i.e. conservative, on the safe side. Wired ÷ blob for this bundle is 0.96.

Long context

Needle-in-haystack: 3/3 verbatim at both 8k and 15k. Three distinctive facts planted at 10% / 50% / 90% of the filler, strict scoring (a fact counts only if the distinctive entity comes back correct). All three returned at 7,813 tokens and all three at 14,566 tokens, verbatim, including the date.

probe	prompt tokens	TTFT	decode	wall	peak footprint
needle 8k	7,813	13.78 s	35.8 tok/s	15.4 s	0.34 GB
needle 15k	14,566	27.42 s	32.2 tok/s	29.1 s	0.46 GB

Free-form generation from a fixed prompt at three depths, 900-token cap (the model self-stopped inside it every time):

depth	prompt tokens	TTFT	decode	generated	wall	peak footprint
3.4k	3,249	6.88 s	35.3 tok/s	847	29.5 s	0.28 GB
8k	7,673	14.84 s	36.6 tok/s	715	36.0 s	0.47 GB
12k	11,643	22.24 s	32.9 tok/s	743	44.4 s	0.46 GB

Decode barely moves with depth — 40.0 tok/s at 2.3k → 32.9 at 11.6k, an 18% decay across a 5× context increase — while peak footprint stays flat. High context costs prefill time and almost nothing else.

Grounding

Three real report questions on a real knowledge graph, full scorer, watchdog attached:

question	prompt tokens	generated	tok/s	inversion	invented attributor	unsupported spans	narration	3rd-person refs	JSON
reputationalRisks	5,306	228	38.4	0	0	0/0	0%	0	ok
headline	3,096	26	39.1	0	0	0/0	0%	0	ok
profileSummary	4,444	140	38.0	0	0	0/0	0%	0	ok

Zero invented attributors on the real graph — the defect the prompt recipe was built to kill, and the one a synthetic corpus could not produce. One scorer flag was hand-adjudicated and dismissed as a false positive: the "motive bait" list fired on the model quoting the subject's own words verbatim from the evidence window, and attributing a statement is not asserting an inner state.

Across a wider comparison this bundle holds the highest report quotation validity of any local model measured here — 67–83% real quotations, against 0–29% for the smaller dense LFMs, and it is the only local model that declines to answer rather than inventing one.

Usage

Swift Package Manager, via CoreAIKit — a community package, not affiliated with Apple, requiring macOS 27 beta:

.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")

ModelID addresses a bundle as repo + path + revision, where path is the subtree in this repo holding one complete bundle (metadata.json + *.aimodel/ + tokenizer/). It downloads from the Hub on first use and is cached afterwards:

import CoreAIKit

let model = ModelID(
    "visible-cx/LFM2.5-2.6B-CoreAI",
    path: "ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64")

var config = ChatSession.Configuration()
config.engineVariant = .sequential   // required for guided / grammar-constrained decoding
config.temperature = nil             // greedy

let chat = try await ChatSession(model: model, configuration: config)
for try await event in chat.streamResponse(to: "…") {
    if case .response(let delta) = event { print(delta, terminator: "") }
}

Pass revision: a Hub commit hash to pin an immutable bundle. ChatSession(bundleAt:) loads a bundle directory already on disk. Leave COREAI_CHUNK_THRESHOLD alone — the manifest sets it.

Integrity

Core AI .aimodel bundles are not byte-reproducible: the exporter is not deterministic even against itself, and two runs of the same command on the same host differ by a few dozen bytes. Verify by digesting the exact published bytes rather than by rebuilding. Every bundle carries main.hash, the raw 32 bytes of sha256(main.mlirb), so a downloaded bundle can be checked against itself; on the Hub the same value is recoverable from the LFS oid without fetching the file.

Status

Artifact	Status
gpu-pipelined/…_mf64 (ctx 4096)	SHIP — measured: 10/10 guided parse and enum-clean, 3.66/3.06 s/row, 38.1–40.0 tok/s, requirement 6.03 GiB from a completed wired trace.
ctx8192/…_mf64	QUALIFIED AT DEPTH — same weights and graph; measured: 3/3 verbatim needle recall at 7,813 tokens, 36.6 tok/s at 8k, 0.47 GB peak footprint.
ctx16384/…_mf64	QUALIFIED AT DEPTH — 3/3 verbatim needle recall at 14,566 tokens at 32.2 tok/s, 0.46 GB peak footprint, plus the 3/3-clean real-graph grounding leg above. Recommended for long-context work.

No oracle or PSNR gate has been run against a PyTorch reference. Qualification is behavioural (parse rate, enum conformance, grounding scoring, clean stop, needle recall) plus the memory instrumentation, not a numerics gate.

License

LiquidAI/LFM2.5-2.6B is released under the LFM Open License v1.0 (lfm1.0), and upstream declares it as license: other + license_name: lfm1.0. These bundles are a derivative of that checkpoint and the same licence and its obligations travel with them — see the upstream licence. Anyone redistributing these files should redistribute the licence with them and comply with its terms. Nothing here relicenses Liquid AI's weights; the contribution is the conversion recipe and the qualification evidence.