harshav/LFM2.5-2.6B-CoreAI

LFM2.5-2.6B — Core AI (int8) — NOT RUNNABLE WITH PUBLIC TOOLING

An int8 Core AI conversion of LiquidAI/LFM2.5-2.6B for Apple silicon. This repository contains no trained weights of its own: it is a quantized format conversion of Liquid AI's model, and all model credit belongs to Liquid AI.

Status: artifact archive, not a runnable release. These are real, gated, verifiable weights, but executing them requires a Core AI runtime patch stack that lives in a non-public Apple repository. If you do not have access to apple/coreai-model-zoo, you can download, hash-verify, inspect and study this conversion — you cannot run it. Details in the next section.

Converted from source revision dca1825886789bd40b94368f53b1d9ada4c94598. Both upstream safetensors shards were SHA-256 verified against that revision before conversion.

Read this first: running it needs a runtime you may not be able to get

This asset will not run on a stock Core AI runtime. It requires a patch stack that lives in Apple's coreai-model-zoo, which is not a public repository. Specifically:

• The graph uses a per-token input contract (input_ids [1,1] plus explicit KV state) that an unpatched runtime at the pinned commit rejects. The patches involved are named coreai-pipelined-per-token-inputs, -static-inputs, -extra-states, coreai-prefix-cache and coreai-shared-product.
• Re-exporting additionally needs that repository's LFM2 overlay, which is what teaches the converter this architecture at all.

Those patches are Apple's, not mine, so they are not redistributed here and this repository cannot make them available to you. Practically:

• With access to coreai-model-zoo: everything needed to build the runtime and reproduce the conversion is pinned in RECIPE.md.
• Without it: you can download and inspect these weights, read every measurement, and reuse the recipe — but you will not be able to execute the asset today. That is a real limitation, stated here rather than buried.

For the same reason no conversion toolkit is shipped: those scripts import modules from that non-public repo, so publishing them would either redistribute code that is not mine or hand you something that cannot run. RECIPE.md describes the recipe precisely instead.

Which file do I want?

The repository root is the portable bundle. The runtime wants a directory containing metadata.json, the *.aimodel/ graph and tokenizer/, and those sit at the top level here. So the recommended download is the root minus the compiled variant — the exact command is under Download.

aimodelc-h16c/ is that same model pre-compiled for one GPU architecture: identical output and speed, about half the cold-load time, and the runtime rejects it on a different architecture. Take it only if you are on h16c and care about load time.

What this is and is not

• It is a decode-optimized single-token-step graph, the shape a chat/completion loop uses.
• It is not a chunked-prefill or batch-serving asset.
• Quality here means teacher-forced top-1 agreement and cosine similarity against an independent fp32 Hugging Face reference, over 5 sequences / 125 positions. That is a regression probe, not a benchmark suite. No MMLU/GSM8K-style numbers are claimed.

Artifacts

Full SHA-256 of each file as published.

File	Bytes	SHA-256	Recipe	Hardware scope	Use
lfm2_5_2_6b_decode_int8hu_attnfp16_block32_sym.aimodel/main.mlirb	3469367807	80540b2ee9183b756adb1ce51f334a94f13c093a554a2538aec66f9d9f3a07b3	int8 blockwise-32, fp16 attention + embedding	any Apple silicon	recommended
aimodelc-h16c/…h16c.aimodelc/…/resources.bin	3468949676	793907c101a01331ffa72aff7c9db5049c89a02ea03acee3174513a93c394e22	same weights, AOT compiled	h16c only	faster cold load
aimodelc-h16c/…h16c.aimodelc/…/original_model_0.mpsgraph	363905	74a24870a8853bf797a6f12f40e7fcb988de014d98166da772b24cab643d5d4f	compiled graph	h16c only	part of the above
tokenizer/tokenizer.json	17905598	695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41	upstream, unmodified	—	required

• The two tokenizer/tokenizer.json copies are byte-identical (same hash above). The one under aimodelc-h16c/tokenizer/ is a convenience copy; you do not need both.
• Portable and compiled are the same model, published together because the compiled one halves cold load but only runs on one architecture, so neither dominates.
• Both were produced from the same pinned commit and have identical compiled storage budgets.

Recipe, and what was rejected

Oracle for every quality number: an independent fp32 reference, Hugging Face's own Lfm2ForCausalLM at transformers ≥ 5.2, teacher-forced over 5 sequences / 125 positions. Cosine is the minimum per-position cosine.

Variant	Bundle	Top-1 vs fp32 oracle	Min cosine	Conversion	Shipped
int8 blockwise-32, fp16 attention + embedding	3.25 GB	122/125	0.997050	5/5	yes
+ attention q/k/v/out to int8	3.19 GB	121/125	0.996949	5/5	no
+ embedding to int8	3.03 GB	123/125	0.996848	5/5	no
attention at fp32 (converter default)	3.42 GB	—	0.997210	5/5	no
int4 blockwise-32 family	2.07–2.34 GB	—	0.51–0.80	4–5/5	no

The shipped arm has the highest minimum cosine of the int8 arms. The ±1 position differences between the three int8 rows are near-tie argmax flips at 125 positions, not a systematic ordering — do not read the embedding row's 123 as "better". int4 is excluded on quality, not size: a minimum cosine of 0.51–0.80 is a different model. Rejected variants are documented and deliberately not uploaded. Full detail in RECIPE.md.

Performance, and an unresolved caveat

Decode here is memory-bandwidth bound — throughput tracks bytes read per token.

Measurement	Protocol	Result
Development runs of this recipe	cold cache + 60 s settle, 5 trials × 3 independent runs, quiet machine	138.15 tok/s, between-run spread 0.31 %
Re-measurement of this published artifact	same protocol, 5 trials	107.98 tok/s, sd 0.500 — machine demonstrably busy: load average 9–17, background disk 50–1400 MB/s

Both are real; neither is cherry-picked; the gap is not explained. Established: this artifact's storage budget is byte-identical to the one measured at 138, and its quality and conversion gate reproduce exactly, so these are measurements of the same recipe rather than of two different models. Not established: the cause. A bandwidth-bound workload losing throughput to competing memory traffic is the obvious candidate, but the machine never went quiet again during the session, so it was never isolated and another regression cannot be ruled out.

Treat 138 as what this recipe has done on an idle M4 Max and 108 as what it did under the stated load, and measure on your own hardware rather than trusting either.

For reference, on the same Mac, MLX reported ~100 tok/s at 8-bit and ~60 tok/s at BF16. Those were throughput-only observations; MLX quality was not measured, so no quality comparison against MLX is claimed.

AOT compilation is throughput-neutral within noise and halves cold load, 9.8 s → 5.1 s.

Reproduce

Every pin, the full quantization spec, the four gates, the measurement protocol and the rejected variants are in RECIPE.md. Summary:

Source model : LiquidAI/LFM2.5-2.6B @ dca1825886789bd40b94368f53b1d9ada4c94598          (public)
Converter    : github.com/apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8  (public)
Runtime      : github.com/apple/coreai-models @ 5ed9981303b38d5a44aa6b45509bc4f6945029f5  (public)
coreai-torch : c89f6a44713249a12a84beec9f3e0cf2206ecc38                                   (public)
LFM2 overlay + runtime patch stack :
               apple/coreai-model-zoo @ ebef921a1f358af66c9ff67e8c6e7d4e24efad0d      (NOT public)
Toolchain    : macOS 27.0 (26A5388g), Xcode 27.0 (27A5228h), Python 3.11.15,
               torch 2.9.0, coremltools 9.0

Verify a rebuild by the gates and the storage budget, not by hashing. The exporter names each externalized call site with a generated UUID (391 in this graph), so two exports of identical weights differ in a few bytes and therefore in SHA-256. The budget to match is Int8 2,621,243,392 + Float16 428,342,276 + Float32 34 + small index types.

Download

# The portable bundle: the .aimodel graph, the tokenizer and the bundle metadata.
# This is the whole repository except the compiled h16c variant (~3.5 GB rather than ~7 GB).
hf download harshav/LFM2.5-2.6B-CoreAI \
  --exclude 'aimodelc-h16c/*' \
  --local-dir ./LFM2.5-2.6B-CoreAI

# The directory you then point the runtime at is ./LFM2.5-2.6B-CoreAI itself.

With a runtime built as described above, the asset is driven as a pipelined Core AI language model with COREAI_CHUNK_THRESHOLD=1, greedy decoding, using the model's own chat template at tokenizer/chat_template.jinja. Operational notes:

• macOS 27.0+ on Apple silicon. Validated on M4 Max.
• Runs on the GPU via an MPSGraph delegate. This is not an ANE asset: the KV dimension is dynamic, and --preferred-compute does not change the emitted delegate.
• Budget roughly bundle size plus KV cache, about 5 GB at 4096 context.
• aimodelc-h16c/ is architecture-locked to h16c; the runtime names the architecture it wanted when it refuses.

Validation evidence

Machine-readable under evidence/: authored_parity_vs_huggingface.json, fp32_reference.json, recipe_quality.json, conversion_gate.json, benchmark.json, compiled_storage_stats.json.

Four separate questions, not interchangeable:

• Authoring fidelity — re-authored module vs Hugging Face, fp32: 21/21 top-1, cosine 1.000000. This caught a real bug: the checkpoint sets rope_parameters.rope_theta = 1e7, and code reading only the legacy top-level key silently defaults to 1e6 — a 10× wrong RoPE that still produces fluent short text. Both the converter overlay and transformers 4.x hit it.
• Quantization damage — vs the fp32 oracle: 122/125, min cosine 0.997050.
• Conversion fidelity — bundle vs its own quantized weights run eagerly: 5/5 exact. Comparing to fp32 here would conflate quantization damage with conversion bugs.
• Throughput — see the caveat above.

Limitations and negative results

• Not runnable without the non-public patch stack. See the top of this card.
• No task-benchmark evaluation; quality is a 125-position regression probe.
• Multilingual support is inherited from upstream and was not re-verified per language; the probe is English.
• int4 rejected at block 32 (cosine 0.51–0.66) and at block 16 (quality recovers, 42 tok/s, ~3× slower than int8).
• --expect-frequent-reshapes measured 84 tok/s and 8.3 GB; not used.
• Speculative decoding not shipped: a static-S verify graph exports and its contract gates, but per-position logits do not match stepped decode.
• 300 tok/s was a target and was not reached by any tested configuration.

License and attribution

The model is Liquid AI's. This repository redistributes a converted, quantized copy under the upstream license, and claims no authorship of the model.

• Upstream: LiquidAI/LFM2.5-2.6B by Liquid AI.
• Upstream license: LFM Open License v1.0 — pinned copy here, included verbatim as LICENSE.upstream. Your use of these weights is governed by it.
• Conversion tooling: Apple's coreai-models, coreai-torch and coreai-model-zoo, at the commits pinned above. No Apple source is redistributed here.
• This repository contributes the conversion recipe, the gates, and the measurements.