Hugging Face's logo

harshav
/
LFM2.5-2.6B-CoreAI

Text Generation
coreai
aimodel
aimodelc
apple-silicon
lfm2
lfm2.5
liquid
edge
int8
community-model-zoo
Model card Files
xet
 Community
LFM2.5-2.6B-CoreAI / README.md
harshav's picture
harshav
Fix provenance and publish reproducible LFM2.5 conversion recipe
90ce993 verified
preview code
|
Raw
History Blame Contribute Delete
13.4 kB
metadata 
license: other
license_name: lfm-open-license-v1.0
license_link: >-
  https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/dca1825886789bd40b94368f53b1d9ada4c94598/LICENSE
base_model: LiquidAI/LFM2.5-2.6B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: coreai
language:
  - ar
  - zh
  - en
  - fr
  - de
  - hi
  - id
  - it
  - ja
  - ko
  - pl
  - pt
  - ru
  - es
  - th
  - vi
tags:
  - coreai
  - aimodel
  - aimodelc
  - apple-silicon
  - lfm2
  - lfm2.5
  - liquid
  - edge
  - int8
  - community-model-zoo

LFM2.5-2.6B — Core AI (int8)

An int8 Core AI conversion of LiquidAI/LFM2.5-2.6B for Apple silicon. This repository contains no trained weights of its own: it is a quantized format conversion of Liquid AI's model, and all model credit belongs to Liquid AI.

Runtime status: runnable with public tooling, but not with an unmodified apple/coreai-models checkout. LFM's fixed-shape convolution state requires two patches from the public, community-maintained john-rocky/coreai-model-zoo. Exact public commits and commands are below.

Converted from source revision dca1825886789bd40b94368f53b1d9ada4c94598. Both upstream safetensors shards were SHA-256 verified against that revision before conversion.

Runtime requirements

This graph has the normal growing key/value cache plus one fixed-shape conv_state. The stock pipelined engine handles only the KV pair. Apply these two public community patches—no per-token-input, static-input, or prefix-cache patch is required for this model:

  1. apps/coreai-shared-product.patch
  2. apps/coreai-pipelined-extra-states.patch

Both come from john-rocky/coreai-model-zoo@95a29d41, a public community repository maintained by Daisuke Majima (john-rocky). It is not an Apple repository. The patches target Apple's public coreai-models runtime at 5ed9981303b38d5a44aa6b45509bc4f6945029f5.

git clone https://github.com/apple/coreai-models.git
git -C coreai-models checkout 5ed9981303b38d5a44aa6b45509bc4f6945029f5

git clone https://github.com/john-rocky/coreai-model-zoo.git
git -C coreai-model-zoo checkout 95a29d41affed3bdf3ea5992ece094a908e21a04

git -C coreai-models apply ../coreai-model-zoo/apps/coreai-shared-product.patch
git -C coreai-models apply ../coreai-model-zoo/apps/coreai-pipelined-extra-states.patch
swift build --package-path coreai-models -c release --product llm-runner

The public community zoo also supplies the LFM2 exporter and its coreai-models overlay. The small toolkit published here contains the 2.6B-specific wrapper, tokenizer/config corrections, quality gates, and locked Python environment used for this artifact. Full reproduction details are in RECIPE.md.

Which file do I want?

The repository root is the portable bundle. The runtime wants a directory containing metadata.json, the *.aimodel/ graph and tokenizer/, and those sit at the top level here. So the recommended download is the root minus the compiled variant — the exact command is under Download.

aimodelc-h16c/ is that same model pre-compiled for one GPU architecture: identical output and speed, about half the cold-load time, and the runtime rejects it on a different architecture. Take it only if you are on h16c and care about load time.

What this is and is not

  • It is a decode-optimized single-token-step graph, the shape a chat/completion loop uses.
  • It is not a chunked-prefill or batch-serving asset.
  • Quality here means teacher-forced top-1 agreement and cosine similarity against an independent fp32 Hugging Face reference, over 5 sequences / 125 positions. That is a regression probe, not a benchmark suite. No MMLU/GSM8K-style numbers are claimed.

Artifacts

Full SHA-256 of each file as published.

File Bytes SHA-256 Recipe Hardware scope Use
lfm2_5_2_6b_decode_int8hu_attnfp16_block32_sym.aimodel/main.mlirb 3469367807 80540b2ee9183b756adb1ce51f334a94f13c093a554a2538aec66f9d9f3a07b3 int8 blockwise-32, fp16 attention + embedding any Apple silicon recommended
aimodelc-h16c/…h16c.aimodelc/…/resources.bin 3468949676 793907c101a01331ffa72aff7c9db5049c89a02ea03acee3174513a93c394e22 same weights, AOT compiled h16c only faster cold load
aimodelc-h16c/…h16c.aimodelc/…/original_model_0.mpsgraph 363905 74a24870a8853bf797a6f12f40e7fcb988de014d98166da772b24cab643d5d4f compiled graph h16c only part of the above
tokenizer/tokenizer.json 17905598 695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41 upstream, unmodified — required
lfm2.5-2.6b-coreai-conversion-e0e375b.tar.gz 57118 665285044be7c0171e1b737167b9892f2799bc2e01e3c5441212887903456f2d exact wrapper + gates at local conversion commit e0e375b — reproduction
  • The two tokenizer/tokenizer.json copies are byte-identical (same hash above). The one under aimodelc-h16c/tokenizer/ is a convenience copy; you do not need both.
  • Portable and compiled are the same model, published together because the compiled one halves cold load but only runs on one architecture, so neither dominates.
  • Both were produced from the same pinned commit and have identical compiled storage budgets.

Recipe, and what was rejected

Oracle for every quality number: an independent fp32 reference, Hugging Face's own Lfm2ForCausalLM at transformers ≥ 5.2, teacher-forced over 5 sequences / 125 positions. Cosine is the minimum per-position cosine.

Variant Bundle Top-1 vs fp32 oracle Min cosine Conversion Shipped
int8 blockwise-32, fp16 attention + embedding 3.25 GB 122/125 0.997050 5/5 yes
+ attention q/k/v/out to int8 3.19 GB 121/125 0.996949 5/5 no
+ embedding to int8 3.03 GB 123/125 0.996848 5/5 no
attention at fp32 (converter default) 3.42 GB — 0.997210 5/5 no
int4 blockwise-32 family 2.07–2.34 GB — 0.51–0.80 4–5/5 no

The shipped arm has the highest minimum cosine of the int8 arms. The ±1 position differences between the three int8 rows are near-tie argmax flips at 125 positions, not a systematic ordering — do not read the embedding row's 123 as "better". int4 is excluded on quality, not size: a minimum cosine of 0.51–0.80 is a different model. Rejected variants are documented and deliberately not uploaded. Full detail in RECIPE.md.

Performance

Decode here is memory-bandwidth bound — throughput tracks bytes read per token.

Measurement Protocol Result
This published artifact 128 prompt tokens, 256 generated, greedy, 5 trials, M4 Max 107.98 decode tok/s, sd 0.500; 111.86 prompt tok/s
Earlier development build of the same recipe separate runs on the same M4 Max 138.15 decode tok/s

Only 107.98 tok/s is attached to the published artifact in evidence/benchmark.json. The earlier 138.15 result was not reproduced after publication, so it is retained as historical context rather than advertised as this download's performance. The workload is memory-bandwidth-bound and was measured while the machine was busy; measure on your own hardware.

For reference, on the same Mac, MLX reported ~100 tok/s at 8-bit and ~60 tok/s at BF16. Those were throughput-only observations; MLX quality was not measured, so no quality comparison against MLX is claimed.

AOT compilation is throughput-neutral within noise and halves cold load, 9.8 s → 5.1 s.

Reproduce

Every pin, the full quantization spec, the four gates, the measurement protocol and the rejected variants are in RECIPE.md. Summary:

Source model : LiquidAI/LFM2.5-2.6B @ dca1825886789bd40b94368f53b1d9ada4c94598          (public)
Converter    : github.com/apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8  (public)
Runtime      : github.com/apple/coreai-models @ 5ed9981303b38d5a44aa6b45509bc4f6945029f5  (public)
Community LFM2 exporter, overlay and two runtime patches:
               github.com/john-rocky/coreai-model-zoo
               @ 95a29d41affed3bdf3ea5992ece094a908e21a04                     (public)
2.6B wrapper : lfm2.5-2.6b-coreai-conversion-e0e375b.tar.gz
               SHA-256 665285044be7c0171e1b737167b9892f2799bc2e01e3c5441212887903456f2d
Toolchain    : macOS 27.0 (26A5388g), Xcode 27.0 (27A5228h), Python 3.11.15,
               torch 2.9.0, coreai-core 1.0.0b2, coreai-torch 0.4.1,
               coreai-opt 0.2.1, coremltools 9.0

Verify a rebuild by the gates and the storage budget, not by hashing. The exporter names each externalized call site with a generated UUID (391 in this graph), so two exports of identical weights differ in a few bytes and therefore in SHA-256. The budget to match is Int8 2,621,243,392 + Float16 428,342,276 + Float32 34 + small index types.

Download 

# The portable bundle: the .aimodel graph, the tokenizer and the bundle metadata.
# This is the whole repository except the compiled h16c variant (~3.5 GB rather than ~7 GB).
hf download harshav/LFM2.5-2.6B-CoreAI \
  --exclude 'aimodelc-h16c/*' \
  --local-dir ./LFM2.5-2.6B-CoreAI

# The directory you then point the runtime at is ./LFM2.5-2.6B-CoreAI itself.

With a runtime built as described above, the asset is driven as a pipelined Core AI language model with COREAI_CHUNK_THRESHOLD=1, greedy decoding, using the model's own chat template at tokenizer/chat_template.jinja. Operational notes:

COREAI_CHUNK_THRESHOLD=1 coreai-models/.build/release/llm-runner \
  --model ./LFM2.5-2.6B-CoreAI \
  --prompt "The capital of France is" \
  --max-tokens 64 --temperature 0 \
  --inference-engine-variant coreai-pipelined \
  --warmup exact --warmup-length 1
  • macOS 27.0+ on Apple silicon. Validated on M4 Max.
  • Runs on the GPU via an MPSGraph delegate. This is not an ANE asset: the KV dimension is dynamic, and --preferred-compute does not change the emitted delegate.
  • Budget roughly bundle size plus KV cache, about 5 GB at 4096 context.
  • aimodelc-h16c/ is architecture-locked to h16c; the runtime names the architecture it wanted when it refuses.

Validation evidence

Machine-readable under evidence/: authored_parity_vs_huggingface.json, fp32_reference.json, recipe_quality.json, conversion_gate.json, benchmark.json, compiled_storage_stats.json.

Four separate questions, not interchangeable:

  • Authoring fidelity — re-authored module vs Hugging Face, fp32: 21/21 top-1, cosine 1.000000. This caught a real bug: the checkpoint sets rope_parameters.rope_theta = 1e7, and code reading only the legacy top-level key silently defaults to 1e6 — a 10× wrong RoPE that still produces fluent short text. Both the converter overlay and transformers 4.x hit it.
  • Quantization damage — vs the fp32 oracle: 122/125, min cosine 0.997050.
  • Conversion fidelity — bundle vs its own quantized weights run eagerly: 5/5 exact. Comparing to fp32 here would conflate quantization damage with conversion bugs.
  • Throughput — see the caveat above.

Limitations and negative results

  • Requires the two public community runtime patches listed above; it does not run on an unmodified apple/coreai-models runtime.
  • No task-benchmark evaluation; quality is a 125-position regression probe.
  • Multilingual support is inherited from upstream and was not re-verified per language; the probe is English.
  • int4 rejected at block 32 (cosine 0.51–0.66) and at block 16 (quality recovers, 42 tok/s, ~3× slower than int8).
  • --expect-frequent-reshapes measured 84 tok/s and 8.3 GB; not used.
  • Speculative decoding not shipped: a static-S verify graph exports and its contract gates, but per-position logits do not match stepped decode.
  • 300 tok/s was a target and was not reached by any tested configuration.

License and attribution

The model is Liquid AI's. This repository redistributes a converted, quantized copy under the upstream license, and claims no authorship of the model.

  • Upstream: LiquidAI/LFM2.5-2.6B by Liquid AI.
  • Upstream license: LFM Open License v1.0 — pinned copy here, included verbatim as LICENSE.upstream. Your use of these weights is governed by it.
  • Official Apple tooling: apple/coreai-models, coreai-core, coreai-torch, and coreai-opt.
  • Community conversion/runtime work: public john-rocky/coreai-model-zoo, maintained by Daisuke Majima, under its BSD-3-Clause license. Its repository includes third-party notices for portions derived from Apple's BSD-licensed coreai-models project.
  • This repository contributes the conversion recipe, the gates, and the measurements.