File size: 2,180 Bytes
13c230b ad055cf 13c230b 3ed78cb ad055cf 13c230b 3ed78cb 13c230b 3ed78cb 13c230b 130936b 13c230b 130936b 13c230b 130936b 13c230b 67e428f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | ---
license: other
tags:
- executorch
- xnnpack
- pte
- on-device
- text-generation
base_model:
- LiquidAI/LFM2.5-1.2B-Instruct
---
# LFM2.5-1.2B-Instruct — ExecuTorch XNNPACK 8da4w
`lfm2_5_1_2b_xnnpack_8da4w.pte` (741 MB)
- **Source**: LiquidAI/LFM2.5-1.2B-Instruct (hybrid conv/attention)
- **License**: LFM Open License v1.0
- **Quantization**: 8da4w (8-bit dynamic activation / 4-bit weight) + 8-bit embedding
(`embedding_quantize: "8,0"`; cuts 1143 MB → 741 MB vs the fp32-embedding v1)
- **Export**: executorch 1.4.0 `export_llm`, dynamic shape, max_seq_length 2048, XNNPACK extended_ops
- **Config**: `llm_params/lfm2_5_1_2b_xnnpack_8da4w_e8.yaml`
## Verification (2026-08-13)
Mac gate (greedy via `native.py`, chat template): correct 2-sentence Rayleigh-scattering
answer, 170.8 tok/s on M-series Mac (reference only). v1 (fp32 embedding) passed 3/3
(Paris / Japanese / haiku) with identical quant settings otherwise.
iPhone 17 Pro / iOS 27 (ETBench, XNNPACK CPU, default threads), re-measured
2026-08-14 on this 8-bit-embedding build:
| metric | value |
|--------|-------|
| load | 0.6 s |
| ttft (short prompt) | 0.05-0.06 s |
| decode | **65-86 tok/s** (86 short answer, 65 at 128 tokens) |
Outputs correct (Paris; coherent 128-token story). The earlier 1143 MB
fp32-embedding build loaded in 1.6 s and decoded 55-81 tok/s, so quantizing the
embedding table cut both the file and the load time without costing throughput.
**Usage note — chat template is required.** This is an instruct model: raw untemplated
text makes it emit `<|im_end|>` immediately (looks like broken generation but is not).
Always wrap prompts as
`<|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`, eos ids [7].
<!-- funnel:v1 -->
---
**More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.
**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.
<!-- /funnel:v1 -->
|