| --- |
| license: other |
| tags: |
| - executorch |
| - xnnpack |
| - pte |
| - on-device |
| - text-generation |
| base_model: |
| - LiquidAI/LFM2.5-1.2B-Instruct |
| --- |
| # LFM2.5-1.2B-Instruct — ExecuTorch XNNPACK 8da4w |
|
|
| `lfm2_5_1_2b_xnnpack_8da4w.pte` (741 MB) |
|
|
| - **Source**: LiquidAI/LFM2.5-1.2B-Instruct (hybrid conv/attention) |
| - **License**: LFM Open License v1.0 |
| - **Quantization**: 8da4w (8-bit dynamic activation / 4-bit weight) + 8-bit embedding |
| (`embedding_quantize: "8,0"`; cuts 1143 MB → 741 MB vs the fp32-embedding v1) |
| - **Export**: executorch 1.4.0 `export_llm`, dynamic shape, max_seq_length 2048, XNNPACK extended_ops |
| - **Config**: `llm_params/lfm2_5_1_2b_xnnpack_8da4w_e8.yaml` |
|
|
| ## Verification (2026-08-13) |
|
|
| Mac gate (greedy via `native.py`, chat template): correct 2-sentence Rayleigh-scattering |
| answer, 170.8 tok/s on M-series Mac (reference only). v1 (fp32 embedding) passed 3/3 |
| (Paris / Japanese / haiku) with identical quant settings otherwise. |
|
|
| iPhone 17 Pro / iOS 27 (ETBench, XNNPACK CPU, default threads), re-measured |
| 2026-08-14 on this 8-bit-embedding build: |
|
|
| | metric | value | |
| |--------|-------| |
| | load | 0.6 s | |
| | ttft (short prompt) | 0.05-0.06 s | |
| | decode | **65-86 tok/s** (86 short answer, 65 at 128 tokens) | |
|
|
| Outputs correct (Paris; coherent 128-token story). The earlier 1143 MB |
| fp32-embedding build loaded in 1.6 s and decoded 55-81 tok/s, so quantizing the |
| embedding table cut both the file and the load time without costing throughput. |
|
|
| **Usage note — chat template is required.** This is an instruct model: raw untemplated |
| text makes it emit `<|im_end|>` immediately (looks like broken generation but is not). |
| Always wrap prompts as |
| `<|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`, eos ids [7]. |
|
|
| <!-- funnel:v1 --> |
|
|
| --- |
|
|
| **More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it. |
|
|
| **Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly. |
|
|
| <!-- /funnel:v1 --> |
|
|
