| --- |
| license: other |
| tags: |
| - executorch |
| - xnnpack |
| - pte |
| - on-device |
| - text-generation |
| base_model: |
| - LiquidAI/LFM2.5-350M |
| --- |
| # LFM2.5-350M — ExecuTorch XNNPACK 8da4w |
|
|
| `lfm2_5_350m_xnnpack_8da4w.pte` (252.8 MB) |
|
|
| - **Source**: LiquidAI/LFM2.5-350M (hybrid conv/attention, 16 layers) |
| - **License**: LFM Open License v1.0 |
| - **Quantization**: 8da4w (8-bit dynamic activation / 4-bit weight) + 8-bit embedding (`embedding_quantize: "8,0"`) |
| - **Export**: executorch 1.4.0 `export_llm`, dynamic shape, max_seq_length 2048, XNNPACK extended_ops |
| - **Config**: `llm_params/lfm2_5_350m_xnnpack_8da4w.yaml` |
|
|
| ## Verification (Mac arm64, 2026-08-13) |
|
|
| Generation gate 3/3 via `examples/models/llama/runner/native.py` (greedy): |
|
|
| | prompt | output | |
| |--------|--------| |
| | capital of France? | "The capital of France is Paris." | |
| | 日本の首都は?(日本語) | 「日本の首都は東京です。」 | |
| | haiku about autumn leaves | 3-line poem, coherent | |
|
|
| Known cosmetic warning: params vocab_size (65536) != tokenizer vocab size. |
| |
| ## iPhone 17 Pro (ETBench, XNNPACK CPU, default threads, 2026-08-13) |
| |
| | metric | value | |
| |--------|-------| |
| | load | 0.5 s | |
| | ttft (short prompt) | 0.03 s | |
| | decode | **171-173 tok/s** | |
| |
| Outputs correct (Paris; coherent 128-token story). |
| |
| **Usage note — chat template is required.** This is an instruct model: raw untemplated |
| text makes it emit `<|im_end|>` immediately (looks like broken generation but is not). |
| Always wrap prompts as |
| `<|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`, eos ids [7]. |
|
|
| <!-- funnel:v1 --> |
|
|
| --- |
|
|
| **More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it. |
|
|
| **Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly. |
|
|
| <!-- /funnel:v1 --> |
|
|
