README.md

---
license: other
tags:
- executorch
- xnnpack
- pte
- on-device
- text-generation
base_model:
- LiquidAI/LFM2.5-350M
---
# LFM2.5-350M — ExecuTorch XNNPACK 8da4w

`lfm2_5_350m_xnnpack_8da4w.pte` (252.8 MB)

- **Source**: LiquidAI/LFM2.5-350M (hybrid conv/attention, 16 layers)
- **License**: LFM Open License v1.0
- **Quantization**: 8da4w (8-bit dynamic activation / 4-bit weight) + 8-bit embedding (`embedding_quantize: "8,0"`)
- **Export**: executorch 1.4.0 `export_llm`, dynamic shape, max_seq_length 2048, XNNPACK extended_ops
- **Config**: `llm_params/lfm2_5_350m_xnnpack_8da4w.yaml`

## Verification (Mac arm64, 2026-08-13)

Generation gate 3/3 via `examples/models/llama/runner/native.py` (greedy):

| prompt | output |
|--------|--------|
| capital of France? | "The capital of France is Paris." |
| 日本の首都は?(日本語) | 「日本の首都は東京です。」 |
| haiku about autumn leaves | 3-line poem, coherent |

Known cosmetic warning: params vocab_size (65536) != tokenizer vocab size.

## iPhone 17 Pro (ETBench, XNNPACK CPU, default threads, 2026-08-13)

| metric | value |
|--------|-------|
| load | 0.5 s |
| ttft (short prompt) | 0.03 s |
| decode | **171-173 tok/s** |

Outputs correct (Paris; coherent 128-token story).

**Usage note — chat template is required.** This is an instruct model: raw untemplated
text makes it emit `<|im_end|>` immediately (looks like broken generation but is not).
Always wrap prompts as
`<|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`, eos ids [7].

<!-- funnel:v1 -->

---

**More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.

**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->