---
license: other
tags:
- executorch
- xnnpack
- pte
- on-device
- text-generation
base_model:
- LiquidAI/LFM2-700M
---
# LFM2-700M — ExecuTorch XNNPACK 8da4w + 8-bit embedding

`lfm2_700m_xnnpack_8da4w_e8.pte` (486.5 MB)

- **Source**: LiquidAI/LFM2-700M
- **License**: LFM Open License v1.0
- **Quantization**: 8da4w linear + 8-bit embedding (`embedding_quantize: "8,0"`)
- **Export**: executorch 1.4.0 `export_llm`, static shape (seq_len=1), max_seq_length 2048,
  XNNPACK extended_ops
- **Config**: `llm_params/lfm2_700m_xnnpack_8da4w_e8.yaml`

## Verification (Mac arm64, 2026-08-21)

`llm_params/gen_static.py`, token-by-token prefill then greedy decode, a fresh process per
prompt so no answer is read through the previous one's cache:

| prompt | answer |
|---|---|
| capital of France? | "The capital city of France is Paris." |
| 17 times 4? | "17 times 4 is 68." |

Decode **91.1 tok/s**, from one pass over every model on this shelf with nothing else
running. That matters more than it sounds: the same file measured a quarter of its rate
while an export was running alongside.

Chat template: ChatML, bos 1, eos [7].

Not measured on a phone.

## Three things checked before exporting

- **`use_sdpa_with_kv_cache` is on.** Upstream's `qwen3_5` config leaves it off with no
  reason given while the equally hybrid `lfm2` config has it on; measured on Qwen3.5-2B in
  one run, that is 8.20 tok/s against 16.64.
- **`dim` and `hidden_dim` both divide by the quantizer's group size.** 8da4w only touches a
  linear whose in_features divide by it, and skips the rest silently — SmolLM2-135M, which
  is 576 wide, came out at 475 MB against fp32's 540 with no warning at all.
- **Every field of the params json is read by the generic path**, via
  `convert/check_params_used.py`. SmolLM3 sets `no_rope_layer_interval`, which `ModelArgs`
  declares and only the MLX and Qualcomm backends read, and it exports fine and then repeats
  a single word forever.

## Running it

```bash
python llm_params/gen_static.py \
  --pte lfm2_700m_xnnpack_8da4w_e8.pte \
  --tokenizer tokenizer.json \
  --prompt $'<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n' \
  --eos_ids "[7]"
```

The 8-bit embedding needs `from executorch.kernels import quantized` before the program is
loaded, and `portable_lib._load_for_executorch` rather than `executorch.runtime`. Without
that the method will not load at all — `kernel
'quantized_decomposed::embedding_byte.dtype_out' not found` — which reads like a broken
export rather than a runtime missing its kernels.

(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models) ·
iOS sample: [executorch-samples](https://github.com/john-rocky/executorch-samples))