---
license: apache-2.0
tags:
- executorch
- xnnpack
- pte
- on-device
- text-generation
base_model:
- Qwen/Qwen3.5-0.8B
---
# Qwen3.5-0.8B — ExecuTorch XNNPACK 8da4w + 8-bit embedding

`qwen3_5_0_8b_xnnpack_8da4w_e8.pte` (651.1 MB — down from 1413 MB without embedding
quantization; the 248320x1024 fp32 embedding was ~1 GB of the first export)

- **Source**: Qwen/Qwen3.5-0.8B
- **License**: Apache-2.0
- **Quantization**: 8da4w linear + 8-bit embedding (`embedding_quantize: "8,0"`)
- **Export**: executorch 1.4.0 `export_llm`, static shape (seq_len=1), max_seq_length 2048, XNNPACK extended_ops
- **Config**: `llm_params/qwen3_5_0_8b_xnnpack_8da4w_e8.yaml`

## Verification (Mac arm64, 2026-08-13)

Generation gate 3/3 via `llm_params/gen_static.py` (token-by-token prefill + greedy decode):

| prompt | output | decode tok/s |
|--------|--------|--------------|
| capital of France? | Paris (correct; adds a wrong "second-largest city in Europe" claim — model-level, same as fp32-embedding export) | 21.0 |
| 日本の首都は?(日本語) | 「日本の首都は **東京** です。」 | 21.4 |
| haiku about autumn leaves | 3-line poem | 21.3 |

Decode ~21 tok/s matches the fp32-embedding export (20.6) — the size cut is free.
Prefill tok/s is sequential-prefill reference only (static export).
Chat template: ChatML, bos 248045, eos [248046, 248044].

iPhone 17 Pro (ETBench, XNNPACK CPU, default threads): decode **10.5 tok/s**, ttft 0.58 s,
load 0.7 s — same speed as the 1413 MB fp32-embedding export (9.2-11.1 tok/s), at 46% of its size.
(Thermal note: a hot device throttles to ~7 tok/s; numbers above are from a cool run.)

<!-- funnel:v1 -->

---

**More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.

**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->