mlboydaisuke/Qwen2.5-1.5B-ExecuTorch

Qwen2.5-1.5B — ExecuTorch XNNPACK 8da4w + 8-bit embedding

qwen2_5_1_5b_xnnpack_8da4w_e8.pte (1035.3 MB)

• Source: Qwen/Qwen2.5-1.5B — the base model, not Instruct. It continues text; it does not answer a chat template.
• License: Apache-2.0
• Quantization: 8da4w linear + 8-bit embedding (embedding_quantize: "8,0")
• Export: executorch 1.4.0 export_llm, static shape (seq_len=1), max_seq_length 2048, XNNPACK extended_ops
• Config: llm_params/qwen2_5_1_5b_xnnpack_8da4w_e8.yaml

Verification (Mac arm64, 2026-08-21)

Completions, against the untouched model on the same prompts. A base model is checked this way because a chat template gets it nowhere: fed one, it echoes the question back.

prompt	this file	eager fp32
"The capital of France is"	turns it into a multiple-choice question and answers A, Paris	" Paris. The capital of France is also the capital of the European Union…"
"Water boils at a temperature of"	"100°C and water freezes at a temperature of 0°C"	"212 °F or 100 °C and ice melts at a temperature of 32 °"
"Seventeen times four equals"	"what number? To determine what number 17 times 4 equals, we need to perform the multiplication"	"what number? To find the product of 17 and 4, we perform the multiplication: 17 *"

The continuations differ, which they will once the weights are 4-bit, but nothing here is wrong where eager is right. Decode 69.5 tok/s, from one pass with nothing else running.

The 0.5B of this pair is not on the shelf. It fails that comparison rather than passing it: asked where water boils it says 215 degrees Fahrenheit, where eager says 212 °F or 100 °C. Half a billion parameters do not survive 4-bit weights, and XNNPACK has no 8-bit path that works — torchao:8da8w cannot be combined with the delegate and qmode: int8 falls over on grouped-query attention. At that size LFM2-350M is smaller, faster and correct.

Converting it

python convert/export_from_safetensors.py qwen2_5_1_5b

Not export_llm directly: this repository ships safetensors only, and load_checkpoint_from_pytorch_model reads pytorch_model.bin, so the code falls back to torchtune's checkpointer. Qwen2.5's converter wants one thing from torchtune — get_mapped_key — and ExecuTorch has its own copy of that function, so the script hands the real one over and stubs the rest.

Running it

python llm_params/gen_static.py \
  --pte qwen2_5_1_5b_xnnpack_8da4w_e8.pte \
  --tokenizer tokenizer.json \
  --prompt 'The capital of France is' \
  --eos_ids "[151643]"

The 8-bit embedding needs from executorch.kernels import quantized before the program is loaded, and portable_lib._load_for_executorch rather than executorch.runtime.

(conversion scripts: executorch-models · iOS sample: executorch-samples)