Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,61 @@
+---
+license: apache-2.0
+tags:
+- executorch
+- xnnpack
+- pte
+- on-device
+- text-generation
+base_model:
+- Qwen/Qwen3.5-4B
+---
+# Qwen3.5-4B — ExecuTorch XNNPACK 8da4w + 8-bit embedding
+
+`qwen3_5_4b_xnnpack_8da4w_e8.pte` (2815.1 MB)
+
+- **Source**: Qwen/Qwen3.5-4B
+- **License**: Apache-2.0
+- **Quantization**: 8da4w linear + 8-bit embedding (`embedding_quantize: "8,0"`). The
+  248320-row embedding is the reason: left in fp32 it is a gigabyte on its own, and at this
+  vocabulary size that is most of what a small model weighs.
+- **Export**: executorch 1.4.0 `export_llm`, static shape (seq_len=1), max_seq_length 2048,
+  XNNPACK extended_ops
+- **Config**: `llm_params/qwen3_5_4b_xnnpack_8da4w_e8.yaml`
+
+## Verification (Mac arm64, 2026-08-21)
+
+`llm_params/gen_static.py`, token-by-token prefill then greedy decode, a fresh process per
+prompt so no answer is read through the previous one's cache:
+
+| prompt | answer | decode |
+|---|---|---|
+| capital of France? | opens a `<think>` block, reasons, answers Paris | 8.6 tok/s |
+| 日本の首都は? | reads the question back in both languages, then answers | 8.0 tok/s |
+
+This size reaches for its thinking mode on questions the 2B answers outright, so a short
+`max_new` may cut it off mid-thought. Budget for it.
+
+Chat template: ChatML, bos 248045, eos [248046, 248044].
+
+Not measured on a phone, and at 2.8 GB it is the largest text model on this shelf. The
+[0.8B](https://huggingface.co/mlboydaisuke/Qwen3.5-0.8B-ExecuTorch) runs at 10.5 tok/s on an
+iPhone 17 Pro at 651 MB; the
+[2B](https://huggingface.co/mlboydaisuke/Qwen3.5-2B-ExecuTorch) is the middle of the three.
+
+## Running it
+
+```bash
+python llm_params/gen_static.py \
+  --pte qwen3_5_4b_xnnpack_8da4w_e8.pte \
+  --tokenizer tokenizer.json \
+  --prompt $'<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n' \
+  --eos_ids "[248046,248044]"
+```
+
+The 8-bit embedding needs `from executorch.kernels import quantized` before the program is
+loaded. Without it the method will not even load — `kernel
+'quantized_decomposed::embedding_byte.dtype_out' not found` — which reads like a broken
+export rather than a runtime missing its kernels.
+
+(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models) ·
+iOS sample: [executorch-samples](https://github.com/john-rocky/executorch-samples))