mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT

Qwen3-0.6B-Code-Expert — LiteRT-LM (int8)

suayptalha/Qwen3-0.6B-Code-Expert, a full fine-tune of Qwen/Qwen3-0.6B on nvidia/OpenCodeReasoning by its author (suayptalha), converted to the .litertlm format for Google's LiteRT-LM runtime. It is the representative model of the recipe How do I run a fine-tuned Hugging Face model on iPhone?: a community fine-tune under 1B parameters, Apache-2.0, converted with one command and loaded in an iOS app through swift-litert-lm.

File	model.litertlm, 613,406,208 bytes
sha256	b9f8587090b56934e4189a9d969193b5c2fcc120b44ae819692451c92e88f480
Quantization	int8 weights (the converter's default)
Context (KV cache)	4096 tokens
Template	the fine-tune's own Qwen3 chat template, embedded verbatim; thought channel <think> / </think>; stop tokens <|im_end|> (151645) and <|endoftext|> (151643)
Source revision	02c021d (2025-05-11)

2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical.

How it was made

git clone https://github.com/john-rocky/hf-to-litertlm && cd hf-to-litertlm
pip install litert-torch ai-edge-quantizer "transformers==5.14.*" huggingface_hub litert-lm
python scripts/convert.py suayptalha/Qwen3-0.6B-Code-Expert     # 217 s on a Mac Studio M4 Max

Stock litert-torch 0.9.3 export (ai-edge-quantizer 0.8.0, transformers 5.14.1, litert-lm-builder 0.15.0), 2026-09-05. The converter's exit gate asks 8 fixed questions through the runtime: this bundle answered 7 of 8 with no degenerate output (the miss: "Roses are red, violets are ___" answered "violet"); median decode during the gate 138 tok/s on the Mac GPU.

Run it

Desktop, with the LiteRT-LM CLI (macOS, Linux, Windows):

pip install litert-lm
litert-lm run --from-huggingface-repo mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT model.litertlm \
  --prompt "What is 17 + 25? Answer briefly." --thinking false --temperature 0 --top-k 1
# 17 + 25 = 42

Keep --thinking false when you sample greedily. With thinking on and --temperature 0 --top-k 1 the same prompt looped on "17 + 25." inside the thought channel for 128 s and never answered; the Qwen3 model card says not to use greedy decoding in thinking mode. With the CLI's default sampler and thinking on (litert-lm 0.17.0), the model thought for one paragraph and answered 17 + 25 = 42.

iOS, in an existing app, through swift-litert-lm (the same runtime, Metal GPU):

import LiteRTFoundation
let chat = try await LiteRTChat(
  huggingFaceRepo: "mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT", fileName: "model.litertlm",
  modalities: [])                       // text-only: no vision or audio tower
for try await delta in chat.stream("What is 17 + 25? Answer briefly.") { print(delta, terminator: "") }

Dependency, stop, release and the verify command are in the recipe.

Android: the file is the format Google AI Edge Gallery imports; not tested on Android here.

Measured (2026-09-05)

Mac Studio, Apple M4 Max, 128 GB, macOS 27.0, prompt "What is 17 + 25? Answer briefly.", reply 17 + 25 = 42 in every row:

path	backend	decode	prefill	first turn after load	note
swift-litert-lm LiteRTChat (LiteRT-LM v0.15.0 xcframework)	GPU	142.7 tok/s	469.2 tok/s	2.2 s	footprint 1,411 MB; prewarm: false
swift-litert-lm LiteRTChat (LiteRT-LM v0.15.0 xcframework)	CPU	33.3 tok/s	95.2 tok/s	11.4 s	footprint 1,107 MB
litert-lm CLI 0.17.0, --thinking false, greedy	default			1.1 s wall
litert-lm CLI 0.16.0, --thinking false, greedy	default			1.0 s wall

Known limits

• One message per conversation on LiteRT-LM (measured on the v0.15.0 Swift binaries, litert-lm 0.17.1 and 0.18.0). A second message on the same Conversation fails with INTERNAL: The new rendered template string does not start with the previous rendered template string. On the v0.15.0 Swift binaries this happened with thinking off and on, GPU and CPU; in my regression run on litert-lm 0.18.0 (Mac, CPU backend, Python API) it happened in 6 of 6 conversations tried (two probes, each run twice; the first message of every conversation returned, the second raised), and my 2026-09-22 run of this template saw the same on litert-lm 0.17.1. The cause is the chat template, not the weights: before each message the runtime re-renders the conversation so far and requires the new render to extend the previous one, and this template (the fine-tune's Qwen3 template) renders the assistant turn after the latest user message with its <think>…</think> block — an empty one when there was no reasoning — and drops the block once a newer user message follows, so the two renders differ even with thinking off (LiteRT-LM#3443 is my report of the same template on another bundle; #3166 describes the mechanism). Workaround: a new conversation per turn with the history in the prompt. A re-pack with LiteRT-LM's own models/qwen3 template, which renders history turns the same way wherever they sit, would be a metadata-only fix for litert-lm 0.18.0 and newer; it is not in this file yet.
• Greedy decoding with thinking on loops (see above). Use the sampler the Qwen3 card recommends for thinking mode (temperature 0.6, top-p 0.95, top-k 20) or turn thinking off.
• Not verified: any iPhone (the recipe's device row is pending), Android, and the converter's --int4 recipe.

Provenance

• Converted and verified by: john-rocky (GitHub) / mlboydaisuke (Hugging Face)
• Recipe: https://github.com/john-rocky/swift-litert-lm/blob/main/docs/recipe-hf-finetune-to-iphone.md
• Measurements: https://github.com/john-rocky/swift-litert-lm/blob/main/docs/recipe-hf-finetune-to-iphone.md#7-verified--unverified (Mac Studio M4 Max, 2026-09-05)
• Commit: john-rocky/hf-to-litertlm@5ffe9ee (converter); john-rocky/swift-litert-lm@09f04f1 (Swift runs)
• Maintained at: https://github.com/john-rocky/swift-litert-lm/issues

License

Apache-2.0, inherited from the fine-tune (its card metadata declares Apache-2.0) and from Qwen3-0.6B. The weights are the fine-tune author's work quantized to int8; nothing else was changed. Qwen3 is by the Qwen team at Alibaba Cloud. This is an independent conversion, not affiliated with the model author, Alibaba, or Google.