mlboydaisuke/Qwen3-ASR-0.6B-ExecuTorch

Qwen3-ASR-0.6B — ExecuTorch (30-language speech recognition)

Speech in, a transcript out, in one .pte with several methods. An 18-layer audio tower turns mel frames into one embedding per 77 ms, those rows are spliced into the prompt where <|audio_pad|> sits, and a 28-layer Qwen3 decoder generates the text — the same decoder architecture as Qwen3-Embedding-0.6B elsewhere on this shelf.

audio_encoder_<N>s   input_features (1, 128, 100N) fp32  ->  audio rows (13N, 1024)
token_embeddings     input_ids      (1, L)         int64 ->  rows      (1, L, 1024)
text_model           rows, positions, slots               ->  logits   (1, L, 151936)

• File: qwen3_asr_0_6b_xnnpack_8da4w.pte — 697.0 MB, fifteen methods
• Source: Qwen/Qwen3-ASR-0.6B — 782.4M parameters
• License: apache-2.0
• Languages: 30, auto-detected — the model emits language <Name> before the text

The recipe

int8 on the audio tower, int4 on the decoder, and the token table quantised to int8 by hand. That split is measured rather than assumed: at the 30-second window, int8 on the tower alone changes no transcript (table below), while the decoder is most of the weight and is where four bits buys the most.

The table needs doing by hand because quantize_'s Linear-only filters skip an nn.Embedding, and tie_word_embeddings does not help — quantize_ replaces lm_head.weight and the embedding keeps pointing at the old fp32 tensor. 151936 x 1024 is 622 MB, and in this bundle it appears twice: once as the table, once as the decoder's output projection. Tying weights in PyTorch does not tie them here.

Unquantised, the same bundle is 3757.2 MB.

Running it

1. The mel. WhisperFeatureExtractor settings: 16 kHz mono, 128 bins, n_fft 400, hop 160. One second of audio is 100 frames.

2. Pick the window, and pick it tight. The file carries an audio encoder for each of 2, 3, 4, 5, 6, 7, 8, 10, 12, 15, 20, 25 and 30 seconds. Use the smallest one your clip fits and zero-pad the mel to exactly 100 x N frames. Do not reach for the 30-second window because it is there — see below.

3. Size the prompt to the window. The chat template puts a run of <|audio_pad|> where the audio goes, and the processor sizes that run from the clip it was given. The graph always emits 13 x N rows, and the two have to agree. Rewrite the run to 13 x N placeholders, then overwrite those rows with the encoder's output.

4. Decode. text_model takes a dynamic sequence length, so the whole prompt goes through in one call and each generated token in a call of its own. positions and slots are both the absolute row index; the cache is causal against the full buffer. Stop at 151643 or 151645.

The window is the thing to get right

The upstream encoder takes a mask so it can ignore padding, and it uses that mask to pack only the valid positions into its output — through a nonzero(). That is what makes the output length depend on the data, and a fixed-shape graph cannot have it. So this conversion pins the window and treats it as fully valid.

That is not free, and here is the price. Fifteen clips in three languages, each transcribed through the model's own masked pipeline and then again with the window padded out and the mask dropped:

padding	transcripts identical to the masked pipeline
tight — the next whole second	14 of 15
every clip padded to 5 s	8 of 15
every clip padded to 10 s	11 of 15
every clip padded to 30 s	8 of 15

It is not a dose-response — 10 s beats 5 s — which says the failure is a near-tie tipping rather than silence gradually swamping the signal. The errors look like 停まりますか becoming 泊まりますか: same reading, wrong character.

Hence the ladder. Thirteen encoder methods cost 3.8 MB in total, because methods in one .pte share their constants: one window is 3753.4 MB in fp32 and thirteen are 3757.2 MB. There is no reason to make a caller over-pad. (Those two fp32 figures are from the pre-rewrite graph; the rewrite moved the published 8da4w file by 0.5 MB, and the +3.8 MB it is making a point about is unaffected.)

Verification

Greedy decode driven entirely by the .pte — its own encoder, its own token table, its own decoder and KV cache — against the whole model in eager on the identical clip and window. Nine clips, Japanese / Portuguese / Russian, windows 2–4 s:

bundle	worst CER	ignoring case and terminal punctuation
fp32 (3757.2 MB, not published)	0.0000	0.0000
8da4w (697.0 MB, this file)	0.1333	0.0667

Both arms run at the same window on purpose, so this measures the conversion and not the window choice, which is measured separately above.

The unquantised bundle is exact — nine of nine, character for character. That is the number that says the graph surgery is right.

Quantised, eight of nine survive, and the two kinds of difference are worth separating because they are not the same thing:

eager  hoje está um dia bonito              один кофе, пожалуйста
pte    Hoje está um dia bonito.             Один кофе, пожалуйста.

Four of the five differing clips are only a capital first letter and a full stop. That is systematic — the quantised bundle punctuates where fp32 does not — and it is a real difference in what the file emits, so it stays in the raw number. It is not a misrecognition.

The ninth clip is:

eager  この電車は東京駅に停まりますか      (does this train stop at Tokyo Station)
pte    この電車は東京駅に泊まりますか。    (does this train stay overnight at ...)

One character, same reading, wrong kanji — a genuine word error. It is also the same clip and the same substitution that the window table above turns up, and the fp32 model flips on it too: tight padding gives 停, padding to 5 s or 10 s or 30 s gives 泊, with no quantisation anywhere. This clip is a near-tie in the model itself, and anything that perturbs it — four-bit weights, a second of silence — tips it. Nine clips of synthetic speech are a control, not a benchmark: they say the conversion reproduces the fp32 model, and nothing about word error rate on real speakers.

What the audio encoder alone costs

Measured standalone at the 30-second window, swapping only the audio half and reading the transcript:

build	size (MB)	Mac ms*	XNNPACK takes	corr vs eager	worst CER†
XNNPACK fp32	746.0	143.6	71.9%	0.999999	0.0000
XNNPACK fp16	381.7	237.6	64.5%	0.999995	0.0000
XNNPACK int8	200.6	123.5	78.0%	0.998266	0.0000
Core ML	374.1	40.6	100%	0.972872	0.0690

*Mac arm64, median of 10 — a reference point for relative cost, not a device number. Torch eager fp32 on the same machine is 600.6 ms, so the XNNPACK build is 4.2x faster than eager.

†The CER column was measured on the pre-rewrite graph (below). The rewrite agrees with it to 1.3e-06 at the tensor level and leaves every transcript in this card unchanged, but the transcripts behind that column were not re-read.

Those speeds moved because the windowed attention stopped calling F.scaled_dot_product_attention. It decomposes to _safe_softmax, whose guard for fully-masked rows is six operations XNNPACK cannot take — and the call has no mask and is_causal=False, so the guard can never fire. There are 18 layers x 4 windows of it at 30 seconds. Writing the softmax out took the fp32 tower from 55.8% delegated at 170.0 ms to 71.9% at 143.6 ms, and int8 from 149.4 ms to 123.5 ms, agreeing with the old graph to 7.1e-07.

Two things worth taking from that table. int8 on the audio tower is free in the only unit that matters, which is why the shipped bundle uses it there. And the Core ML build is four times faster and does change transcripts — correlation flagged it at 0.973 and the transcripts agree with the flag, so it is not published.

Converting it yourself: the checkpoint does not load

Qwen3ASRForConditionalGeneration.from_pretrained("Qwen/Qwen3-ASR-0.6B") on transformers 5.15.0 returns a fully randomly initialised model and only warns. generate still runs; only the text is meaningless. Four things drifted:

	checkpoint	transformers 5.15.0
config	thinker_config.audio_config / .text_config	both at the top level
encoder type	qwen3_asr_audio_encoder	qwen3_asr_encoder
tensors	thinker.model.*, thinker.audio_tower.*	model.language_model.*, model.audio_tower.*
projector	audio_tower.proj1 / proj2	multi_modal_projector.linear_1 / linear_2

Because the config is not read, the audio tower is built at its default 24 layers against an 18-layer checkpoint and nothing matches at all. Qwen's model card points at their own qwen-asr package; transformers' built-in implementation targets a different layout. convert/export_qwen3_asr.py rebuilds the config from thinker_config, renames the tensors, and asserts that nothing was left behind — and then checks the result by transcribing known speech, because a remap that loads cleanly can still be wrong.

python convert/fetch_hf.py Qwen/Qwen3-ASR-0.6B
python convert/export_qwen3_asr.py bundle 8da4w 2 3 4 5 6 7 8 10 12 15 20 25 30
python convert/check_qwen3_asr.py

torch.export -> to_edge_transform_and_lower(partitioner) -> .pte (conversion scripts: executorch-models)