MarkChen1214/Breeze-ASR-25-MLX-5bit

Breeze-ASR-25 MLX 5-bit

Uniform affine 5-bit MLX conversion of MediaTek-Research/Breeze-ASR-25, an Apache-2.0 Whisper-large-v2 fine-tune for Taiwan Mandarin and Mandarin-English code switching.

This is an independent community conversion, not an official MediaTek release. Every quantizable Linear and Embedding module uses 5 bits with group size 64. The weight file is 1,069,126,236 bytes, 65.32% smaller (2.88x) than the verified 3,082,904,296-byte BF16 MLX base.

Quantization

• Uniform affine 5-bit weights, group size 64
• Every quantizable Linear and Embedding module uses Q5
• Non-quantizable tensors remain in their original floating-point storage
• model.safetensors: 1,069,126,236 bytes
• SHA-256: 9d04f350111b6a0234117e4e3d3b50adff1c64b9c6e8a8811028db855b4d0e5b
• Exact allocation: quant_bitmap.json

Runtime

The included mlx_whisper_mixed.py provides the verified short-form runtime. It uses fused attention, text-only greedy decoding, automatic language detection, a 64-token ceiling, and conservative exact-loop termination.

from mlx_whisper_mixed import load_model, transcribe_short_with_model

model = load_model("MarkChen1214/Breeze-ASR-25-MLX-5bit")
result = transcribe_short_with_model(
    model,
    "audio.wav",
    context_seconds=10,
)
print(result["text"])

Audio longer than the selected context is rejected rather than silently truncated. Use context_buckets=(10, 30) for variable short-form input, or chunk long-form audio explicitly.

Tested with Python 3.12, MLX 0.32.0, and mlx-whisper 0.4.3.

Full Common Voice 16.1 zh-TW verification

The primary evaluation uses all 4,976 examples (5.155 hours) from the Common Voice 16.1 zh-TW test split. It specifically reads the mirror's original column containing raw Mozilla audio, not its MP-SENet-denoised audio column. Nothing is sampled, dropped, or truncated: 4,974 clips use a 10-second context and the two longer clips use 30 seconds.

Both MLX models use the same M4 Pro, audio cache, utterance order, automatic language detection, text-only greedy decoding, fused attention, and 64-token ceiling. CER applies OpenCC s2tw to both sides and removes whitespace.

Artifact	Weights	Errors / chars	CER	Stable-power RTFx*
MLX BF16 control	3,082.9 MB	3,861 / 39,477	9.78%	14.7x
MLX Q5 candidate	1,069.1 MB	4,065 / 39,477	10.30%	17.7x

Q5 adds 0.5168 CER points versus the paired BF16 control. A 100,000-draw paired utterance bootstrap gives a 95% interval of +0.2695 to +0.7824 points. Both models produced 4,976/4,976 non-empty transcripts.

MediaTek reports 7.97 MER for Breeze-ASR-25 (Twister) on CommonVoice16-zh-TW in the official paper. That value is useful author-reported context, but it is not used as the quantization baseline: the paper does not disclose its exact decoder or text-normalization contract. The paired MLX BF16 result above is the reproducible measure of Q5's quality cost.

Timing

*Quality and timing use the same complete 4,976 row identities and references. Quality uses raw original audio; timing uses the mirror's MP-SENet-denoised audio column because those full runs maintained a valid stable power state. The stable normal-power timing measured 14.7x BF16 and 17.7x Q5, making Q5 1.20x faster in the matched lane.

The raw-audio quality runs occurred during a low-battery incident, so their observed 5.3x BF16 and 13.9x Q5 figures are deliberately rejected and are not reported in the table. This distinction prevents a charger transition or near-empty-battery throttling from becoming a speed claim.

Warm-up and one-time model loading are excluded from RTFx. WAV loading, mel preprocessing, and decoding are included. Hardware is a MacBook Pro Mac16,7 with Apple M4 Pro and 48 GB memory.

Secondary code-switch verification

On a deterministic 250-row ML2021 Mandarin-English code-switch set, the same automatic-language runtime measured 10.44% CER / 10.3x RTFx for Q5 and 10.05% CER / 8.6x RTFx for BF16. This remains secondary evidence; Common Voice zh-TW is almost entirely Mandarin and cannot establish broad English code-switch quality by itself.

Provenance

• Source: MediaTek-Research/Breeze-ASR-25 at cffe7ccb404d025296a00758d0a33468bec3a9d0
• Source weight SHA-256: c5d952b3bc03ea277209aff0ef5b5c4c055d74449ff794c02d8f4e315fdef6b6
• Audited MLX base: eoleedi/Breeze-ASR-25-mlx at 172151e3fd93ab6dc841d639f23dd48b24bc8301
• BF16 audit: 1,258/1,258 comparable tensors passed; maximum absolute difference 2.98e-08
• Q5 weight SHA-256: 9d04f350111b6a0234117e4e3d3b50adff1c64b9c6e8a8811028db855b4d0e5b

Limitations

• Q5 does not match BF16 exactly: its measured full-set cost is +0.5168 CER points under the protocol above.
• The optimized path is intended for short-form input and does not provide word timestamps with fused attention.
• Quality and timing use the same row identities but different audio representations (original raw audio for CER, denoised audio for RTFx). Rebenchmark raw audio on the deployment machine if that preprocessing distinction matters to capacity planning.
• Results are hardware- and power-state-specific; benchmark the deployment machine independently.

License and attribution

Apache-2.0, following the source checkpoint. Please cite the original Breeze ASR 25 authors and model when using this conversion.