majentik/MERaLiON-3-3B-ASR-MLX-8bit

MERaLiON-3-3B-ASR — MLX 8bit

Summary

MLX affine + MLX (8-bit) variant of MERaLiON/MERaLiON-3-3B-ASR.

Why this variant

Highest-quality MLX variant at ~4.2 GB. Closest to upstream BF16 quality; pick on 16+ GB Macs when accuracy is paramount.

Hardware compatibility

Device	VRAM	Recommendation
Apple Silicon (8 GB)	~4.5 GB	Marginal; 16+ GB preferred
Apple Silicon (16+ GB)	~4.5 GB	Highest quality MLX variant

Reproduce

python3 -m pipelines.mlx_direct_quantize \
  --model meralion3-3b-asr \
  --bits 8 --mode affine --group-size 64 \
  --base-dir ./MERaLiON-3-3B-ASR-bf16 \
  --out-dir ./MERaLiON-3-3B-ASR-MLX-8bit

Reproduced at commit 149d7b6871b8e158c4bfd81b537a324beafca10a.

Evaluation

benchmarks pending — populated after the eval-harness workstream lands.

Audio inference smoke

PASS — this variant passed the audio-inference smoke gate on 2026-07-02.

Gate design: character error rate (CER) is measured against an MLX-BF16 self-reference baseline (the same MLX audio path run at BF16 — not the archived torch reference, which is unstable on 2 of the 4 clips). Only the stable gating clips (2_hokkien.wav, 4_codeswitch_singlish.wav — Hokkien and Singlish) decide the verdict; the other clips are transcribed and recorded as informational. Threshold: CER ≤ 0.05 on every gating clip.

clip	CER vs MLX-BF16	gating	wall_s
1_cantonese.wav	3.0	no	0.534
2_hokkien.wav	0.01	yes	1.01
3_codeswitch_tamil.wav	0.796875	no	0.433
4_codeswitch_singlish.wav	0.0	yes	0.235

• Decoder-only smoke (BOS forward, finite logits): pass
• Total wall clock: 108.786 s
• Smoke run 2026-07-02 on Apple Silicon via the source repo's pipelines/meralion3_mlx package (driver: pipelines/meralion3_mlx_smoke.py).

Family

• 2bit — majentik/MERaLiON-3-3B-ASR-MLX-2bit (private — pending requantization; failed the audio-inference smoke gate)
• 3bit — majentik/MERaLiON-3-3B-ASR-MLX-3bit (private — pending requantization; failed the audio-inference smoke gate)
• 4bit — majentik/MERaLiON-3-3B-ASR-MLX-4bit (private — pending requantization; failed the audio-inference smoke gate)
• 5bit — majentik/MERaLiON-3-3B-ASR-MLX-5bit (private — pending requantization; failed the audio-inference smoke gate)
• 6bit — majentik/MERaLiON-3-3B-ASR-MLX-6bit (private — pending requantization; failed the audio-inference smoke gate)
• MXFP4 — majentik/MERaLiON-3-3B-ASR-MLX-MXFP4 (private — pending requantization; failed the audio-inference smoke gate)
• NVFP4 — majentik/MERaLiON-3-3B-ASR-MLX-NVFP4 (private — pending requantization; failed the audio-inference smoke gate)

Provenance

• Source SHA: 149d7b6871b8e158c4bfd81b537a324beafca10a
• Calibration hash: n/a (post-training quantization, no calibration data)
• Uploaded: 2026-05-27T13:06:26Z

Toolchain:

• huggingface_hub: 1.11.0
• mlx: 0.31.2
• mlx-lm: 0.31.3
• pipeline: pipelines/mlx_direct_quantize.py
• python: 3.12.13
• safetensors: 0.7.0
• transformers: 5.5.4

License

Released under other. Upstream license of the base model applies.

Upstream License Notice

On 2026-06-06 the upstream MERaLiON team published the official licence for MERaLiON/MERaLiON-3-3B-ASR: the MERaLiON Public Licence (A*STAR, © 2026).

This artifact is a quantized Derivative Work of MERaLiON/MERaLiON-3-3B-ASR and is provided under that licence:

• Modification: MLX quantization of the Gemma-2 text decoder; the audio path (speech encoder, audio adapter, LayerNorm bridge) is bit-exact BF16.
• Creators: A*STAR / MERaLiON — creator identity and the upstream copyright notice are retained.
• No warranty: this artifact is provided WITHOUT WARRANTY of any kind; use at your own risk.
• Licence: licensed under the MERaLiON Public Licence, which incorporates:
  • the Gemma Terms of Use (the text decoder is Gemma-2-2b-it) — "Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms";
  • the OpenAI Whisper-Large-v3 community license (speech encoder);
  • MIT and Apache 2.0.
• Out of scope: per the upstream licence, this model is not intended for use in math and coding tasks.

Audio path bit-exactness

Only the Gemma-2-2b-it text decoder under text_decoder.* is quantized. The Whisper-style speech encoder (speech_encoder.*), audio adapter (speech_audio_adapter.*), and LayerNorm bridge (ln_speech.*) are passed through in BF16 — bit-exact to the upstream weights. ASR-quality regressions vs upstream are therefore attributable to text-decoder quantization alone.