MOSS-Transcribe-Diarize-MLX-8bit

8-bit MLX quantization of OpenMOSS-Team/MOSS-Transcribe-Diarize — timestamped transcription + speaker diarization ([S01]/[S02]/… turn labels) running natively on Apple Silicon.

At a glance

Source	OpenMOSS-Team/MOSS-Transcribe-Diarize @ d7231bbae2587a4af278735eb765b318c4f64edd (bf16, 1.8 GB)
Format	MLX, single safetensors tree
Quantization	8-bit affine, group-size 64 (qwen3 decoder only, 197 tensors)
Preserved bf16	Whisper-Medium encoder (367 tensors) + VQAdaptor (6) + decoder 1-D tensors
Disk size	1.26 GB

Verification (2026-07-10, M-series Mac)

The MLX BF16 harness output is verbatim-identical to the upstream torch reference (manual greedy loop) on the test clip, timestamps included. Quantized gating vs the MLX-BF16 self-reference on two clips (code-switched Singlish; a two-speaker Hokkien dialog with correct [S01]/[S02] turns):

variant	Singlish CER	Hokkien-dialog CER	gate (≤0.05)
8bit	0.000	0.000	pass
6bit	0.000	0.111	fail (coherent, minor drift)
4bit	0.227	0.239	fail (coherent, visible drift)

Usage

Runs on the MOSS-TD MLX harness in majek/pipelines/moss_td_mlx (Whisper encoder + VQAdaptor in MLX, decoder via stock mlx_lm.models.qwen3):

python -m pipelines.moss_td_mlx.generate \
    --path majentik/MOSS-Transcribe-Diarize-MLX-8bit \
    --wav your_clip.wav

Output format: [start][SNN]text[end] segments, speaker-labelled. Stock mlx-lm alone cannot load this composite audio architecture.

Compatibility notes

• tokenizer_config.json here carries extra_special_tokens in dict form — the upstream list form crashes transformers 4.57's tokenizer loading; this pack loads cleanly.
• The upstream torch modeling code requires a newer transformers than 4.57 (torch_compilable_check import, and .generate() degenerates there); the MLX harness has no such dependency.

Reproduce

python -m pipelines.mlx_direct_quantize --model moss-transcribe-diarize \
    --base-dir <upstream snapshot> --out-dir <out> \
    --bits 8 --mode affine --group-size 64