MOSS-Transcribe-Diarize-MLX-8bit

8-bit MLX quantization of OpenMOSS-Team/MOSS-Transcribe-Diarize — timestamped transcription + speaker diarization ([S01]/[S02]/… turn labels) running natively on Apple Silicon.

At a glance

Source	OpenMOSS-Team/MOSS-Transcribe-Diarize @ d7231bbae2587a4af278735eb765b318c4f64edd (bf16, 1.8 GB)
Format	MLX, single safetensors tree
Quantization	8-bit affine, group-size 64 (qwen3 decoder only, 197 tensors)
Preserved bf16	Whisper-Medium encoder (367 tensors) + VQAdaptor (6) + decoder 1-D tensors
Disk size	1.26 GB

Audio path

The MOSS audio tower remains partly unquantized in BF16: model.whisper_encoder.* (367 tensors) and model.vq_adaptor.* (6 tensors) pass through in BF16, while model.language_model.* is quantized.

Audio inference smoke

PASS — this variant passed the audio-inference smoke gate on 2026-08-02.

CER ≤ 0.05 on the configured gating clips against the MLX-BF16 self-reference.

clip	CER vs MLX-BF16	gating	wall_s
1_cantonese.wav	0.020202	no	0.281
2_hokkien.wav	0.0	yes	0.502
3_codeswitch_tamil.wav	0.013158	no	0.257
4_codeswitch_singlish.wav	0.0	yes	0.118

• Decoder-only smoke (one-token forward, finite logits): pass
• Total wall clock: 4.772 s

Family

All MLX variants of this model ship together:

• majentik/MOSS-Transcribe-Diarize-MLX-8bit (this repo)
• majentik/MOSS-Transcribe-Diarize-MLX-6bit (retired — repository removed; no current artifact)
• majentik/MOSS-Transcribe-Diarize-MLX-4bit (retired — repository removed; no current artifact)

Usage

Runs on the MOSS-TD MLX harness in majek/pipelines/moss_td_mlx (Whisper encoder + VQAdaptor in MLX, decoder via stock mlx_lm.models.qwen3):

python -m pipelines.moss_td_mlx.generate \
    --path majentik/MOSS-Transcribe-Diarize-MLX-8bit \
    --wav your_clip.wav

Output format: [start][SNN]text[end] segments, speaker-labelled. Stock mlx-lm alone cannot load this composite audio architecture.

Compatibility notes

• tokenizer_config.json here carries extra_special_tokens in dict form — the upstream list form crashes transformers 4.57's tokenizer loading; this pack loads cleanly.
• The upstream torch modeling code requires a newer transformers than 4.57 (torch_compilable_check import, and .generate() degenerates there); the MLX harness has no such dependency.

Reproduce

python -m pipelines.mlx_direct_quantize --model moss-transcribe-diarize \
    --base-dir <upstream snapshot> --out-dir <out> \
    --bits 8 --mode affine --group-size 64