MOSS-Transcribe-preview-2B-MLX-4bit

4-bit MLX quantization of OpenMOSS-Team/MOSS-Transcribe-preview-2B -- an English speech-to-text model (Qwen3-Omni-MoE audio encoder + gated-MLP adapter + Qwen3-1.7B decoder) -- running natively on Apple Silicon.

At a glance


Source	`OpenMOSS-Team/MOSS-Transcribe-preview-2B` @ `c98175cb20e48bd9be4e95f6c85f2af18899f780` (bf16, 4.84 GB)
Architecture	Qwen3-Omni-MoE audio encoder (32 layers) + gated-MLP adapter + Qwen3-1.7B decoder (28 layers)
Format	MLX, single safetensors tree
Quantization	4-bit affine, group-size 64 (Qwen3 decoder only, 197 tensors; audio encoder + adapter preserved bf16)
Disk size	2.2 GB

Verification (2026-07-12, M-series Mac)

Audio encoder FP32 parity vs the upstream torch reference: max-abs-diff 3.6e-7 (bit-exact to floating-point precision).
Transcription smoke on clean English clips of varied length: CER 0.000 at 8/6/and 4-bit vs ground truth (word-perfect), e.g. "The quick brown fox jumps over the lazy dog near the riverbank."

Sibling variants

Variant	Size	Status
MOSS-Transcribe-preview-2B-MLX-8bit	3.0 GB	public
MOSS-Transcribe-preview-2B-MLX-6bit	2.6 GB	public
MOSS-Transcribe-preview-2B-MLX-4bit	2.2 GB	public

Usage

Runs on the MOSS-Transcribe MLX harness in majek/pipelines/moss_transcribe_mlx (Qwen3-Omni audio encoder vendored from mlx-vlm, decoder via stock mlx_lm.models.qwen3):

python -m pipelines.moss_transcribe_mlx.generate \
    --path majentik/MOSS-Transcribe-preview-2B-MLX-4bit \
    --wav your_clip.wav

Stock mlx-lm/mlx-vlm alone cannot load this composite audio architecture. This is an English ASR model (Open ASR Leaderboard); non-English audio is out of domain.

Reproduce

python -m pipelines.mlx_direct_quantize --model moss-transcribe-preview-2b \
    --base-dir <upstream snapshot> --out-dir <out> \
    --bits 4 --mode affine --group-size 64