RadixArk/Muse-Glimmer-q4k-dynamic-MLX

Muse Glimmer — MLX Q4_K dynamic (group size 32)

Overview

An Apple Silicon (MLX) checkpoint of Muse Glimmer, produced as a code-preserving repack of the vendor's searched dynamic K-quant GGUF muse-glimmer-30B-kquant-dynamic.gguf (per-layer mixed 4/5/6-bit widths chosen by the vendor's quantization search). Every weight keeps the GGUF's exact quantization code. The highest-fidelity Muse Glimmer MLX variant. Serve it with SGLang's MLX backend on a Mac with 48 GB of unified memory or more. Text only — the MLX backend has no vision path.

Model Specifications

• Source: the vendor GGUF release → gguf_weights/muse-glimmer-30B-kquant-dynamic.gguf (sha256 513109c8…), revision aa336138… (the 2026-08-10 "definitive Muse Glimmer 30B GGUFs" re-publish; its quantized payload is byte-identical to the earlier the previous vendor GGUF — this repack's tensors are bit-identical to the previous revision of this repo, so all evaluation results below are unchanged)
• Format: MLX-quantized safetensors, 5 shards (~20 GB); tokenizer / config / chat template from the vendor HF export
• Quantization: per-layer searched mix of 4/5/6-bit affine at group size 32; Q6_K blocks converted via lossless sub-scale run merge (35,902,464 superblocks hard-asserted, zero violations, no requantization)
• Fused-projection note: 28 layers mix q/gate bit widths in the vendor scheme; the narrower side is losslessly bit-promoted (identical codes, wider packing) so the fused projection is uniform — recorded per-tensor in the artifact manifest
• Fidelity: quantization codes bit-identical to the GGUF; recombined scales stored as bf16 (≤2⁻⁸ relative rounding)
• Architecture: 28B dense, 52 layers (13 full-attention NoPE + 39 sliding-window-2048), context 131,072
• Modality: text only

Evaluation Results

Measured through SGLang MLX on an M5 Pro (64 GB), with the vendor llama.cpp fork serving the source GGUF on the same machine as the reference. GSM8K: 200 questions, no-thinking chat template, temperature 0. CIMemories: single trial, DeepSeek-R1-0528 judge (single-trial judge variance applies).

Benchmark	This checkpoint (SGLang MLX)	llama.cpp, same GGUF
GSM8K	0.975	0.965
CIMemories violation (lower better)	0.00%	0.00%
CIMemories coverage (higher better)	79.5%	77.0%

Best zero-violation CIMemories coverage among the Muse Glimmer MLX variants.

Performance (M5 Pro 64 GB, 1k-in/1k-out greedy)

Batch size	1	2	4	6	8
Decode tok/s (aggregate)	12.6	23.4	41.3	42.6	49.1

The extra bits cost ~10–15% throughput vs the gs128 mirror (decode is memory-bandwidth-bound); still ahead of llama.cpp on the same GGUF codes at every batch size above 1.

Serving with SGLang

SGLANG_USE_MLX=1 SGLANG_MLX_CACHE_LIMIT_GB=8 \
python -m sglang.launch_server \
  --model-path RadixArk/Muse-Glimmer-q4k-dynamic-MLX \
  --trust-remote-code \
  --reasoning-parser muse \
  --tool-call-parser muse \
  --disable-radix-cache \
  --mem-fraction-static 0.85 \
  --host 0.0.0.0 --port 30000

SGLANG_MLX_CACHE_LIMIT_GB=8 caps the MLX buffer cache (keeps the footprint within a 48 GB machine at no throughput cost). Keep --disable-radix-cache. See the Muse Glimmer page in the SGLang cookbook for the full deployment matrix.

Sibling checkpoints

• RadixArk/Muse-Glimmer-q4-MLX — fastest MLX variant, quantized from the BF16 HF export
• RadixArk/Muse-Glimmer-q4km-gs128-MLX — code-preserving mirror of the vendor's recommended GGUF (balanced pick)