Vontra/Muse-Glimmer-30B-MLX-4bit

Muse Glimmer 30B — MLX 4-bit

A native Apple-silicon conversion of meta-models/Muse-Glimmer-30B, converted to uniform affine 4-bit MLX weights for text-and-image inference with MLX-VLM.

Original model · MLX-VLM · MLX · More Vontra conversions

About this conversion

This repository contains a uniform MLX 4-bit conversion of Muse Glimmer 30B, a dense agentic language model with a dedicated perception encoder. It preserves the upstream tokenizer, ATEM chat template, image processor configuration, generation configuration, licence, and usage policy.

Item	Value
Base model	meta-models/Muse-Glimmer-30B
Format	MLX safetensors
Quantization	Affine 4-bit for eligible modules, group size 64
Mixed-precision overrides	None
Conversion/runtime stack	MLX-VLM with MLX on Apple silicon
Weight shards	4
Weight size	19.41 GB (18.08 GiB)
Context length	131,072 tokens
Maximum visual tokens	4,096 per image
Architecture	muse_glimmer

Modules that MLX-VLM does not classify as quantizable remain at their source-compatible precision, so the effective whole-checkpoint bits-per-weight is higher than four. The quantization metadata records the actual eligible-module recipe.

Muse Glimmer support was validated against the official MLX-VLM source at commit 5262cb6, reporting package version 0.6.12. An older MLX-VLM or oMLX bundle may report Model type muse_glimmer not supported; update to a build containing the upstream Muse implementation before loading this checkpoint.

Apple-silicon performance

This checkpoint was load-tested, text-generation tested, and benchmarked on:

Hardware	Configuration
Host	Mac Studio
Chip	Apple M3 Ultra
CPU	32 cores (24 performance + 8 efficiency)
Unified memory	256 GB
Runtime	MLX-VLM 0.6.12 source revision 5262cb6 with the oMLX MLX runtime

A warmed local text-only test produced:

Measurement	Result
Decode (median)	37.70 tokens/s
Individual decode runs	37.70 / 37.58 / 38.19 tokens/s
Reported peak memory	21.33 GB
Timed runs	3 × 256 generated tokens
Warm-up	256 generated tokens
Prompt	83 tokens after chat templating

The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, images, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance. Image prefill is not included in this decode benchmark.

Runtime setup

Until Muse Glimmer support reaches the MLX-VLM build supplied by your application, install the exact official source revision used for validation:

python -m pip install \
  "mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm.git@5262cb6a27c797c5c3daf64b17d757924c5c474f"

Download the checkpoint if a local copy is preferred:

hf download Vontra/Muse-Glimmer-30B-MLX-4bit \
  --local-dir ~/.omlx/models/Vontra/Muse-Glimmer-30B-MLX-4bit

Text generation

from mlx_vlm import generate, load

model, processor = load("Vontra/Muse-Glimmer-30B-MLX-4bit")
tokenizer = processor.tokenizer if hasattr(processor, "tokenizer") else processor
messages = [
    {"role": "user", "content": "Explain unified memory on Apple silicon."}
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    reasoning_strength="medium",
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=512,
    temperature=0.0,
)
print(result.text)

Muse Glimmer supports low, medium, high, and xhigh reasoning strengths. Higher settings can spend more tokens reasoning before returning the final answer.

Image and text generation

The multimodal path was smoke-tested locally with a real image. Include an image content item so the ATEM template emits the required patch token:

from mlx_vlm import generate, load

model, processor = load("Vontra/Muse-Glimmer-30B-MLX-4bit")
tokenizer = processor.tokenizer if hasattr(processor, "tokenizer") else processor
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Describe this image in one sentence."},
        ],
    }
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
result = generate(
    model,
    processor,
    prompt,
    image=["image.jpg"],
    max_tokens=256,
    temperature=0.0,
)
print(result.text)

Architecture

Muse Glimmer combines a dense causal transformer with a dedicated perception encoder for interleaved text and image input.

Architecture detail	Upstream value
Total parameters	~29.6B
Transformer layers	52
Hidden size	6,656
Attention heads / KV heads	32 / 2
Head dimension	128
FFN intermediate size	19,968
Attention pattern	local / local / local / global
Sliding window	2,048 tokens
Vocabulary size	202,048
Perception encoder	~1.8B-parameter ViT-G/14, 50 layers
Context length	131,072 tokens

The model supports agentic task completion, tool use through the upstream ATEM protocol, controllable reasoning effort, multilingual input, failure recovery, and multimodal understanding. See the original model card for upstream benchmarks, training details, intended uses, limitations, and safety guidance.

Conversion and validation notes

• Source weights: upstream BF16 checkpoint at revision a4e59da52a7bc87ae7251dd5545c0dd437c44b68.
• Quantization mode: affine, 4-bit, group size 64, without mixed-precision overrides.
• The upstream tokenizer, ATEM chat template, processor configuration, generation configuration, licence, and usage policy are included.
• All 2,278 converted tensors and all 4 indexed shards were checked locally.
• The checkpoint was exercised through full text generation; the Muse architecture was also smoke-tested with image input.
• Quantization can reduce output quality relative to BF16. Use a higher-precision variant when quality matters more than memory use.
• This release does not include or claim support for the upstream speculative drafter.

This is a community conversion, not an official Meta release. Validate quality, safety, and numerical behaviour on representative workloads before production use.

Licence, usage policy, and attribution

The upstream model is released under the Apache License 2.0. The upstream LICENSE and USAGE_POLICY.md files are included in this repository; use is subject to both the licence and the upstream usage policy.

All model design, training, benchmark, and upstream documentation credit belongs to Meta and the original contributors. The MLX conversion, Apple-silicon validation, compatibility work, and model card are provided by Vontra.