mlx-community/Mage-VL-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs

Video and image understanding, running locally on a Mac. This is an OptiQ mixed-precision quant of microsoft/Mage-VL, a 5B vision-language model that reads images and video. It is the first video model in the OptiQ lineup, and it runs entirely on Apple Silicon with no PyTorch. 3.7 GB on disk.

Mage-VL pairs a from-scratch Mage-ViT visual encoder with a Qwen3-4B language model. OptiQ quantizes the language tower to mixed 4/8-bit and keeps the vision tower at bf16 in a sidecar, so the same checkpoint does text, image, and video.

It works on image and video

Both of these ran through the quantized model on Apple Silicon, MLX only:

Image (examples/dog.jpg):

A dog is sitting on a rug in front of a patterned rug.

Video (frames sampled from examples/soccer-broadcast.mp4):

A man is reporting on a soccer game.

What it is

Property	Value
Base	microsoft/Mage-VL (Mage-ViT vision + Qwen3-4B, ~5B params)
Method	OptiQ mixed-precision, sensitivity-driven (bf16 reference)
Language tower	per-layer 4/8-bit: 164 layers at 4-bit, 90 at 8-bit
Achieved bits-per-weight	5.90
Vision tower	bf16, kept in `optiq/optiq_vision.safetensors` (297 tensors)
On disk	3.7 GB (3.0 GB language + 0.63 GB vision)
Video	frame sampling (uniform frames), no neural codec required

The vision tower was reimplemented in MLX and validated bit-exact against the reference (max abs diff 1.7e-3 in float32). Following llama.cpp's naming for mixed quants, the "4bit" label denotes the family, not the weighted average.

Capability Score

Six-metric mean (the standard OptiQ text eval). Strong for a 4B, and it does function calling well once the tools are in the prompt. Long-context multi-hop (HashHop) is the weak spot.

Metric	Score
MMLU (5-shot, 969 samples)	74.6%
GSM8K (1000 samples)	88.7%
IFEval (full set, strict)	68.6%
BFCL-V3 simple (200 calls)	88.5%
HumanEval (164 problems, pass@1)	76.2%
HashHop (long-context retrieval)	25.0%
Capability Score (mean of 6)	70.27

Run it

Mage-VL ships under an architecture stock mlx-lm does not know, so import optiq registers it, and OptiQ loads the vision sidecar:

pip install "mlx-optiq>=0.4.7"

For image and video input, serve it with an OpenAI + Anthropic-compatible endpoint:

optiq serve --model mlx-community/Mage-VL-OptiQ-4bit

Then send an image (or sampled video frames) as image_url content. Text-only generation also loads directly:

import optiq  # registers the mage_vl arch + vision sidecar
from mlx_lm import load, generate

model, tok = load("mlx-community/Mage-VL-OptiQ-4bit")
print(generate(model, tok, prompt="Hello", max_tokens=100))

Video is handled as sampled frames (the DCVC neural codec in the base repo is an optional efficiency path and is not needed here).

Model tree for mlx-community/Mage-VL-OptiQ-4bit

Base model

microsoft/Mage-VL

Quantized

(9)

this model