README.md

---
license: apache-2.0
library_name: mlx
pipeline_tag: image-text-to-text
base_model: microsoft/Mage-VL
tags:
- mlx
- optiq
- quantized
- 4bit
- mixed-precision
- mage-vl
- multimodal
- vision-language-model
- video-understanding
- image-text-to-text
- apple-silicon
---

# mlx-community/Mage-VL-OptiQ-4bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

**Video and image understanding, running locally on a Mac.** This is an OptiQ mixed-precision quant of [microsoft/Mage-VL](https://huggingface.co/microsoft/Mage-VL), a 5B vision-language model that reads images and video. It is the first video model in the OptiQ lineup, and it runs entirely on Apple Silicon with no PyTorch. 3.7 GB on disk.

Mage-VL pairs a from-scratch Mage-ViT visual encoder with a Qwen3-4B language model. OptiQ quantizes the language tower to mixed 4/8-bit and keeps the vision tower at bf16 in a sidecar, so the same checkpoint does text, image, and video.

## It works on image and video

Both of these ran through the quantized model on Apple Silicon, MLX only:

**Image** (`examples/dog.jpg`):
> A dog is sitting on a rug in front of a patterned rug.

**Video** (frames sampled from `examples/soccer-broadcast.mp4`):
> A man is reporting on a soccer game.

## What it is

| Property | Value |
|---|---|
| Base | microsoft/Mage-VL (Mage-ViT vision + Qwen3-4B, ~5B params) |
| Method | OptiQ mixed-precision, sensitivity-driven (bf16 reference) |
| Language tower | per-layer 4/8-bit: 164 layers at 4-bit, 90 at 8-bit |
| Achieved bits-per-weight | 5.90 |
| Vision tower | bf16, kept in `optiq/optiq_vision.safetensors` (297 tensors) |
| On disk | 3.7 GB (3.0 GB language + 0.63 GB vision) |
| Video | frame sampling (uniform frames), no neural codec required |

The vision tower was reimplemented in MLX and validated bit-exact against the reference (max abs diff 1.7e-3 in float32). Following `llama.cpp`'s naming for mixed quants, the "4bit" label denotes the family, not the weighted average.

## Capability Score

Six-metric mean (the standard OptiQ text eval). Strong for a 4B, and it does function calling well once the tools are in the prompt. Long-context multi-hop (HashHop) is the weak spot.

| Metric | Score |
|---|---:|
| MMLU (5-shot, 969 samples) | 74.6% |
| GSM8K (1000 samples) | 88.7% |
| IFEval (full set, strict) | 68.6% |
| BFCL-V3 simple (200 calls) | 88.5% |
| HumanEval (164 problems, pass@1) | 76.2% |
| HashHop (long-context retrieval) | 25.0% |
| **Capability Score** (mean of 6) | **70.27** |

## Run it

Mage-VL ships under an architecture stock mlx-lm does not know, so `import optiq` registers it, and OptiQ loads the vision sidecar:

```bash
pip install "mlx-optiq>=0.4.7"
```

For image and video input, serve it with an OpenAI + Anthropic-compatible endpoint:

```bash
optiq serve --model mlx-community/Mage-VL-OptiQ-4bit
```

Then send an image (or sampled video frames) as `image_url` content. Text-only generation also loads directly:

```python
import optiq  # registers the mage_vl arch + vision sidecar
from mlx_lm import load, generate

model, tok = load("mlx-community/Mage-VL-OptiQ-4bit")
print(generate(model, tok, prompt="Hello", max_tokens=100))
```

Video is handled as sampled frames (the DCVC neural codec in the base repo is an optional efficiency path and is not needed here).

## Links

- **Project website:** [mlx-optiq.com](https://mlx-optiq.com/)
- **All OptiQ quants:** [mlx-optiq.com/models](https://mlx-optiq.com/models)
- **Base model:** [microsoft/Mage-VL](https://huggingface.co/microsoft/Mage-VL)