---
license: apache-2.0
base_model: microsoft/Mage-VL
tags:
- mlx
- apple-silicon
- vision-language
- image-text-to-text
- video
- streaming
pipeline_tag: image-text-to-text
library_name: mlx
---

# Mage-VL 4-bit (MLX, Apple Silicon)

An **MLX** (4-bit) conversion of [microsoft/Mage-VL](https://huggingface.co/microsoft/Mage-VL) —
a 5B image/video vision-language model (Qwen3-4B text backbone + a from-scratch
**Mage-ViT** "Codec-ViT" vision encoder) — that runs on Apple Silicon.

Port code, converter, and validators: **https://github.com/rsravanreddy/Mage-VL-MLX**

- Weights: `3.1 GB` (4-bit, group size 64)
- Runs image, video (frame-sampling), and the **streaming event gate** locally on a Mac.

## Usage

```bash
# 1. install the MLX stack + register the mage_vl plugin
pip install mlx mlx-lm mlx-vlm numpy pillow tokenizers jinja2 av
git clone https://github.com/rsravanreddy/Mage-VL-MLX && cd Mage-VL-MLX
ln -s "$PWD/mage_vl" "$(python -c 'import mlx_vlm,os;print(os.path.dirname(mlx_vlm.__file__))')/models/mage_vl"

# 2. download these weights
hf download sr29/Mage-VL-mlx-4bit --local-dir mage-vl-mlx

# 3. run (image or video)
python scripts/generate.py --mlx mage-vl-mlx --tokenizer-src mage-vl-mlx \
    --image path/to/image.jpg --prompt "Describe this image."
python scripts/generate.py --mlx mage-vl-mlx --tokenizer-src mage-vl-mlx \
    --video path/to/video.mp4 --num-frames 8 --prompt "What is happening?"
```

## Performance (Apple M4, 16GB)

| model | weights | image decode | image peak RAM |
|-------|--------:|-------------:|---------------:|
| 4-bit | 3.1 GB  | 30.6 tok/s   | 4.65 GB        |
| 8-bit | 5.0 GB  | 19.1 tok/s   | 6.55 GB        |

4-bit is recommended for 16GB; 8-bit gives richer output if you have RAM.

## Validation

- **Image preprocessing**: bit-exact vs the HF `Qwen2VLImageProcessor` (max_abs_diff 0.0).
- **Vision tower**: numerically matches the reference weights end-to-end
  (`max_abs_diff 3.0e-4`, fp32, full 24 layers).
- **Streaming Mamba mixer**: matches a canonical selective-scan reference (`4.3e-7`).
- **Generation**: qualitatively correct on image + video.
- **Not yet done**: full end-to-end logit parity vs the HF model (memory-gated);
  the codec (token-reduction) video backend needs the external codec engine.

## Streaming

Mage-VL's proactive **event gate** (`streammind_gate`) is ported
(`mage_vl/streaming.py`). See `scripts/stream.py` for a per-frame silent/speak
timeline. Note: the gate weights (`streammind_gate.safetensors`) are separate and
downloaded from the upstream Mage-VL repo.

## License & attribution

Apache-2.0. Derivative of [microsoft/Mage-VL](https://huggingface.co/microsoft/Mage-VL)
(Apache-2.0); reuses the Qwen3 language model from
[mlx-vlm](https://github.com/Blaizzy/mlx-vlm). Weights converted, not retrained.