Mage-VL-8bit

microsoft/Mage-VL (4B codec-native multimodal model, Apache-2.0) quantized to 8-bit for MLX on Apple Silicon.

Converted with mlx_vlm.convert (affine, group size 64, 9.023 bits/weight): the Qwen3-4B language model — embeddings, attention/MLP projections, lm_head — is int8; the Mage-ViT vision tower stays bf16. 5.0 GB on disk vs 9.5 GB upstream.

Quality

Gated against the bf16 checkpoint and the PyTorch reference on the model's own examples/dog.jpg, greedy decoding:

comparison	result
int8 vs PyTorch reference	48/48 tokens identical
int8 vs bf16 MLX	48/48 tokens identical
final-position logits vs reference	cos 0.9982 (bf16 baseline: 0.9996)

Measured on an M5 Max (macOS 27): resident floor 5.0 GB (bf16: 8.85), image-QA decode 88.3 tok/s (bf16: 54.9). Activation memory is unchanged by quantization — budget ~3 GB for a single image and ~14 GB for 16-frame video QA at a 4096-token budget.

Use

Python — the mage_vl architecture is in mlx-vlm PR #1745 (until it merges, install from the mage-vl branch). No trust_remote_code needed — the port ships a torch-free processor:

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("mlx-community/Mage-VL-8bit")
prompt = apply_chat_template(processor, model.config, "Describe this image in detail.", num_images=1)
print(generate(model, processor, prompt, image=["dog.jpg"], max_tokens=64).text)

Swift — xocialize/mage-vl-swift (≥ v0.2.0) serves this checkpoint as an MLXEngine package (imageAnalysis + videoAnalysis):

try await engine.register(
    MageVLPackage.registration,
    configuration: MageVLConfiguration(quant: .int8))

Provenance

Upstream weights: microsoft/Mage-VL, Apache-2.0. This repo excludes the DCVC neural-codec runtime and the StreamMind gate weights (CUDA-only / not used by the MLX ports). The neural_codec Python sources are retained verbatim from upstream for trust_remote_code completeness.