Instructions to use ajh-code/Mage-VL-XPO3-NVFP4-W4A4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ajh-code/Mage-VL-XPO3-NVFP4-W4A4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ajh-code/Mage-VL-XPO3-NVFP4-W4A4", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ajh-code/Mage-VL-XPO3-NVFP4-W4A4", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ajh-code/Mage-VL-XPO3-NVFP4-W4A4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ajh-code/Mage-VL-XPO3-NVFP4-W4A4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajh-code/Mage-VL-XPO3-NVFP4-W4A4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ajh-code/Mage-VL-XPO3-NVFP4-W4A4

SGLang

How to use ajh-code/Mage-VL-XPO3-NVFP4-W4A4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ajh-code/Mage-VL-XPO3-NVFP4-W4A4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajh-code/Mage-VL-XPO3-NVFP4-W4A4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ajh-code/Mage-VL-XPO3-NVFP4-W4A4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajh-code/Mage-VL-XPO3-NVFP4-W4A4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ajh-code/Mage-VL-XPO3-NVFP4-W4A4 with Docker Model Runner:
```
docker model run hf.co/ajh-code/Mage-VL-XPO3-NVFP4-W4A4
```

Mage-VL · XPO3 NVFP4 W4A4 · Runtime V2

Native Blackwell NVFP4 with a fast hybrid default and an exact shared-activation pure-W4A4 profile.

Our Quantization series combines the best of Speed, Quality, and Size.

Original Mage-VL · FP8 W8A8/W8A16 · Arands.com · updates

Download

Component	Purpose	Size
`model-*.safetensors`	Complete Mage-VL checkpoint	4.26 GB
`streammind_gate.safetensors`	Upstream proactive streaming gate	1.07 GB
Complete repository	Runtime, processor, codec support, and weights	5.62 GB

The root config.json is a recognized Hugging Face query file, so downloads are tracked and this repository appears as a quantization of microsoft/Mage-VL.

Quick start

Tested on Linux x86-64, Python 3.11, CUDA 13.1, PyTorch 2.13.0+cu130, comfy-kitchen==0.2.22, and NVIDIA Blackwell SM120.

hf download ajh-code/Mage-VL-XPO3-NVFP4-W4A4 --local-dir mage-vl-quant
cd mage-vl-quant
python3.11 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt

CUDA_HOME=/usr/local/cuda-13.1 CUDA_VISIBLE_DEVICES=0 \
  python inference.py --mode offline --model . --image image.jpg \
  --question "Describe this image in detail."

Runtime V2 ships two profiles without changing the checkpoint weights:

Profile	Activation policy	Intended use
`hybrid_fast` (default)	W4A4 prefill, packed-weight W4A16 at `M=1`	Recommended speed
`pure_w4a4_v2`	W4A4 for every quantized language-projection GEMM	Pure-W4A4 research and deployment

Select the pure-W4A4 V2 profile:

MAGE_VL_RUNTIME_PROFILE=pure_w4a4_v2 CUDA_VISIBLE_DEVICES=0 \
  python inference.py --mode offline --model . --image image.jpg \
  --question "Describe this image."

The pure profile independently shares one exact dynamic NVFP4 activation across Q/K/V and another across gate/up at M=1. Disable either feature for isolation, or both for the exact ordinary pure-W4A4 fallback:

MAGE_VL_RUNTIME_PROFILE=pure_w4a4_v2 \
MAGE_VL_W4A4_SHARED_QKV_ACTIVATION=0 \
MAGE_VL_W4A4_SHARED_GATE_UP_ACTIVATION=0 \
CUDA_VISIBLE_DEVICES=0 \
  python inference.py --mode offline --model . --image image.jpg \
  --question "Describe this image."

Measured performance

Runtime V2 packaged profiles — RTX 5080

Same packaged repository, 1,657-token multimodal prompt, greedy decode, exactly 32 new tokens, three warmups, 15 measurements, median total generate() latency:

Profile	Generate 32	Peak process allocation	Output gate
Ordinary pure W4A4, sharing off	2.4953 s	4,663 MiB	exact control
`pure_w4a4_v2`	2.0957 s	4,663 MiB	exact to control
`hybrid_fast` (default)	0.7178 s	4,663 MiB	accepted hybrid output

The V2 sharing stack reduces matched pure-W4A4 latency by 16.0% (1.191x throughput) with identical generated tokens. That is a comparison against ordinary pure W4A4, not against the hybrid default. hybrid_fast remains the recommended speed profile and is 2.92x faster than pure_w4a4_v2 in this protocol.

Earlier three-way control — RTX 5060 Ti

The prior 1,657-token, fixed-32 protocol provides the BF16/FP8 context:

Runtime	Resident allocation	Prefill	Generate 32	Derived decode
BF16	9,044 MiB	0.6011 s	1.4277 s	36.69 tok/s
Ordinary FP8 W8A8	5,579 MiB	0.4541 s	2.0543 s	19.14 tok/s
FP8 W8A8/W8A16 fused	5,579 MiB	0.4535 s	1.0177 s	53.21 tok/s
XPO3 NVFP4 W4A4/W4A16	4,063 MiB	0.4233 s	1.0120 s	51.14 tok/s

These are matched local measurements, not universal end-to-end claims. On the 5060 Ti control, XPO3 was 1.41x faster than BF16 for fixed work and used 55.1% less resident model allocation.

What changed in Runtime V2

Added named hybrid_fast and pure_w4a4_v2 profiles.
Added independently configurable exact activation sharing for Q/K/V and gate/up at M=1.
Preserved exact ordinary-W4A4 fallback for larger M and feature-off runs.
Kept both model shards, the checkpoint index, and StreamMind weights byte-for-byte unchanged; existing users only need the updated small files.
Passed exact repeat, unmatched-call, M=2 fallback, restore/reinstall, 32-token packaged generation, and 252-payload immutability gates.

Quantization policy

All 252 Qwen3 language projections use the native packed NVFP4 checkpoint. Large-token prefill runs W4A4. In hybrid_fast, M=1 reads those same packed weights through a specialized BF16-activation/FP32-accumulation GEMV, avoiding activation-quantization and padded-GEMM overhead. In pure_w4a4_v2, every quantized language-projection GEMM uses NVFP4 weights and NVFP4 activations while sharing the exact activation conversion where multiple projections consume the same tensor. No second weight copy is stored.

“Pure W4A4” describes those 252 language-projection GEMMs, not the complete end-to-end model: Mage-ViT, embeddings, LM head, normalization, RoPE, StreamMind, outputs, and accumulators remain at their documented higher precision. The ten-image gate found no new sharing-specific failure and was 10/10 exact against same-device ordinary W4A4 for tokens, decoded answers, and saved final-logit tensor hashes.

Validated scope

Gate	Result
Quantized language projections	252 / 252
Image suite	Ten deterministic description, OCR, count, spatial, and detail cases
Packaging	Complete sharded Safetensors checkpoint with root `config.json`
Unchanged BF16 components	Mage-ViT, embeddings, LM head, norms, RoPE, StreamMind
Current hardware target	NVIDIA Blackwell SM120

Image understanding is the validated release path. The upstream video, codec, and StreamMind files are retained for completeness, but their quantized end-to-end paths have not yet received the same release gate.

Validate the download

python validate_release.py

MANIFEST.json records the size and SHA-256 of every distributed file except itself. Hashing the model and StreamMind checkpoints takes a little while.

License and attribution

Mage-VL and this derivative package are released under Apache-2.0. The model architecture, processor, codec utilities, and original BF16 weights derive from microsoft/Mage-VL. The quantized runtime modifications are identified in the bundled source and THIRD_PARTY_NOTICES.md.

Downloads last month: -

Safetensors

Model size

3B params

Tensor type

BF16

F8_E4M3

Model tree for ajh-code/Mage-VL-XPO3-NVFP4-W4A4

Base model

microsoft/Mage-VL

Quantized

(8)

this model