Instructions to use olka-fi/MiniMax-M3-MXFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use olka-fi/MiniMax-M3-MXFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="olka-fi/MiniMax-M3-MXFP4", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("olka-fi/MiniMax-M3-MXFP4", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("olka-fi/MiniMax-M3-MXFP4", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use olka-fi/MiniMax-M3-MXFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "olka-fi/MiniMax-M3-MXFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "olka-fi/MiniMax-M3-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/olka-fi/MiniMax-M3-MXFP4

SGLang

How to use olka-fi/MiniMax-M3-MXFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "olka-fi/MiniMax-M3-MXFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "olka-fi/MiniMax-M3-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "olka-fi/MiniMax-M3-MXFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "olka-fi/MiniMax-M3-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use olka-fi/MiniMax-M3-MXFP4 with Docker Model Runner:
```
docker model run hf.co/olka-fi/MiniMax-M3-MXFP4
```

MiniMax-M3-MXFP4 / vllm_patch /README.md

olka-fi

Add files using upload-large-folder tool

b46b1ad verified 30 days ago

preview code

Raw

History Blame Contribute Delete

2.26 kB

vLLM runtime patch (ships with this MXFP4 checkpoint)

Why

This checkpoint quantizes the routed experts to MXFP4 (the rest stays native MXFP8). MiniMax-M3 uses the clamped SwiGLU-OAI activation (swiglu_limit: 7.0 in text_config).

The stock vllm/vllm-openai:minimax-m3 image has a bug in the compressed-tensors MXFP4 MoE method: get_fused_moe_quant_config() builds the kernel quant config without forwarding the SwiGLU clamp params, so gemm1_clamp_limit stays None and loading fails with:

AssertionError: SWIGLUOAI_UNINTERLEAVE requires clamp_limit

This is purely a runtime bug — the checkpoint and its config.json are correct (swiglu_limit=7.0 is reachable via hf_text_config).

The fix

compressed_tensors_moe_w4a4_mxfp4.py here is the patched file. It forwards swiglu_limit / swiglu_alpha / swiglu_beta (read from the already-populated MoE config) into the kernel quant config:

W4A16 (weight-only, this checkpoint's path): pass gemm1_alpha, gemm1_beta, swiglu_limit to make_mxfp4_moe_quant_config — which maps gemm1_clamp_limit = swiglu_limit per backend.
W4A4 (cutlass): set the fields on the returned config directly (that builder doesn't expose them).

Send the same change upstream to the fork.

How to apply (no image rebuild)

Bind-mount the file over the buggy one. Path is for the image's Python 3.12:

docker run --gpus all --privileged --ipc=host -p 8000:8000 \
  -v /mnt/storage:/root/.cache/huggingface \
  -v /mnt/storage/M3-MXFP4/vllm_patch/compressed_tensors_moe_w4a4_mxfp4.py:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w4a4_mxfp4.py \
  vllm/vllm-openai:minimax-m3 /root/.cache/huggingface/M3-MXFP4 \
  --block-size 128 \
  --tool-call-parser minimax_m3 --enable-auto-tool-choice \
  --reasoning-parser minimax_m3 \
  --load-format fastsafetensors

Verify after startup: the model loads past the activation assertion. If you bump the VRAM ceiling, add --gpu-memory-utilization 0.85 and a smaller --max-model-len (the 1M default makes the KV cache huge).

Patched against image vllm/vllm-openai:minimax-m3 (vllm 0.1.dev17492+g454b47db8).