---
base_model: MiniMaxAI/MiniMax-M3
license: other
license_name: minimax-community
license_link: LICENSE
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- minimax
- mixture-of-experts
- moe
- w4a16
- int4
- gptq
- compressed-tensors
- quantized
- vllm
---

# MiniMax-M3 — W4A16 (GPTQ)

4-bit weight quantization of **MiniMax-M3** produced with **GPTQ** via
[llm-compressor](https://github.com/vllm-project/llm-compressor), stored in the
**compressed-tensors** `pack-quantized` format for serving with vLLM / SGLang. Routed-expert
weights keep the base `w1/w2/w3` naming, so the checkpoint loads against the native M3
architecture.

## Quantization

| | |
|---|---|
| Scheme | W4A16 (4-bit weights, 16-bit activations) |
| Weights | int4, symmetric, **group size 128** |
| Embeddings | int4, group size 64 |
| `lm_head` | bf16 (kept full precision) |
| Left in bf16 | router gates, lightning-indexer projections, RMSNorms, vision tower |
| KV cache | bf16 |
| Method | GPTQ — Hessian error minimization with cross-layer error propagation |
| Format | compressed-tensors (`pack-quantized`) |

All 60 decoder layers are quantized (3 dense + 57 sparse/MoE, 128 experts each).

## Calibration

Calibrated on **deployment-realistic agentic trajectories** — 427 multi-turn tool-use
rollouts (~8M tokens) rendered through the M3 chat template at up to 32k sequence length,
padded with a small amount of general-instruction data. The goal is to match the model's real
serving distribution (multi-turn, tool-calling) rather than generic web text.

## Quality

Mean weight-reconstruction error vs the bf16 base is **≈0.125** relative (cosine ≈ 0.993) —
the expected magnitude for int4 group-128 quantization. This is a weight-space measure;
evaluate downstream quality on your own target benchmark.

## Usage

```bash
# requires a vLLM build with MiniMax-M3 support
vllm serve Sebesky/MiniMax-M3-W4A16-GPTQ
```

```python
from transformers import AutoModelForImageTextToText, AutoTokenizer
model = AutoModelForImageTextToText.from_pretrained("Sebesky/MiniMax-M3-W4A16-GPTQ", device_map="auto")
tok = AutoTokenizer.from_pretrained("Sebesky/MiniMax-M3-W4A16-GPTQ")
```

## License

Inherits the [MiniMax Community License](LICENSE) from the base model (non-commercial use).