Instructions to use cyankiwi/MiniMax-M2.1-AWQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cyankiwi/MiniMax-M2.1-AWQ-4bit with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cyankiwi/MiniMax-M2.1-AWQ-4bit", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cyankiwi/MiniMax-M2.1-AWQ-4bit", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("cyankiwi/MiniMax-M2.1-AWQ-4bit", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cyankiwi/MiniMax-M2.1-AWQ-4bit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cyankiwi/MiniMax-M2.1-AWQ-4bit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyankiwi/MiniMax-M2.1-AWQ-4bit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cyankiwi/MiniMax-M2.1-AWQ-4bit

SGLang

How to use cyankiwi/MiniMax-M2.1-AWQ-4bit with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cyankiwi/MiniMax-M2.1-AWQ-4bit" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyankiwi/MiniMax-M2.1-AWQ-4bit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cyankiwi/MiniMax-M2.1-AWQ-4bit" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyankiwi/MiniMax-M2.1-AWQ-4bit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cyankiwi/MiniMax-M2.1-AWQ-4bit with Docker Model Runner:
```
docker model run hf.co/cyankiwi/MiniMax-M2.1-AWQ-4bit
```

Tokenizer set to Mistral?

by zmarty - opened Dec 27, 2025

Discussion

zmarty

Dec 27, 2025

vllm serve
/models/awq/cyankiwi-MiniMax-M2.1-AWQ-4bit
--served-model-name MiniMax-M2.1-AWQ
--max-num-seqs 10
--max-model-len 128000
--gpu-memory-utilization 0.95
--tensor-parallel-size 2
--pipeline-parallel-size 1
--enable-auto-tool-choice
--tool-call-parser minimax_m2
--reasoning-parser minimax_m2_append_think
--trust-remote-code
--host 0.0.0.0
--port 8000

(APIServer pid=6372) The tokenizer you are loading from '/models/awq/cyankiwi-MiniMax-M2.1-AWQ-4bit' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the fix_mistral_regex=True flag when loading this tokenizer to fix this issue.

cpatonn

cyankiwi org Dec 29, 2025

Thank you for your interest in the model. This warning usually occurs when loads models in file directory format e.g., /models/awq/cyankiwi-MiniMax-M2.1-AWQ-4bit and not through the repo id e.g., cyankiwi/MiniMax-M2.1-AWQ-4bit.

Does the model work in your environment?

zmarty

Dec 29, 2025

Yes, the model works, thank you. Also what do you think about this Reddit comment? https://www.reddit.com/r/LocalLLaMA/s/w7OWQX7P7b

cpatonn

cyankiwi org Dec 29, 2025

Thank you for raising this comment to me. I am well awared of recent llm-compressor bugs, which I use llm-compressor version of more one month ago to quantize this model.

In addition, the model definition was modified during runtime to have all tokens routed to all experts for calibration :)

paolovic

Dec 30, 2025

•

edited Dec 30, 2025

did you have to set the flag fix_mistral_regex=True or does it run without it? @zmarty

zmarty

Jan 2

did you have to set the flag fix_mistral_regex=True or does it run without it? @zmarty

It runs without it. On 2x RTX Pro 6000

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment