Instructions to use saricles/MiniMax-M2.7-NVFP4-GB10 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use saricles/MiniMax-M2.7-NVFP4-GB10 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="saricles/MiniMax-M2.7-NVFP4-GB10", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("saricles/MiniMax-M2.7-NVFP4-GB10", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("saricles/MiniMax-M2.7-NVFP4-GB10", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use saricles/MiniMax-M2.7-NVFP4-GB10 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "saricles/MiniMax-M2.7-NVFP4-GB10"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "saricles/MiniMax-M2.7-NVFP4-GB10",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/saricles/MiniMax-M2.7-NVFP4-GB10

SGLang

How to use saricles/MiniMax-M2.7-NVFP4-GB10 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "saricles/MiniMax-M2.7-NVFP4-GB10" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "saricles/MiniMax-M2.7-NVFP4-GB10",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "saricles/MiniMax-M2.7-NVFP4-GB10" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "saricles/MiniMax-M2.7-NVFP4-GB10",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use saricles/MiniMax-M2.7-NVFP4-GB10 with Docker Model Runner:
```
docker model run hf.co/saricles/MiniMax-M2.7-NVFP4-GB10
```

model-00017-of-00027.safetensors is truncated in storage (4,998,786,780 of 5,033,633,825 bytes)

by Themanintheroom - opened 9 days ago

Discussion

Themanintheroom

9 days ago

Thanks for publishing this GB10-tuned quant — the published benchmarks match our 2x DGX Spark (GB10) setup nicely.

While downloading we found that model-00017-of-00027.safetensors cannot be fully downloaded: the stored object appears truncated at 4,998,786,780 bytes, while the repo metadata advertises 5,033,633,825 bytes (x-linked-etag a5353f3684c88badbb65e9dca4d2e628c6421cfab46b3d128f66f1d99887dbce).

Evidence, reproduced from two unrelated networks (NL consumer ISP and Hetzner DE):

hf download (huggingface_hub 1.14 and 1.24 + hf_xet) consistently fails with: File size mismatch: expected 5033633825 bytes but downloaded 4998786780
HEAD on the resolve URL: x-linked-size: 5033633825
GET -r 4998786770-4998786790 returns HTTP 206 with only 10 bytes (object ends at offset 4,998,786,779)
GET -r 4998786781- returns HTTP 416 Range Not Satisfiable
All other 26 shards download and verify fine

Could you re-upload this shard (or trigger a re-push of the repo)? Happy to verify afterwards — we have the other 26 shards cached and can test quickly on 2x GB10 hardware.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment