Text Generation
Transformers
Safetensors
minimax_m2
neuralmagic
redhat
llmcompressor
quantized
FP4
conversational
custom_code
8-bit precision
compressed-tensors
Instructions to use RedHatAI/MiniMax-M2.5-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RedHatAI/MiniMax-M2.5-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="RedHatAI/MiniMax-M2.5-NVFP4", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("RedHatAI/MiniMax-M2.5-NVFP4", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("RedHatAI/MiniMax-M2.5-NVFP4", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use RedHatAI/MiniMax-M2.5-NVFP4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "RedHatAI/MiniMax-M2.5-NVFP4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RedHatAI/MiniMax-M2.5-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/RedHatAI/MiniMax-M2.5-NVFP4
- SGLang
How to use RedHatAI/MiniMax-M2.5-NVFP4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "RedHatAI/MiniMax-M2.5-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RedHatAI/MiniMax-M2.5-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "RedHatAI/MiniMax-M2.5-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RedHatAI/MiniMax-M2.5-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use RedHatAI/MiniMax-M2.5-NVFP4 with Docker Model Runner:
docker model run hf.co/RedHatAI/MiniMax-M2.5-NVFP4
File size: 3,711 Bytes
6cfaff9 c49ebe3 6cfaff9 c49ebe3 6cfaff9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | {
"schema_version": "0.2.2",
"evaluation_id": "aime25/RedHatAI/MiniMax-M2.5-NVFP4/1777382291.417811",
"evaluation_timestamp": "3303957",
"retrieved_timestamp": "1777382291.417811",
"source_metadata": {
"source_name": "lighteval",
"source_type": "evaluation_run",
"source_organization_name": "RedHatAI",
"evaluator_relationship": "third_party"
},
"eval_library": {
"name": "lighteval",
"version": "v0.13.0"
},
"model_info": {
"name": "RedHatAI/MiniMax-M2.5-NVFP4",
"id": "RedHatAI/MiniMax-M2.5-NVFP4",
"developer": "RedHatAI",
"inference_engine": {
"name": "vllm"
},
"additional_details": {
"provider": "hosted_vllm",
"base_url": "http://0.0.0.0:8003/v1",
"concurrent_requests": "8",
"verbose": "False",
"api_max_retry": "8",
"api_retry_sleep": "1.0",
"api_retry_multiplier": "2.0",
"timeout": "2400.0",
"num_seeds_merged": "8"
}
},
"evaluation_results": [
{
"evaluation_name": "aime25",
"source_data": {
"dataset_name": "aime25",
"source_type": "hf_dataset",
"hf_repo": "yentinglin/aime_2025",
"hf_split": "train"
},
"evaluation_timestamp": "3305786",
"metric_config": {
"evaluation_description": "pass@k:k=1&n=1",
"lower_is_better": false,
"score_type": "continuous",
"min_score": 0.0,
"max_score": 1.0
},
"score_details": {
"score": 0.7708333333333334,
"details": {
"seed_scores": "[0.8, 0.7, 0.8333333333333334, 0.7666666666666667, 0.8333333333333334, 0.7666666666666667, 0.6666666666666666, 0.8]",
"seed_values": "[1234, 1356, 3344, 4158, 42, 5322, 5678, 9843]"
},
"uncertainty": {
"standard_error": {
"value": 0.021304202581158678,
"method": "across_seeds"
},
"num_samples": 8
}
},
"generation_config": {
"generation_args": {
"temperature": 1.0,
"top_p": 0.95,
"top_k": 40.0,
"max_tokens": 64000,
"max_attempts": 1
},
"additional_details": {
"repetition_penalty": "1.0",
"presence_penalty": "1.5",
"seed": "1234",
"min_p": "0.0"
}
}
},
{
"evaluation_name": "aime25",
"source_data": {
"dataset_name": "aime25",
"source_type": "hf_dataset",
"hf_repo": "yentinglin/aime_2025",
"hf_split": "train"
},
"evaluation_timestamp": "3305786",
"metric_config": {
"evaluation_description": "avg@n:n=1",
"lower_is_better": false,
"score_type": "continuous",
"min_score": 0.0,
"max_score": 1.0
},
"score_details": {
"score": 0.7708333333333334,
"details": {
"seed_scores": "[0.8, 0.7, 0.8333333333333334, 0.7666666666666667, 0.8333333333333334, 0.7666666666666667, 0.6666666666666666, 0.8]",
"seed_values": "[1234, 1356, 3344, 4158, 42, 5322, 5678, 9843]"
},
"uncertainty": {
"standard_error": {
"value": 0.021304202581158678,
"method": "across_seeds"
},
"num_samples": 8
}
},
"generation_config": {
"generation_args": {
"temperature": 1.0,
"top_p": 0.95,
"top_k": 40.0,
"max_tokens": 64000,
"max_attempts": 1
},
"additional_details": {
"repetition_penalty": "1.0",
"presence_penalty": "1.5",
"seed": "1234",
"min_p": "0.0"
}
}
}
]
} |