Hugging Face's logo

MiniMaxAI
/
MiniMax-M1-40k

Text Generation
Transformers
Safetensors
minimax_m1
vllm
conversational
custom_code
Model card Files
xet
Community
7
realolipop commited on
Commit
7248a14
·
verified ·
1 Parent(s): 32c3b05

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -52
README.md CHANGED
@@ -95,61 +95,28 @@ foundation for next-generation language model agents to reason and tackle real-w
95
 
96
  ## 2. Evaluation
97
 
98
- <!-- **Performance of MiniMax-M1 on core benchmarks.**
99
-
100
- | **Tasks** | **OpenAI-o3** | **Gemini 2.5<br>Pro (06-05)** | **Claude<br>4 Opus** | **Seed-<br>Thinking-<br>v1.5** | **DeepSeek-<br>R1** | **DeepSeek-<br>R1-0528** | **Qwen3-<br>235B-A22B** | **MiniMax-<br>M1-40K** | **MiniMax-<br>M1-80K** |
101
- |:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
102
- | *Extended<br>Thinking* | *100k* | *64k* | *64k* | *32k* | *32k* | *64k* | *32k* | *40K* | *80K* |
103
- | ***Mathematics*** |
104
- | AIME 2024 | 91.6 | 92.0 | 76.0 | 86.7 | 79.8 | 91.4 | 85.7 | 83.3 | 86.0 |
105
- | AIME 2025 | 88.9 | 88.0 | 75.5 | 74.0 | 70.0 | 87.5 | 81.5 | 74.6 | 76.9 |
106
- | MATH-500 | 98.1 | 98.8 | 98.2 | 96.7 | 97.3 | 98.0 | 96.2 | 96.0 | 96.8 |
107
- | ***General Coding*** |
108
- | LiveCodeBench<br>*(24/8~25/5)* | 75.8 | 77.1 | 56.6 | 67.5 | 55.9 | 73.1 | 65.9 | 62.3 | 65.0 |
109
- | FullStackBench | 69.3 | -- | 70.3 | 69.9 | 70.1 | 69.4 | 62.9 | 67.6 | 68.3 |
110
- | ***Reasoning & Knowledge*** |
111
- | GPQA Diamond | 83.3 | 86.4 | 79.6 | 77.3 | 71.5 | 81.0 | 71.1 | 69.2 | 70.0 |
112
- | HLE *(no tools)* | 20.3 | 21.6 | 10.7 | 8.2 | 8.6\* | 17.7\* | 7.6\* | 7.2\* | 8.4\* |
113
- | ZebraLogic | 95.8 | 91.6 | 95.1 | 84.4 | 78.7 | 95.1 | 80.3 | 80.1 | 86.8 |
114
- | MMLU-Pro | 85.0 | 86.0 | 85.0 | 87.0 | 84.0 | 85.0 | 83.0 | 80.6 | 81.1 |
115
- | ***Software Engineering*** |
116
- | SWE-bench Verified| 69.1 | 67.2 | 72.5 | 47.0 | 49.2 | 57.6 | 34.4 | 55.6 | 56.0 |
117
- | ***Long Context*** |
118
- | OpenAI-MRCR *(128k)* | 56.5 | 76.8 | 48.9 | 54.3 | 35.8 | 51.5 | 27.7 | 76.1 | 73.4 |
119
- | OpenAI-MRCR *(1M)* | -- | 58.8 | -- | -- | -- | -- | -- | 58.6 | 56.2 |
120
- | LongBench-v2 | 58.8 | 65.0 | 55.6 | 52.5 | 58.3 | 52.1 | 50.1 | 61.0 | 61.5 |
121
- | ***Agentic Tool Use*** |
122
- | TAU-bench *(airline)* | 52.0 | 50.0 | 59.6 | 44.0 | -- | 53.5 | 34.7 | 60.0 | 62.0 |
123
- | TAU-bench *(retail)* | 73.9 | 67.0 | 81.4 | 55.7 | -- | 63.9 | 58.6 | 67.8 | 63.5 |
124
- | ***Factuality*** |
125
- | SimpleQA | 49.4 | 54.0 | -- | 12.9 | 30.1 | 27.8 | 11.0 | 17.9 | 18.5 |
126
- | ***General Assistant*** |
127
- | MultiChallenge | 56.5 | 51.8 | 45.8 | 43.0 | 40.7 | 45.0 | 40.0 | 44.7 | 44.7 |
128
-
129
- \* conducted on the text-only HLE subset. -->
130
-
131
  **Performance of MiniMax-M1 on core benchmarks.**
132
 
133
- | **Category** | **Task** | **OpenAI-o3** | **Gemini 2.5 Pro (06-05)** | **Claude 4 Opus** | **Seed-Thinking-v1.5** | **DeepSeek-R1** | **DeepSeek-R1-0528** | **Qwen3-235B-A22B** | **MiniMax-M1-40K** | **MiniMax-M1-80K** |
134
  |:---|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
135
- | | *Extended Thinking* | *100k* | *64k* | *64k* | *32k* | *32k* | *64k* | *32k* | *40K* | *80K* |
136
- | ***Mathematics*** | AIME 2024 | 91.6 | 92.0 | 76.0 | 86.7 | 79.8 | 91.4 | 85.7 | 83.3 | 86.0 |
137
- | | AIME 2025 | 88.9 | 88.0 | 75.5 | 74.0 | 70.0 | 87.5 | 81.5 | 74.6 | 76.9 |
138
- | | MATH-500 | 98.1 | 98.8 | 98.2 | 96.7 | 97.3 | 98.0 | 96.2 | 96.0 | 96.8 |
139
- | ***General Coding*** | LiveCodeBench *(24/8~25/5)* | 75.8 | 77.1 | 56.6 | 67.5 | 55.9 | 73.1 | 65.9 | 62.3 | 65.0 |
140
- | | FullStackBench | 69.3 | -- | 70.3 | 69.9 | 70.1 | 69.4 | 62.9 | 67.6 | 68.3 |
141
- | ***Reasoning & Knowledge***| GPQA Diamond | 83.3 | 86.4 | 79.6 | 77.3 | 71.5 | 81.0 | 71.1 | 69.2 | 70.0 |
142
- | | HLE *(no tools)* | 20.3 | 21.6 | 10.7 | 8.2 | 8.6\* | 17.7\* | 7.6\* | 7.2\* | 8.4\* |
143
- | | ZebraLogic | 95.8 | 91.6 | 95.1 | 84.4 | 78.7 | 95.1 | 80.3 | 80.1 | 86.8 |
144
- | | MMLU-Pro | 85.0 | 86.0 | 85.0 | 87.0 | 84.0 | 85.0 | 83.0 | 80.6 | 81.1 |
145
- | ***Software Engineering***| SWE-bench Verified| 69.1 | 67.2 | 72.5 | 47.0 | 49.2 | 57.6 | 34.4 | 55.6 | 56.0 |
146
- | ***Long Context*** | OpenAI-MRCR *(128k)* | 56.5 | 76.8 | 48.9 | 54.3 | 35.8 | 51.5 | 27.7 | 76.1 | 73.4 |
147
- | | OpenAI-MRCR *(1M)* | -- | 58.8 | -- | -- | -- | -- | -- | 58.6 | 56.2 |
148
- | | LongBench-v2 | 58.8 | 65.0 | 55.6 | 52.5 | 58.3 | 52.1 | 50.1 | 61.0 | 61.5 |
149
- | ***Agentic Tool Use***| TAU-bench *(airline)* | 52.0 | 50.0 | 59.6 | 44.0 | -- | 53.5 | 34.7 | 60.0 | 62.0 |
150
- | | TAU-bench *(retail)* | 73.9 | 67.0 | 81.4 | 55.7 | -- | 63.9 | 58.6 | 67.8 | 63.5 |
151
- | ***Factuality*** | SimpleQA | 49.4 | 54.0 | -- | 12.9 | 30.1 | 27.8 | 11.0 | 17.9 | 18.5 |
152
- | ***General Assistant***| MultiChallenge | 56.5 | 51.8 | 45.8 | 43.0 | 40.7 | 45.0 | 40.0 | 44.7 | 44.7 |
153
 
154
  \* conducted on the text-only HLE subset.
155
 
 
95
 
96
  ## 2. Evaluation
97
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
98
  **Performance of MiniMax-M1 on core benchmarks.**
99
 
100
+ | **Category** | **Task** | **MiniMax-M1-40K** | **MiniMax-M1-80K** | **OpenAI-o3** | **Gemini 2.5 Pro (06-05)** | **Claude 4 Opus** | **Seed-Thinking-v1.5** | **DeepSeek-R1** | **DeepSeek-R1-0528** | **Qwen3-235B-A22B** |
101
  |:---|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
102
+ | | *Extended Thinking* | *40K* | *80K* | *100k* | *64k* | *64k* | *32k* | *32k* | *64k* | *32k* |
103
+ | ***Mathematics*** | AIME 2024 | 83.3 | 86.0 | 91.6 | 92.0 | 76.0 | 86.7 | 79.8 | 91.4 | 85.7 |
104
+ | | AIME 2025 | 74.6 | 76.9 | 88.9 | 88.0 | 75.5 | 74.0 | 70.0 | 87.5 | 81.5 |
105
+ | | MATH-500 | 96.0 | 96.8 | 98.1 | 98.8 | 98.2 | 96.7 | 97.3 | 98.0 | 96.2 |
106
+ | ***General Coding*** | LiveCodeBench *(24/8~25/5)* | 62.3 | 65.0 | 75.8 | 77.1 | 56.6 | 67.5 | 55.9 | 73.1 | 65.9 |
107
+ | | FullStackBench | 67.6 | 68.3 | 69.3 | -- | 70.3 | 69.9 | 70.1 | 69.4 | 62.9 |
108
+ | ***Reasoning & Knowledge***| GPQA Diamond | 69.2 | 70.0 | 83.3 | 86.4 | 79.6 | 77.3 | 71.5 | 81.0 | 71.1 |
109
+ | | HLE *(no tools)* | 7.2\* | 8.4\* | 20.3 | 21.6 | 10.7 | 8.2 | 8.6\* | 17.7\* | 7.6\* |
110
+ | | ZebraLogic | 80.1 | 86.8 | 95.8 | 91.6 | 95.1 | 84.4 | 78.7 | 95.1 | 80.3 |
111
+ | | MMLU-Pro | 80.6 | 81.1 | 85.0 | 86.0 | 85.0 | 87.0 | 84.0 | 85.0 | 83.0 |
112
+ | ***Software Engineering***| SWE-bench Verified| 55.6 | 56.0 | 69.1 | 67.2 | 72.5 | 47.0 | 49.2 | 57.6 | 34.4 |
113
+ | ***Long Context*** | OpenAI-MRCR *(128k)* | 76.1 | 73.4 | 56.5 | 76.8 | 48.9 | 54.3 | 35.8 | 51.5 | 27.7 |
114
+ | | OpenAI-MRCR *(1M)* | 58.6 | 56.2 | -- | 58.8 | -- | -- | -- | -- | -- |
115
+ | | LongBench-v2 | 61.0 | 61.5 | 58.8 | 65.0 | 55.6 | 52.5 | 58.3 | 52.1 | 50.1 |
116
+ | ***Agentic Tool Use***| TAU-bench *(airline)* | 60.0 | 62.0 | 52.0 | 50.0 | 59.6 | 44.0 | -- | 53.5 | 34.7 |
117
+ | | TAU-bench *(retail)* | 67.8 | 63.5 | 73.9 | 67.0 | 81.4 | 55.7 | -- | 63.9 | 58.6 |
118
+ | ***Factuality*** | SimpleQA | 17.9 | 18.5 | 49.4 | 54.0 | -- | 12.9 | 30.1 | 27.8 | 11.0 |
119
+ | ***General Assistant***| MultiChallenge | 44.7 | 44.7 | 56.5 | 51.8 | 45.8 | 43.0 | 40.7 | 45.0 | 40.0 |
120
 
121
  \* conducted on the text-only HLE subset.
122