Image-Text-to-Text
Transformers
Safetensors
kimi_k3
feature-extraction
compressed-tensors
conversational
custom_code
Eval Results
8-bit precision
Instructions to use moonshotai/Kimi-K3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moonshotai/Kimi-K3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K3", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("moonshotai/Kimi-K3", trust_remote_code=True, device_map="auto") - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use moonshotai/Kimi-K3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "moonshotai/Kimi-K3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/moonshotai/Kimi-K3
- SGLang
How to use moonshotai/Kimi-K3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "moonshotai/Kimi-K3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "moonshotai/Kimi-K3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use moonshotai/Kimi-K3 with Docker Model Runner:
docker model run hf.co/moonshotai/Kimi-K3
Update README.md
#67
by usermma - opened
README.md
CHANGED
|
@@ -162,7 +162,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 162 |
<td align="center" style="vertical-align: middle; text-align: center">GPQA Diamond</td>
|
| 163 |
<td align="center" style="vertical-align: middle; text-align: center">93.5</td>
|
| 164 |
<td align="center" style="vertical-align: middle; text-align: center">92.6</td>
|
| 165 |
-
<td align="center" style="vertical-align: middle; text-align: center">94.1</td>
|
| 166 |
<td align="center" style="vertical-align: middle; text-align: center">91.0</td>
|
| 167 |
<td align="center" style="vertical-align: middle; text-align: center">93.5</td>
|
| 168 |
<td align="center" style="vertical-align: middle; text-align: center">91.2</td>
|
|
@@ -171,14 +171,14 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 171 |
<td align="center" style="vertical-align: middle; text-align: center">CritPt</td>
|
| 172 |
<td align="center" style="vertical-align: middle; text-align: center">23.4</td>
|
| 173 |
<td align="center" style="vertical-align: middle; text-align: center">28.6</td>
|
| 174 |
-
<td align="center" style="vertical-align: middle; text-align: center">32.3</td>
|
| 175 |
<td align="center" style="vertical-align: middle; text-align: center">20.9</td>
|
| 176 |
<td align="center" style="vertical-align: middle; text-align: center">27.1</td>
|
| 177 |
<td align="center" style="vertical-align: middle; text-align: center">20.9</td>
|
| 178 |
</tr>
|
| 179 |
<tr>
|
| 180 |
<td align="center" style="vertical-align: middle; text-align: center">AA-LCR</td>
|
| 181 |
-
<td align="center" style="vertical-align: middle; text-align: center">74.7</td>
|
| 182 |
<td align="center" style="vertical-align: middle; text-align: center">70.0</td>
|
| 183 |
<td align="center" style="vertical-align: middle; text-align: center">73.7</td>
|
| 184 |
<td align="center" style="vertical-align: middle; text-align: center">67.7</td>
|
|
@@ -188,7 +188,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 188 |
<tr>
|
| 189 |
<td align="center" style="vertical-align: middle; text-align: center">HLE-Full</td>
|
| 190 |
<td align="center" style="vertical-align: middle; text-align: center">43.5 / 56.0</td>
|
| 191 |
-
<td align="center" style="vertical-align: middle; text-align: center">53.3 / 63.0</td>
|
| 192 |
<td align="center" style="vertical-align: middle; text-align: center">44.5 / 58.0</td>
|
| 193 |
<td align="center" style="vertical-align: middle; text-align: center">49.8 / 57.9</td>
|
| 194 |
<td align="center" style="vertical-align: middle; text-align: center">41.4 / 52.2</td>
|
|
@@ -201,14 +201,14 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 201 |
<td align="center" style="vertical-align: middle; text-align: center">DeepSWE</td>
|
| 202 |
<td align="center" style="vertical-align: middle; text-align: center">67.5</td>
|
| 203 |
<td align="center" style="vertical-align: middle; text-align: center">70.0</td>
|
| 204 |
-
<td align="center" style="vertical-align: middle; text-align: center">73.0</td>
|
| 205 |
<td align="center" style="vertical-align: middle; text-align: center">59.0</td>
|
| 206 |
<td align="center" style="vertical-align: middle; text-align: center">67.0</td>
|
| 207 |
<td align="center" style="vertical-align: middle; text-align: center">46.2</td>
|
| 208 |
</tr>
|
| 209 |
<tr>
|
| 210 |
<td align="center" style="vertical-align: middle; text-align: center">ProgramBench</td>
|
| 211 |
-
<td align="center" style="vertical-align: middle; text-align: center">77.8</td>
|
| 212 |
<td align="center" style="vertical-align: middle; text-align: center">76.8</td>
|
| 213 |
<td align="center" style="vertical-align: middle; text-align: center">77.6</td>
|
| 214 |
<td align="center" style="vertical-align: middle; text-align: center">71.9</td>
|
|
@@ -219,7 +219,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 219 |
<td align="center" style="vertical-align: middle; text-align: center">Terminal-Bench 2.1</td>
|
| 220 |
<td align="center" style="vertical-align: middle; text-align: center">88.3</td>
|
| 221 |
<td align="center" style="vertical-align: middle; text-align: center">88.0</td>
|
| 222 |
-
<td align="center" style="vertical-align: middle; text-align: center">88.8</td>
|
| 223 |
<td align="center" style="vertical-align: middle; text-align: center">84.6</td>
|
| 224 |
<td align="center" style="vertical-align: middle; text-align: center">83.4</td>
|
| 225 |
<td align="center" style="vertical-align: middle; text-align: center">82.7</td>
|
|
@@ -227,7 +227,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 227 |
<tr>
|
| 228 |
<td align="center" style="vertical-align: middle; text-align: center">FrontierSWE</td>
|
| 229 |
<td align="center" style="vertical-align: middle; text-align: center">81.2</td>
|
| 230 |
-
<td align="center" style="vertical-align: middle; text-align: center">86.6</td>
|
| 231 |
<td align="center" style="vertical-align: middle; text-align: center">71.3</td>
|
| 232 |
<td align="center" style="vertical-align: middle; text-align: center">66.7</td>
|
| 233 |
<td align="center" style="vertical-align: middle; text-align: center">64.9</td>
|
|
@@ -235,7 +235,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 235 |
</tr>
|
| 236 |
<tr>
|
| 237 |
<td align="center" style="vertical-align: middle; text-align: center">SWE-Marathon</td>
|
| 238 |
-
<td align="center" style="vertical-align: middle; text-align: center">42.0</td>
|
| 239 |
<td align="center" style="vertical-align: middle; text-align: center">35.0</td>
|
| 240 |
<td align="center" style="vertical-align: middle; text-align: center">39.0</td>
|
| 241 |
<td align="center" style="vertical-align: middle; text-align: center">40.0</td>
|
|
@@ -245,7 +245,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 245 |
<tr>
|
| 246 |
<td align="center" style="vertical-align: middle; text-align: center">PostTrainBench</td>
|
| 247 |
<td align="center" style="vertical-align: middle; text-align: center">36.6</td>
|
| 248 |
-
<td align="center" style="vertical-align: middle; text-align: center">41.4</td>
|
| 249 |
<td align="center" style="vertical-align: middle; text-align: center">34.6</td>
|
| 250 |
<td align="center" style="vertical-align: middle; text-align: center">34.1</td>
|
| 251 |
<td align="center" style="vertical-align: middle; text-align: center">28.4</td>
|
|
@@ -254,7 +254,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 254 |
<tr>
|
| 255 |
<td align="center" style="vertical-align: middle; text-align: center">MLS-Bench-Lite</td>
|
| 256 |
<td align="center" style="vertical-align: middle; text-align: center">48.3</td>
|
| 257 |
-
<td align="center" style="vertical-align: middle; text-align: center">49.9</td>
|
| 258 |
<td align="center" style="vertical-align: middle; text-align: center">46.2</td>
|
| 259 |
<td align="center" style="vertical-align: middle; text-align: center">42.8</td>
|
| 260 |
<td align="center" style="vertical-align: middle; text-align: center">35.5</td>
|
|
@@ -263,7 +263,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 263 |
<tr>
|
| 264 |
<td align="center" style="vertical-align: middle; text-align: center">SciCode</td>
|
| 265 |
<td align="center" style="vertical-align: middle; text-align: center">58.7</td>
|
| 266 |
-
<td align="center" style="vertical-align: middle; text-align: center">60.2</td>
|
| 267 |
<td align="center" style="vertical-align: middle; text-align: center">56.1</td>
|
| 268 |
<td align="center" style="vertical-align: middle; text-align: center">53.5</td>
|
| 269 |
<td align="center" style="vertical-align: middle; text-align: center">56.1</td>
|
|
@@ -272,7 +272,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 272 |
<tr>
|
| 273 |
<td align="center" style="vertical-align: middle; text-align: center">Kimi Code Bench 2.0</td>
|
| 274 |
<td align="center" style="vertical-align: middle; text-align: center">72.9</td>
|
| 275 |
-
<td align="center" style="vertical-align: middle; text-align: center">76.9</td>
|
| 276 |
<td align="center" style="vertical-align: middle; text-align: center">64.8</td>
|
| 277 |
<td align="center" style="vertical-align: middle; text-align: center">71.7</td>
|
| 278 |
<td align="center" style="vertical-align: middle; text-align: center">69.0</td>
|
|
@@ -283,7 +283,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 283 |
</tr>
|
| 284 |
<tr>
|
| 285 |
<td align="center" style="vertical-align: middle; text-align: center">BrowseComp</td>
|
| 286 |
-
<td align="center" style="vertical-align: middle; text-align: center">91.2</td>
|
| 287 |
<td align="center" style="vertical-align: middle; text-align: center">88.0</td>
|
| 288 |
<td align="center" style="vertical-align: middle; text-align: center">90.4</td>
|
| 289 |
<td align="center" style="vertical-align: middle; text-align: center">84.3</td>
|
|
@@ -292,7 +292,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 292 |
</tr>
|
| 293 |
<tr>
|
| 294 |
<td align="center" style="vertical-align: middle; text-align: center">DeepSearchQA (F1)</td>
|
| 295 |
-
<td align="center" style="vertical-align: middle; text-align: center">95.0</td>
|
| 296 |
<td align="center" style="vertical-align: middle; text-align: center">94.2</td>
|
| 297 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 298 |
<td align="center" style="vertical-align: middle; text-align: center">93.1</td>
|
|
@@ -301,7 +301,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 301 |
</tr>
|
| 302 |
<tr>
|
| 303 |
<td align="center" style="vertical-align: middle; text-align: center">ResearchRubrics</td>
|
| 304 |
-
<td align="center" style="vertical-align: middle; text-align: center">76.2</td>
|
| 305 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 306 |
<td align="center" style="vertical-align: middle; text-align: center">73.8</td>
|
| 307 |
<td align="center" style="vertical-align: middle; text-align: center">73.5</td>
|
|
@@ -311,7 +311,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 311 |
<tr>
|
| 312 |
<td align="center" style="vertical-align: middle; text-align: center">GDPval-AA v2 (Elo)</td>
|
| 313 |
<td align="center" style="vertical-align: middle; text-align: center">1686</td>
|
| 314 |
-
<td align="center" style="vertical-align: middle; text-align: center">1747</td>
|
| 315 |
<td align="center" style="vertical-align: middle; text-align: center">1736</td>
|
| 316 |
<td align="center" style="vertical-align: middle; text-align: center">1593</td>
|
| 317 |
<td align="center" style="vertical-align: middle; text-align: center">1491</td>
|
|
@@ -320,7 +320,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 320 |
<tr>
|
| 321 |
<td align="center" style="vertical-align: middle; text-align: center">Toolathlon-Verified</td>
|
| 322 |
<td align="center" style="vertical-align: middle; text-align: center">76.5</td>
|
| 323 |
-
<td align="center" style="vertical-align: middle; text-align: center">77.9</td>
|
| 324 |
<td align="center" style="vertical-align: middle; text-align: center">74.9</td>
|
| 325 |
<td align="center" style="vertical-align: middle; text-align: center">76.2</td>
|
| 326 |
<td align="center" style="vertical-align: middle; text-align: center">73.5</td>
|
|
@@ -328,7 +328,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 328 |
</tr>
|
| 329 |
<tr>
|
| 330 |
<td align="center" style="vertical-align: middle; text-align: center">MCPMark-Verified</td>
|
| 331 |
-
<td align="center" style="vertical-align: middle; text-align: center">94.5</td>
|
| 332 |
<td align="center" style="vertical-align: middle; text-align: center">87.4</td>
|
| 333 |
<td align="center" style="vertical-align: middle; text-align: center">92.9</td>
|
| 334 |
<td align="center" style="vertical-align: middle; text-align: center">76.4</td>
|
|
@@ -338,7 +338,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 338 |
<tr>
|
| 339 |
<td align="center" style="vertical-align: middle; text-align: center">MCP-Atlas</td>
|
| 340 |
<td align="center" style="vertical-align: middle; text-align: center">84.2</td>
|
| 341 |
-
<td align="center" style="vertical-align: middle; text-align: center">84.7</td>
|
| 342 |
<td align="center" style="vertical-align: middle; text-align: center">83.6</td>
|
| 343 |
<td align="center" style="vertical-align: middle; text-align: center">83.6</td>
|
| 344 |
<td align="center" style="vertical-align: middle; text-align: center">82.8</td>
|
|
@@ -346,7 +346,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 346 |
</tr>
|
| 347 |
<tr>
|
| 348 |
<td align="center" style="vertical-align: middle; text-align: center">AutomationBench</td>
|
| 349 |
-
<td align="center" style="vertical-align: middle; text-align: center">30.8</td>
|
| 350 |
<td align="center" style="vertical-align: middle; text-align: center">29.1</td>
|
| 351 |
<td align="center" style="vertical-align: middle; text-align: center">29.7</td>
|
| 352 |
<td align="center" style="vertical-align: middle; text-align: center">27.2</td>
|
|
@@ -356,7 +356,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 356 |
<tr>
|
| 357 |
<td align="center" style="vertical-align: middle; text-align: center">JobBench</td>
|
| 358 |
<td align="center" style="vertical-align: middle; text-align: center">54.3</td>
|
| 359 |
-
<td align="center" style="vertical-align: middle; text-align: center">57.4</td>
|
| 360 |
<td align="center" style="vertical-align: middle; text-align: center">45.4</td>
|
| 361 |
<td align="center" style="vertical-align: middle; text-align: center">48.4</td>
|
| 362 |
<td align="center" style="vertical-align: middle; text-align: center">38.3</td>
|
|
@@ -365,7 +365,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 365 |
<tr>
|
| 366 |
<td align="center" style="vertical-align: middle; text-align: center">AA-Briefcase (Elo)</td>
|
| 367 |
<td align="center" style="vertical-align: middle; text-align: center">1548</td>
|
| 368 |
-
<td align="center" style="vertical-align: middle; text-align: center">1583</td>
|
| 369 |
<td align="center" style="vertical-align: middle; text-align: center">1495</td>
|
| 370 |
<td align="center" style="vertical-align: middle; text-align: center">1354</td>
|
| 371 |
<td align="center" style="vertical-align: middle; text-align: center">1158</td>
|
|
@@ -375,7 +375,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 375 |
<td align="center" style="vertical-align: middle; text-align: center">Agents' Last Exam</td>
|
| 376 |
<td align="center" style="vertical-align: middle; text-align: center">28.3</td>
|
| 377 |
<td align="center" style="vertical-align: middle; text-align: center">25.7<sup>†</sup></td>
|
| 378 |
-
<td align="center" style="vertical-align: middle; text-align: center">29.6</td>
|
| 379 |
<td align="center" style="vertical-align: middle; text-align: center">27.0</td>
|
| 380 |
<td align="center" style="vertical-align: middle; text-align: center">26.6</td>
|
| 381 |
<td align="center" style="vertical-align: middle; text-align: center">20.4</td>
|
|
@@ -383,7 +383,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 383 |
<tr>
|
| 384 |
<td align="center" style="vertical-align: middle; text-align: center">APEX-Agents</td>
|
| 385 |
<td align="center" style="vertical-align: middle; text-align: center">41.0</td>
|
| 386 |
-
<td align="center" style="vertical-align: middle; text-align: center">43.3</td>
|
| 387 |
<td align="center" style="vertical-align: middle; text-align: center">39.9</td>
|
| 388 |
<td align="center" style="vertical-align: middle; text-align: center">39.4</td>
|
| 389 |
<td align="center" style="vertical-align: middle; text-align: center">38.5</td>
|
|
@@ -392,7 +392,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 392 |
<tr>
|
| 393 |
<td align="center" style="vertical-align: middle; text-align: center">OfficeQA Pro</td>
|
| 394 |
<td align="center" style="vertical-align: middle; text-align: center">63.3</td>
|
| 395 |
-
<td align="center" style="vertical-align: middle; text-align: center">69.9</td>
|
| 396 |
<td align="center" style="vertical-align: middle; text-align: center">63.2</td>
|
| 397 |
<td align="center" style="vertical-align: middle; text-align: center">63.9</td>
|
| 398 |
<td align="center" style="vertical-align: middle; text-align: center">60.9</td>
|
|
@@ -400,7 +400,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 400 |
</tr>
|
| 401 |
<tr>
|
| 402 |
<td align="center" style="vertical-align: middle; text-align: center">SpreadsheetBench 2</td>
|
| 403 |
-
<td align="center" style="vertical-align: middle; text-align: center">34.8</td>
|
| 404 |
<td align="center" style="vertical-align: middle; text-align: center">34.7</td>
|
| 405 |
<td align="center" style="vertical-align: middle; text-align: center">32.4</td>
|
| 406 |
<td align="center" style="vertical-align: middle; text-align: center">31.6</td>
|
|
@@ -410,7 +410,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 410 |
<tr>
|
| 411 |
<td align="center" style="vertical-align: middle; text-align: center">OSWorld-Verified</td>
|
| 412 |
<td align="center" style="vertical-align: middle; text-align: center">84.8</td>
|
| 413 |
-
<td align="center" style="vertical-align: middle; text-align: center">85.0</td>
|
| 414 |
<td align="center" style="vertical-align: middle; text-align: center">83.0</td>
|
| 415 |
<td align="center" style="vertical-align: middle; text-align: center">83.4</td>
|
| 416 |
<td align="center" style="vertical-align: middle; text-align: center">79.0</td>
|
|
@@ -419,7 +419,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 419 |
<tr>
|
| 420 |
<td align="center" style="vertical-align: middle; text-align: center">OSWorld 2.0</td>
|
| 421 |
<td align="center" style="vertical-align: middle; text-align: center">58.3</td>
|
| 422 |
-
<td align="center" style="vertical-align: middle; text-align: center">66.1</td>
|
| 423 |
<td align="center" style="vertical-align: middle; text-align: center">62.6</td>
|
| 424 |
<td align="center" style="vertical-align: middle; text-align: center">55.7</td>
|
| 425 |
<td align="center" style="vertical-align: middle; text-align: center">49.5</td>
|
|
@@ -429,14 +429,14 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 429 |
<td align="center" style="vertical-align: middle; text-align: center">SaaS-Bench</td>
|
| 430 |
<td align="center" style="vertical-align: middle; text-align: center">60.1</td>
|
| 431 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 432 |
-
<td align="center" style="vertical-align: middle; text-align: center">61.4</td>
|
| 433 |
<td align="center" style="vertical-align: middle; text-align: center">56.1</td>
|
| 434 |
<td align="center" style="vertical-align: middle; text-align: center">43.8</td>
|
| 435 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 436 |
</tr>
|
| 437 |
<tr>
|
| 438 |
<td align="center" style="vertical-align: middle; text-align: center">τ³-Banking</td>
|
| 439 |
-
<td align="center" style="vertical-align: middle; text-align: center">33.4</td>
|
| 440 |
<td align="center" style="vertical-align: middle; text-align: center">26.8</td>
|
| 441 |
<td align="center" style="vertical-align: middle; text-align: center">33.0</td>
|
| 442 |
<td align="center" style="vertical-align: middle; text-align: center">27.6</td>
|
|
@@ -445,7 +445,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 445 |
</tr>
|
| 446 |
<tr>
|
| 447 |
<td align="center" style="vertical-align: middle; text-align: center">Harvey Lab-AA</td>
|
| 448 |
-
<td align="center" style="vertical-align: middle; text-align: center">94.6</td>
|
| 449 |
<td align="center" style="vertical-align: middle; text-align: center">93.6</td>
|
| 450 |
<td align="center" style="vertical-align: middle; text-align: center">87.2</td>
|
| 451 |
<td align="center" style="vertical-align: middle; text-align: center">91.1</td>
|
|
@@ -455,7 +455,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 455 |
<tr>
|
| 456 |
<td align="center" style="vertical-align: middle; text-align: center">CorpFin v2</td>
|
| 457 |
<td align="center" style="vertical-align: middle; text-align: center">71.6</td>
|
| 458 |
-
<td align="center" style="vertical-align: middle; text-align: center">71.8</td>
|
| 459 |
<td align="center" style="vertical-align: middle; text-align: center">64.4</td>
|
| 460 |
<td align="center" style="vertical-align: middle; text-align: center">66.7</td>
|
| 461 |
<td align="center" style="vertical-align: middle; text-align: center">68.4</td>
|
|
@@ -464,7 +464,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 464 |
<tr>
|
| 465 |
<td align="center" style="vertical-align: middle; text-align: center">Finance Agent v2</td>
|
| 466 |
<td align="center" style="vertical-align: middle; text-align: center">54.4</td>
|
| 467 |
-
<td align="center" style="vertical-align: middle; text-align: center">56.3</td>
|
| 468 |
<td align="center" style="vertical-align: middle; text-align: center">53.8</td>
|
| 469 |
<td align="center" style="vertical-align: middle; text-align: center">53.9</td>
|
| 470 |
<td align="center" style="vertical-align: middle; text-align: center">51.8</td>
|
|
@@ -473,7 +473,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 473 |
<tr>
|
| 474 |
<td align="center" style="vertical-align: middle; text-align: center">Legal Research Bench</td>
|
| 475 |
<td align="center" style="vertical-align: middle; text-align: center">44.2</td>
|
| 476 |
-
<td align="center" style="vertical-align: middle; text-align: center">49.5</td>
|
| 477 |
<td align="center" style="vertical-align: middle; text-align: center">48.1</td>
|
| 478 |
<td align="center" style="vertical-align: middle; text-align: center">43.8</td>
|
| 479 |
<td align="center" style="vertical-align: middle; text-align: center">40.4</td>
|
|
@@ -485,7 +485,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 485 |
<tr>
|
| 486 |
<td align="center" style="vertical-align: middle; text-align: center">WorldVQA ForceAnswer</td>
|
| 487 |
<td align="center" style="vertical-align: middle; text-align: center">51.0</td>
|
| 488 |
-
<td align="center" style="vertical-align: middle; text-align: center">56.7</td>
|
| 489 |
<td align="center" style="vertical-align: middle; text-align: center">41.8</td>
|
| 490 |
<td align="center" style="vertical-align: middle; text-align: center">39.1</td>
|
| 491 |
<td align="center" style="vertical-align: middle; text-align: center">38.5</td>
|
|
@@ -493,7 +493,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 493 |
</tr>
|
| 494 |
<tr>
|
| 495 |
<td align="center" style="vertical-align: middle; text-align: center">OmniDocBench</td>
|
| 496 |
-
<td align="center" style="vertical-align: middle; text-align: center">91.1</td>
|
| 497 |
<td align="center" style="vertical-align: middle; text-align: center">89.8</td>
|
| 498 |
<td align="center" style="vertical-align: middle; text-align: center">85.8</td>
|
| 499 |
<td align="center" style="vertical-align: middle; text-align: center">87.9</td>
|
|
@@ -504,14 +504,14 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 504 |
<td align="center" style="vertical-align: middle; text-align: center">PerceptionBench</td>
|
| 505 |
<td align="center" style="vertical-align: middle; text-align: center">58.5</td>
|
| 506 |
<td align="center" style="vertical-align: middle; text-align: center">57.2</td>
|
| 507 |
-
<td align="center" style="vertical-align: middle; text-align: center">59.7</td>
|
| 508 |
<td align="center" style="vertical-align: middle; text-align: center">47.2</td>
|
| 509 |
<td align="center" style="vertical-align: middle; text-align: center">55.8</td>
|
| 510 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 511 |
</tr>
|
| 512 |
<tr>
|
| 513 |
<td align="center" style="vertical-align: middle; text-align: center">Video-MME (w. sub)</td>
|
| 514 |
-
<td align="center" style="vertical-align: middle; text-align: center">90.0</td>
|
| 515 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 516 |
<td align="center" style="vertical-align: middle; text-align: center">89.5</td>
|
| 517 |
<td align="center" style="vertical-align: middle; text-align: center">86.0</td>
|
|
@@ -520,7 +520,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 520 |
</tr>
|
| 521 |
<tr>
|
| 522 |
<td align="center" style="vertical-align: middle; text-align: center">MMVU</td>
|
| 523 |
-
<td align="center" style="vertical-align: middle; text-align: center">82.1</td>
|
| 524 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 525 |
<td align="center" style="vertical-align: middle; text-align: center">81.2</td>
|
| 526 |
<td align="center" style="vertical-align: middle; text-align: center">79.2</td>
|
|
@@ -530,7 +530,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 530 |
<tr>
|
| 531 |
<td align="center" style="vertical-align: middle; text-align: center">BabyVision w/ python</td>
|
| 532 |
<td align="center" style="vertical-align: middle; text-align: center">85.7</td>
|
| 533 |
-
<td align="center" style="vertical-align: middle; text-align: center">90.5</td>
|
| 534 |
<td align="center" style="vertical-align: middle; text-align: center">88.9</td>
|
| 535 |
<td align="center" style="vertical-align: middle; text-align: center">81.2</td>
|
| 536 |
<td align="center" style="vertical-align: middle; text-align: center">83.6</td>
|
|
@@ -539,8 +539,8 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 539 |
<tr>
|
| 540 |
<td align="center" style="vertical-align: middle; text-align: center">MMMU-Pro</td>
|
| 541 |
<td align="center" style="vertical-align: middle; text-align: center">81.6 / 83.4</td>
|
| 542 |
-
<td align="center" style="vertical-align: middle; text-align: center">81.2 / 86.5</td>
|
| 543 |
-
<td align="center" style="vertical-align: middle; text-align: center">83.0 / 84.6</td>
|
| 544 |
<td align="center" style="vertical-align: middle; text-align: center">78.9 / 82.7</td>
|
| 545 |
<td align="center" style="vertical-align: middle; text-align: center">81.2 / 83.2</td>
|
| 546 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
|
@@ -548,7 +548,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 548 |
<tr>
|
| 549 |
<td align="center" style="vertical-align: middle; text-align: center">CharXiv (RQ)</td>
|
| 550 |
<td align="center" style="vertical-align: middle; text-align: center">84.8 / 91.3</td>
|
| 551 |
-
<td align="center" style="vertical-align: middle; text-align: center">88.9 / 93.5</td>
|
| 552 |
<td align="center" style="vertical-align: middle; text-align: center">84.6 / 89.1</td>
|
| 553 |
<td align="center" style="vertical-align: middle; text-align: center">80.5 / 89.9</td>
|
| 554 |
<td align="center" style="vertical-align: middle; text-align: center">84.1 / 89.0</td>
|
|
@@ -557,8 +557,8 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 557 |
<tr>
|
| 558 |
<td align="center" style="vertical-align: middle; text-align: center">MathVision</td>
|
| 559 |
<td align="center" style="vertical-align: middle; text-align: center">94.3 / 97.8</td>
|
| 560 |
-
<td align="center" style="vertical-align: middle; text-align: center">94.8 / 98.6</td>
|
| 561 |
-
<td align="center" style="vertical-align: middle; text-align: center">95.8 / 97.8</td>
|
| 562 |
<td align="center" style="vertical-align: middle; text-align: center">86.7 / 97.1</td>
|
| 563 |
<td align="center" style="vertical-align: middle; text-align: center">92.2 / 96.8</td>
|
| 564 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
|
@@ -566,7 +566,7 @@ Kimi K3 is an open-weight, native multimodal agentic model and our most capable
|
|
| 566 |
<tr>
|
| 567 |
<td align="center" style="vertical-align: middle; text-align: center">ZeroBench (pass@5)</td>
|
| 568 |
<td align="center" style="vertical-align: middle; text-align: center">23.0 / 41.0</td>
|
| 569 |
-
<td align="center" style="vertical-align: middle; text-align: center">23.0 / 46.0</td>
|
| 570 |
<td align="center" style="vertical-align: middle; text-align: center">17.0 / 35.0</td>
|
| 571 |
<td align="center" style="vertical-align: middle; text-align: center">17.0 / 34.0</td>
|
| 572 |
<td align="center" style="vertical-align: middle; text-align: center">22.0 / 41.0</td>
|
|
@@ -677,4 +677,4 @@ Both the code repository and the model weights are released under the [Kimi K3 L
|
|
| 677 |
|
| 678 |
## 8. Contact Us
|
| 679 |
|
| 680 |
-
If you have any questions, please reach out at [support@moonshot.ai](mailto:support@moonshot.ai).
|
|
|
|
| 162 |
<td align="center" style="vertical-align: middle; text-align: center">GPQA Diamond</td>
|
| 163 |
<td align="center" style="vertical-align: middle; text-align: center">93.5</td>
|
| 164 |
<td align="center" style="vertical-align: middle; text-align: center">92.6</td>
|
| 165 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>94.1</strong></td>
|
| 166 |
<td align="center" style="vertical-align: middle; text-align: center">91.0</td>
|
| 167 |
<td align="center" style="vertical-align: middle; text-align: center">93.5</td>
|
| 168 |
<td align="center" style="vertical-align: middle; text-align: center">91.2</td>
|
|
|
|
| 171 |
<td align="center" style="vertical-align: middle; text-align: center">CritPt</td>
|
| 172 |
<td align="center" style="vertical-align: middle; text-align: center">23.4</td>
|
| 173 |
<td align="center" style="vertical-align: middle; text-align: center">28.6</td>
|
| 174 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>32.3</strong></td>
|
| 175 |
<td align="center" style="vertical-align: middle; text-align: center">20.9</td>
|
| 176 |
<td align="center" style="vertical-align: middle; text-align: center">27.1</td>
|
| 177 |
<td align="center" style="vertical-align: middle; text-align: center">20.9</td>
|
| 178 |
</tr>
|
| 179 |
<tr>
|
| 180 |
<td align="center" style="vertical-align: middle; text-align: center">AA-LCR</td>
|
| 181 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>74.7</strong></td>
|
| 182 |
<td align="center" style="vertical-align: middle; text-align: center">70.0</td>
|
| 183 |
<td align="center" style="vertical-align: middle; text-align: center">73.7</td>
|
| 184 |
<td align="center" style="vertical-align: middle; text-align: center">67.7</td>
|
|
|
|
| 188 |
<tr>
|
| 189 |
<td align="center" style="vertical-align: middle; text-align: center">HLE-Full</td>
|
| 190 |
<td align="center" style="vertical-align: middle; text-align: center">43.5 / 56.0</td>
|
| 191 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>53.3 / 63.0</strong></td>
|
| 192 |
<td align="center" style="vertical-align: middle; text-align: center">44.5 / 58.0</td>
|
| 193 |
<td align="center" style="vertical-align: middle; text-align: center">49.8 / 57.9</td>
|
| 194 |
<td align="center" style="vertical-align: middle; text-align: center">41.4 / 52.2</td>
|
|
|
|
| 201 |
<td align="center" style="vertical-align: middle; text-align: center">DeepSWE</td>
|
| 202 |
<td align="center" style="vertical-align: middle; text-align: center">67.5</td>
|
| 203 |
<td align="center" style="vertical-align: middle; text-align: center">70.0</td>
|
| 204 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>73.0</strong></td>
|
| 205 |
<td align="center" style="vertical-align: middle; text-align: center">59.0</td>
|
| 206 |
<td align="center" style="vertical-align: middle; text-align: center">67.0</td>
|
| 207 |
<td align="center" style="vertical-align: middle; text-align: center">46.2</td>
|
| 208 |
</tr>
|
| 209 |
<tr>
|
| 210 |
<td align="center" style="vertical-align: middle; text-align: center">ProgramBench</td>
|
| 211 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>77.8</strong></td>
|
| 212 |
<td align="center" style="vertical-align: middle; text-align: center">76.8</td>
|
| 213 |
<td align="center" style="vertical-align: middle; text-align: center">77.6</td>
|
| 214 |
<td align="center" style="vertical-align: middle; text-align: center">71.9</td>
|
|
|
|
| 219 |
<td align="center" style="vertical-align: middle; text-align: center">Terminal-Bench 2.1</td>
|
| 220 |
<td align="center" style="vertical-align: middle; text-align: center">88.3</td>
|
| 221 |
<td align="center" style="vertical-align: middle; text-align: center">88.0</td>
|
| 222 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>88.8</strong></td>
|
| 223 |
<td align="center" style="vertical-align: middle; text-align: center">84.6</td>
|
| 224 |
<td align="center" style="vertical-align: middle; text-align: center">83.4</td>
|
| 225 |
<td align="center" style="vertical-align: middle; text-align: center">82.7</td>
|
|
|
|
| 227 |
<tr>
|
| 228 |
<td align="center" style="vertical-align: middle; text-align: center">FrontierSWE</td>
|
| 229 |
<td align="center" style="vertical-align: middle; text-align: center">81.2</td>
|
| 230 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>86.6</strong></td>
|
| 231 |
<td align="center" style="vertical-align: middle; text-align: center">71.3</td>
|
| 232 |
<td align="center" style="vertical-align: middle; text-align: center">66.7</td>
|
| 233 |
<td align="center" style="vertical-align: middle; text-align: center">64.9</td>
|
|
|
|
| 235 |
</tr>
|
| 236 |
<tr>
|
| 237 |
<td align="center" style="vertical-align: middle; text-align: center">SWE-Marathon</td>
|
| 238 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>42.0</strong></td>
|
| 239 |
<td align="center" style="vertical-align: middle; text-align: center">35.0</td>
|
| 240 |
<td align="center" style="vertical-align: middle; text-align: center">39.0</td>
|
| 241 |
<td align="center" style="vertical-align: middle; text-align: center">40.0</td>
|
|
|
|
| 245 |
<tr>
|
| 246 |
<td align="center" style="vertical-align: middle; text-align: center">PostTrainBench</td>
|
| 247 |
<td align="center" style="vertical-align: middle; text-align: center">36.6</td>
|
| 248 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>41.4</strong></td>
|
| 249 |
<td align="center" style="vertical-align: middle; text-align: center">34.6</td>
|
| 250 |
<td align="center" style="vertical-align: middle; text-align: center">34.1</td>
|
| 251 |
<td align="center" style="vertical-align: middle; text-align: center">28.4</td>
|
|
|
|
| 254 |
<tr>
|
| 255 |
<td align="center" style="vertical-align: middle; text-align: center">MLS-Bench-Lite</td>
|
| 256 |
<td align="center" style="vertical-align: middle; text-align: center">48.3</td>
|
| 257 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>49.9</strong></td>
|
| 258 |
<td align="center" style="vertical-align: middle; text-align: center">46.2</td>
|
| 259 |
<td align="center" style="vertical-align: middle; text-align: center">42.8</td>
|
| 260 |
<td align="center" style="vertical-align: middle; text-align: center">35.5</td>
|
|
|
|
| 263 |
<tr>
|
| 264 |
<td align="center" style="vertical-align: middle; text-align: center">SciCode</td>
|
| 265 |
<td align="center" style="vertical-align: middle; text-align: center">58.7</td>
|
| 266 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>60.2</strong></td>
|
| 267 |
<td align="center" style="vertical-align: middle; text-align: center">56.1</td>
|
| 268 |
<td align="center" style="vertical-align: middle; text-align: center">53.5</td>
|
| 269 |
<td align="center" style="vertical-align: middle; text-align: center">56.1</td>
|
|
|
|
| 272 |
<tr>
|
| 273 |
<td align="center" style="vertical-align: middle; text-align: center">Kimi Code Bench 2.0</td>
|
| 274 |
<td align="center" style="vertical-align: middle; text-align: center">72.9</td>
|
| 275 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>76.9</strong></td>
|
| 276 |
<td align="center" style="vertical-align: middle; text-align: center">64.8</td>
|
| 277 |
<td align="center" style="vertical-align: middle; text-align: center">71.7</td>
|
| 278 |
<td align="center" style="vertical-align: middle; text-align: center">69.0</td>
|
|
|
|
| 283 |
</tr>
|
| 284 |
<tr>
|
| 285 |
<td align="center" style="vertical-align: middle; text-align: center">BrowseComp</td>
|
| 286 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>91.2</strong></td>
|
| 287 |
<td align="center" style="vertical-align: middle; text-align: center">88.0</td>
|
| 288 |
<td align="center" style="vertical-align: middle; text-align: center">90.4</td>
|
| 289 |
<td align="center" style="vertical-align: middle; text-align: center">84.3</td>
|
|
|
|
| 292 |
</tr>
|
| 293 |
<tr>
|
| 294 |
<td align="center" style="vertical-align: middle; text-align: center">DeepSearchQA (F1)</td>
|
| 295 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>95.0</strong></td>
|
| 296 |
<td align="center" style="vertical-align: middle; text-align: center">94.2</td>
|
| 297 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 298 |
<td align="center" style="vertical-align: middle; text-align: center">93.1</td>
|
|
|
|
| 301 |
</tr>
|
| 302 |
<tr>
|
| 303 |
<td align="center" style="vertical-align: middle; text-align: center">ResearchRubrics</td>
|
| 304 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>76.2</strong></td>
|
| 305 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 306 |
<td align="center" style="vertical-align: middle; text-align: center">73.8</td>
|
| 307 |
<td align="center" style="vertical-align: middle; text-align: center">73.5</td>
|
|
|
|
| 311 |
<tr>
|
| 312 |
<td align="center" style="vertical-align: middle; text-align: center">GDPval-AA v2 (Elo)</td>
|
| 313 |
<td align="center" style="vertical-align: middle; text-align: center">1686</td>
|
| 314 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>1747</strong></td>
|
| 315 |
<td align="center" style="vertical-align: middle; text-align: center">1736</td>
|
| 316 |
<td align="center" style="vertical-align: middle; text-align: center">1593</td>
|
| 317 |
<td align="center" style="vertical-align: middle; text-align: center">1491</td>
|
|
|
|
| 320 |
<tr>
|
| 321 |
<td align="center" style="vertical-align: middle; text-align: center">Toolathlon-Verified</td>
|
| 322 |
<td align="center" style="vertical-align: middle; text-align: center">76.5</td>
|
| 323 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>77.9</strong></td>
|
| 324 |
<td align="center" style="vertical-align: middle; text-align: center">74.9</td>
|
| 325 |
<td align="center" style="vertical-align: middle; text-align: center">76.2</td>
|
| 326 |
<td align="center" style="vertical-align: middle; text-align: center">73.5</td>
|
|
|
|
| 328 |
</tr>
|
| 329 |
<tr>
|
| 330 |
<td align="center" style="vertical-align: middle; text-align: center">MCPMark-Verified</td>
|
| 331 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>94.5</strong></td>
|
| 332 |
<td align="center" style="vertical-align: middle; text-align: center">87.4</td>
|
| 333 |
<td align="center" style="vertical-align: middle; text-align: center">92.9</td>
|
| 334 |
<td align="center" style="vertical-align: middle; text-align: center">76.4</td>
|
|
|
|
| 338 |
<tr>
|
| 339 |
<td align="center" style="vertical-align: middle; text-align: center">MCP-Atlas</td>
|
| 340 |
<td align="center" style="vertical-align: middle; text-align: center">84.2</td>
|
| 341 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>84.7</strong></td>
|
| 342 |
<td align="center" style="vertical-align: middle; text-align: center">83.6</td>
|
| 343 |
<td align="center" style="vertical-align: middle; text-align: center">83.6</td>
|
| 344 |
<td align="center" style="vertical-align: middle; text-align: center">82.8</td>
|
|
|
|
| 346 |
</tr>
|
| 347 |
<tr>
|
| 348 |
<td align="center" style="vertical-align: middle; text-align: center">AutomationBench</td>
|
| 349 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>30.8</strong></td>
|
| 350 |
<td align="center" style="vertical-align: middle; text-align: center">29.1</td>
|
| 351 |
<td align="center" style="vertical-align: middle; text-align: center">29.7</td>
|
| 352 |
<td align="center" style="vertical-align: middle; text-align: center">27.2</td>
|
|
|
|
| 356 |
<tr>
|
| 357 |
<td align="center" style="vertical-align: middle; text-align: center">JobBench</td>
|
| 358 |
<td align="center" style="vertical-align: middle; text-align: center">54.3</td>
|
| 359 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>57.4</strong></td>
|
| 360 |
<td align="center" style="vertical-align: middle; text-align: center">45.4</td>
|
| 361 |
<td align="center" style="vertical-align: middle; text-align: center">48.4</td>
|
| 362 |
<td align="center" style="vertical-align: middle; text-align: center">38.3</td>
|
|
|
|
| 365 |
<tr>
|
| 366 |
<td align="center" style="vertical-align: middle; text-align: center">AA-Briefcase (Elo)</td>
|
| 367 |
<td align="center" style="vertical-align: middle; text-align: center">1548</td>
|
| 368 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>1583</strong></td>
|
| 369 |
<td align="center" style="vertical-align: middle; text-align: center">1495</td>
|
| 370 |
<td align="center" style="vertical-align: middle; text-align: center">1354</td>
|
| 371 |
<td align="center" style="vertical-align: middle; text-align: center">1158</td>
|
|
|
|
| 375 |
<td align="center" style="vertical-align: middle; text-align: center">Agents' Last Exam</td>
|
| 376 |
<td align="center" style="vertical-align: middle; text-align: center">28.3</td>
|
| 377 |
<td align="center" style="vertical-align: middle; text-align: center">25.7<sup>†</sup></td>
|
| 378 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>29.6</strong></td>
|
| 379 |
<td align="center" style="vertical-align: middle; text-align: center">27.0</td>
|
| 380 |
<td align="center" style="vertical-align: middle; text-align: center">26.6</td>
|
| 381 |
<td align="center" style="vertical-align: middle; text-align: center">20.4</td>
|
|
|
|
| 383 |
<tr>
|
| 384 |
<td align="center" style="vertical-align: middle; text-align: center">APEX-Agents</td>
|
| 385 |
<td align="center" style="vertical-align: middle; text-align: center">41.0</td>
|
| 386 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>43.3</strong></td>
|
| 387 |
<td align="center" style="vertical-align: middle; text-align: center">39.9</td>
|
| 388 |
<td align="center" style="vertical-align: middle; text-align: center">39.4</td>
|
| 389 |
<td align="center" style="vertical-align: middle; text-align: center">38.5</td>
|
|
|
|
| 392 |
<tr>
|
| 393 |
<td align="center" style="vertical-align: middle; text-align: center">OfficeQA Pro</td>
|
| 394 |
<td align="center" style="vertical-align: middle; text-align: center">63.3</td>
|
| 395 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>69.9</strong></td>
|
| 396 |
<td align="center" style="vertical-align: middle; text-align: center">63.2</td>
|
| 397 |
<td align="center" style="vertical-align: middle; text-align: center">63.9</td>
|
| 398 |
<td align="center" style="vertical-align: middle; text-align: center">60.9</td>
|
|
|
|
| 400 |
</tr>
|
| 401 |
<tr>
|
| 402 |
<td align="center" style="vertical-align: middle; text-align: center">SpreadsheetBench 2</td>
|
| 403 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>34.8</strong></td>
|
| 404 |
<td align="center" style="vertical-align: middle; text-align: center">34.7</td>
|
| 405 |
<td align="center" style="vertical-align: middle; text-align: center">32.4</td>
|
| 406 |
<td align="center" style="vertical-align: middle; text-align: center">31.6</td>
|
|
|
|
| 410 |
<tr>
|
| 411 |
<td align="center" style="vertical-align: middle; text-align: center">OSWorld-Verified</td>
|
| 412 |
<td align="center" style="vertical-align: middle; text-align: center">84.8</td>
|
| 413 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>85.0</strong></td>
|
| 414 |
<td align="center" style="vertical-align: middle; text-align: center">83.0</td>
|
| 415 |
<td align="center" style="vertical-align: middle; text-align: center">83.4</td>
|
| 416 |
<td align="center" style="vertical-align: middle; text-align: center">79.0</td>
|
|
|
|
| 419 |
<tr>
|
| 420 |
<td align="center" style="vertical-align: middle; text-align: center">OSWorld 2.0</td>
|
| 421 |
<td align="center" style="vertical-align: middle; text-align: center">58.3</td>
|
| 422 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>66.1</strong></td>
|
| 423 |
<td align="center" style="vertical-align: middle; text-align: center">62.6</td>
|
| 424 |
<td align="center" style="vertical-align: middle; text-align: center">55.7</td>
|
| 425 |
<td align="center" style="vertical-align: middle; text-align: center">49.5</td>
|
|
|
|
| 429 |
<td align="center" style="vertical-align: middle; text-align: center">SaaS-Bench</td>
|
| 430 |
<td align="center" style="vertical-align: middle; text-align: center">60.1</td>
|
| 431 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 432 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>61.4</strong></td>
|
| 433 |
<td align="center" style="vertical-align: middle; text-align: center">56.1</td>
|
| 434 |
<td align="center" style="vertical-align: middle; text-align: center">43.8</td>
|
| 435 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 436 |
</tr>
|
| 437 |
<tr>
|
| 438 |
<td align="center" style="vertical-align: middle; text-align: center">τ³-Banking</td>
|
| 439 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>33.4</strong></td>
|
| 440 |
<td align="center" style="vertical-align: middle; text-align: center">26.8</td>
|
| 441 |
<td align="center" style="vertical-align: middle; text-align: center">33.0</td>
|
| 442 |
<td align="center" style="vertical-align: middle; text-align: center">27.6</td>
|
|
|
|
| 445 |
</tr>
|
| 446 |
<tr>
|
| 447 |
<td align="center" style="vertical-align: middle; text-align: center">Harvey Lab-AA</td>
|
| 448 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>94.6</strong></td>
|
| 449 |
<td align="center" style="vertical-align: middle; text-align: center">93.6</td>
|
| 450 |
<td align="center" style="vertical-align: middle; text-align: center">87.2</td>
|
| 451 |
<td align="center" style="vertical-align: middle; text-align: center">91.1</td>
|
|
|
|
| 455 |
<tr>
|
| 456 |
<td align="center" style="vertical-align: middle; text-align: center">CorpFin v2</td>
|
| 457 |
<td align="center" style="vertical-align: middle; text-align: center">71.6</td>
|
| 458 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>71.8</strong></td>
|
| 459 |
<td align="center" style="vertical-align: middle; text-align: center">64.4</td>
|
| 460 |
<td align="center" style="vertical-align: middle; text-align: center">66.7</td>
|
| 461 |
<td align="center" style="vertical-align: middle; text-align: center">68.4</td>
|
|
|
|
| 464 |
<tr>
|
| 465 |
<td align="center" style="vertical-align: middle; text-align: center">Finance Agent v2</td>
|
| 466 |
<td align="center" style="vertical-align: middle; text-align: center">54.4</td>
|
| 467 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>56.3</strong></td>
|
| 468 |
<td align="center" style="vertical-align: middle; text-align: center">53.8</td>
|
| 469 |
<td align="center" style="vertical-align: middle; text-align: center">53.9</td>
|
| 470 |
<td align="center" style="vertical-align: middle; text-align: center">51.8</td>
|
|
|
|
| 473 |
<tr>
|
| 474 |
<td align="center" style="vertical-align: middle; text-align: center">Legal Research Bench</td>
|
| 475 |
<td align="center" style="vertical-align: middle; text-align: center">44.2</td>
|
| 476 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>49.5</strong></td>
|
| 477 |
<td align="center" style="vertical-align: middle; text-align: center">48.1</td>
|
| 478 |
<td align="center" style="vertical-align: middle; text-align: center">43.8</td>
|
| 479 |
<td align="center" style="vertical-align: middle; text-align: center">40.4</td>
|
|
|
|
| 485 |
<tr>
|
| 486 |
<td align="center" style="vertical-align: middle; text-align: center">WorldVQA ForceAnswer</td>
|
| 487 |
<td align="center" style="vertical-align: middle; text-align: center">51.0</td>
|
| 488 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>56.7</strong></td>
|
| 489 |
<td align="center" style="vertical-align: middle; text-align: center">41.8</td>
|
| 490 |
<td align="center" style="vertical-align: middle; text-align: center">39.1</td>
|
| 491 |
<td align="center" style="vertical-align: middle; text-align: center">38.5</td>
|
|
|
|
| 493 |
</tr>
|
| 494 |
<tr>
|
| 495 |
<td align="center" style="vertical-align: middle; text-align: center">OmniDocBench</td>
|
| 496 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>91.1</strong></td>
|
| 497 |
<td align="center" style="vertical-align: middle; text-align: center">89.8</td>
|
| 498 |
<td align="center" style="vertical-align: middle; text-align: center">85.8</td>
|
| 499 |
<td align="center" style="vertical-align: middle; text-align: center">87.9</td>
|
|
|
|
| 504 |
<td align="center" style="vertical-align: middle; text-align: center">PerceptionBench</td>
|
| 505 |
<td align="center" style="vertical-align: middle; text-align: center">58.5</td>
|
| 506 |
<td align="center" style="vertical-align: middle; text-align: center">57.2</td>
|
| 507 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>59.7</strong></td>
|
| 508 |
<td align="center" style="vertical-align: middle; text-align: center">47.2</td>
|
| 509 |
<td align="center" style="vertical-align: middle; text-align: center">55.8</td>
|
| 510 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 511 |
</tr>
|
| 512 |
<tr>
|
| 513 |
<td align="center" style="vertical-align: middle; text-align: center">Video-MME (w. sub)</td>
|
| 514 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>90.0</strong></td>
|
| 515 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 516 |
<td align="center" style="vertical-align: middle; text-align: center">89.5</td>
|
| 517 |
<td align="center" style="vertical-align: middle; text-align: center">86.0</td>
|
|
|
|
| 520 |
</tr>
|
| 521 |
<tr>
|
| 522 |
<td align="center" style="vertical-align: middle; text-align: center">MMVU</td>
|
| 523 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>82.1</strong></td>
|
| 524 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
| 525 |
<td align="center" style="vertical-align: middle; text-align: center">81.2</td>
|
| 526 |
<td align="center" style="vertical-align: middle; text-align: center">79.2</td>
|
|
|
|
| 530 |
<tr>
|
| 531 |
<td align="center" style="vertical-align: middle; text-align: center">BabyVision w/ python</td>
|
| 532 |
<td align="center" style="vertical-align: middle; text-align: center">85.7</td>
|
| 533 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>90.5</strong></td>
|
| 534 |
<td align="center" style="vertical-align: middle; text-align: center">88.9</td>
|
| 535 |
<td align="center" style="vertical-align: middle; text-align: center">81.2</td>
|
| 536 |
<td align="center" style="vertical-align: middle; text-align: center">83.6</td>
|
|
|
|
| 539 |
<tr>
|
| 540 |
<td align="center" style="vertical-align: middle; text-align: center">MMMU-Pro</td>
|
| 541 |
<td align="center" style="vertical-align: middle; text-align: center">81.6 / 83.4</td>
|
| 542 |
+
<td align="center" style="vertical-align: middle; text-align: center">81.2 / <strong>86.5</strong></td>
|
| 543 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>83.0</strong> / 84.6</td>
|
| 544 |
<td align="center" style="vertical-align: middle; text-align: center">78.9 / 82.7</td>
|
| 545 |
<td align="center" style="vertical-align: middle; text-align: center">81.2 / 83.2</td>
|
| 546 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
|
|
|
| 548 |
<tr>
|
| 549 |
<td align="center" style="vertical-align: middle; text-align: center">CharXiv (RQ)</td>
|
| 550 |
<td align="center" style="vertical-align: middle; text-align: center">84.8 / 91.3</td>
|
| 551 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>88.9 / 93.5</strong></td>
|
| 552 |
<td align="center" style="vertical-align: middle; text-align: center">84.6 / 89.1</td>
|
| 553 |
<td align="center" style="vertical-align: middle; text-align: center">80.5 / 89.9</td>
|
| 554 |
<td align="center" style="vertical-align: middle; text-align: center">84.1 / 89.0</td>
|
|
|
|
| 557 |
<tr>
|
| 558 |
<td align="center" style="vertical-align: middle; text-align: center">MathVision</td>
|
| 559 |
<td align="center" style="vertical-align: middle; text-align: center">94.3 / 97.8</td>
|
| 560 |
+
<td align="center" style="vertical-align: middle; text-align: center">94.8 / <strong>98.6</strong></td>
|
| 561 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>95.8</strong> / 97.8</td>
|
| 562 |
<td align="center" style="vertical-align: middle; text-align: center">86.7 / 97.1</td>
|
| 563 |
<td align="center" style="vertical-align: middle; text-align: center">92.2 / 96.8</td>
|
| 564 |
<td align="center" style="vertical-align: middle; text-align: center">—</td>
|
|
|
|
| 566 |
<tr>
|
| 567 |
<td align="center" style="vertical-align: middle; text-align: center">ZeroBench (pass@5)</td>
|
| 568 |
<td align="center" style="vertical-align: middle; text-align: center">23.0 / 41.0</td>
|
| 569 |
+
<td align="center" style="vertical-align: middle; text-align: center"><strong>23.0 / 46.0</strong></td>
|
| 570 |
<td align="center" style="vertical-align: middle; text-align: center">17.0 / 35.0</td>
|
| 571 |
<td align="center" style="vertical-align: middle; text-align: center">17.0 / 34.0</td>
|
| 572 |
<td align="center" style="vertical-align: middle; text-align: center">22.0 / 41.0</td>
|
|
|
|
| 677 |
|
| 678 |
## 8. Contact Us
|
| 679 |
|
| 680 |
+
If you have any questions, please reach out at [support@moonshot.ai](mailto:support@moonshot.ai).
|