Image-Text-to-Text
Transformers
Safetensors
inkling2_loop
text-generation
conversational
Mixture of Experts
8-bit precision
modelopt
Instructions to use thinkingmachines/Inkling-2-Mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingmachines/Inkling-2-Mini with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling-2-Mini") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("thinkingmachines/Inkling-2-Mini", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingmachines/Inkling-2-Mini with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingmachines/Inkling-2-Mini" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Mini", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thinkingmachines/Inkling-2-Mini
- SGLang
How to use thinkingmachines/Inkling-2-Mini with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Mini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Mini", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Mini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Mini", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thinkingmachines/Inkling-2-Mini with Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling-2-Mini
File size: 54,831 Bytes
1fb4134 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 | ---
license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
pipeline_tag: image-text-to-text
tags:
- conversational
- image-text-to-text
- moe
library_name: transformers
---
# Inkling 2 Mini
<img src="https://cdn-uploads.huggingface.co/production/uploads/630e8f0bf6f6d700f50ebd2e/AvmDwmrWRMnKjOWvmLieg.png" style="display: block;margin-left: auto;margin-right: auto;width: 30%;">
<p align="center">
<a href="https://tinker.thinkingmachines.ai/playground">Playground</a> |
<a href="https://github.com/thinking-machines-lab/tinker-cookbook">Tinker Cookbook</a>
</p>
## 1. General Information
Inkling 2 Mini is a general-purpose multimodal model that accepts text and image inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages. The model is designed to be used by developers building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems, and is suitable for general-purpose conversational use, instruction-following, and other natural language and visual tasks. It is released with open weights to support research, fine-tuning and integration into third-party products by downstream developers.
**Languages:** English, general multilingual support
**Model provider:** Thinking Machines Lab, Inc.
**Date of release:** November 20, 2026
**License:** Apache 2.0
[Model Acceptable Use Policy](https://thinkingmachines.ai/model-acceptable-use-policy/)
## 2. Getting Started
The weights are available for download through Hugging Face [here](https://huggingface.co/thinkingmachines/Inkling-2-Mini).
Developers can also access the model using the Tinker fine-tuning platform, which provides API access.
* For accessing our model via Tinker, you can get started by referring to the Tinker Cookbook [here](https://github.com/thinking-machines-lab/tinker-cookbook), and installing the tml-renderers package [here](https://pypi.org/project/tml-renderers/).
* Tinker playground also provides inference-only access to the model [here](https://tinker.thinkingmachines.ai/playground).
### Required hardware
The model is distributed as a single checkpoint of about 20GB, in mixed NVFP4 and MXFP8 precisions.
Given its small size, it is possible to run this checkpoint on local AI compute for inference. We support:
* 1x NVIDIA DGX Spark
* Apple silicon devices with 32GB or more memory (MacBooks, Mac minis and Studios)
* Other GPUs and platforms are possible, with 32GB or more of VRAM recommended
### Required software
Inkling 2 Mini supports local inference using the following frameworks, along with their dependency libraries:
* [Hugging Face](https://hf.co/blog/inkling2mini)
* [SGLang](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling-2-Mini)
* [vLLM](https://recipes.vllm.ai/thinkingmachines/Inkling-2-Mini)
* [TokenSpeed](https://lightseek.org/tokenspeed/recipes/models#inkling-2-mini)
* NVIDIA Dynamo
* [Unsloth](https://unsloth.ai/docs/inkling-2-mini), including quantized GGUF versions
* llama.cpp, for quantized GGUF versions
* [Llama App](https://llama.app/)
* Ollama and Exo Labs, on Apple silicon
Inkling 2 Mini can also be post-trained with:
* [Miles](https://miles.radixark.com/docs/models/thinkingmachines/inkling-2-mini)
* [SkyRL](https://docs.skyrl.ai/docs/models/inkling2)
## 3. Model Properties
### Model type
Multimodal autoregressive looped transformer
### Architecture type
A 24-layer decoder-only transformer with sparse Mixture-of-Experts (MoE) feed-forward layers. The middle 12 layers run three times with shared weights, giving 48 layer applications per token from 24 layers of weights. In each layer, each token is routed to 8 of 128 experts. Attention is a hybrid of local and global layers. The model is natively multimodal – images are encoded via a vision transformer – with image and text inputs projected into a shared hidden space and processed jointly by the decoder.
### Parameters
26.4B, 3.4B active
Note: each extra pass through the looped layers adds about 1.0B effective active parameters. Since the model runs with three passes, there are 5.4B effective active parameters per token.
### Numerics support
NVFP4 and MXFP8
### Input modalities
* Text: UTF-8 encoded text
* Image: Any pixel-based image input. For optimal performance, each image dimension should be between 40px to 4096px.
Inkling 2 Mini supports a context window of up to 1M tokens.
### Output modalities
Text
## 4. Training
Training data includes a broad variety of content types, including text, images, video. Training data for the model was drawn from publicly available sources, acquired from third-parties, generated internally, or synthetically generated or augmented. Publicly available data includes content from the public internet and publicly accessible repositories.
Data curation includes cleaning, processing, and modifying datasets. These processing steps, which vary by data type, may include deduplication and filtering to remove junk or other low-quality data, or to advance safety or other objectives.
## 5. Evaluations
<table>
<thead>
<tr><th style="border-right-color:rgba(128,128,128,0.45)"></th><th style="border-right-color:rgba(128,128,128,0.45)">Inkling 2 Mini</th><th style="font-weight:400">Gemma 4 31B</th><th style="font-weight:400">Muse Glimmer</th><th style="font-weight:400">Qwen3.8-27B</th><th style="font-weight:400">Nemotron 3.5 Super VL</th></tr>
</thead>
<tbody>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Model Info</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Params (B)<br><span style="font-size:11px;opacity:0.7">activated / total</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">5/26</td><td style="text-align:center;vertical-align:middle">31/31</td><td style="text-align:center;vertical-align:middle">30/30</td><td style="text-align:center;vertical-align:middle">27/27</td><td style="text-align:center;vertical-align:middle">12/125</td></tr>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Third Party Index</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA Index (v5.0)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">20.1</span></td><td style="text-align:center">4.4</td><td style="text-align:center">6.3</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">20.1</span></td><td style="text-align:center">13.6</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Vals Index (v2.2)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">46.5%</span></td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">29.8%</span></td></tr>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Coding and Autoresearch</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">FrontierCode v2.0<br><span style="font-size:11px;opacity:0.7">stirrup v0.2.1</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">15.1%</span></td><td style="text-align:center;vertical-align:middle">0.0%</td><td style="text-align:center;vertical-align:middle">1.5%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">14.9%</span></td><td style="text-align:center;vertical-align:middle">6.1%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">DeepSWE v1.1<br><span style="font-size:11px;opacity:0.7">mini-swe-agent v2.4.6</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">48.2%</span></td><td style="text-align:center;vertical-align:middle">0.3%</td><td style="text-align:center;vertical-align:middle">4.4%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">42.2%</span></td><td style="text-align:center;vertical-align:middle">24.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Terminal Bench v4.0<br><span style="font-size:11px;opacity:0.7">mini-swe-agent v2.4.6</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">5.6%</span></td><td style="text-align:center;vertical-align:middle">0.0%</td><td style="text-align:center;vertical-align:middle">0.5%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">5.6%</span></td><td style="text-align:center;vertical-align:middle">0.0%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Terminal Bench v2.1<br><span style="font-size:11px;opacity:0.7">Inkling 2 Mini in pi v0.83.0</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">71.6%</span></td><td style="text-align:center;vertical-align:middle">43.4%</td><td style="text-align:center;vertical-align:middle">51.7%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">73.0%</span></td><td style="text-align:center;vertical-align:middle">60.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">SWEBench Pro v2 (Hard)<br><span style="font-size:11px;opacity:0.7">mini-swe-agent v2.4.6</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">77.9%</span></td><td style="text-align:center;vertical-align:middle">14.2%</td><td style="text-align:center;vertical-align:middle">44.3%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">73.8%</span></td><td style="text-align:center;vertical-align:middle">44.1%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">SWEBench Multilingual<br><span style="font-size:11px;opacity:0.7">Inkling 2 Mini with bash only</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">73.7%</span></td><td style="text-align:center;vertical-align:middle">43.0%</td><td style="text-align:center;vertical-align:middle">66.9%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">70.0%</span></td><td style="text-align:center;vertical-align:middle">67.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">SWEBench Verified<br><span style="font-size:11px;opacity:0.7">bash only</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">79.2%</span></td><td style="text-align:center;vertical-align:middle">46.1%</td><td style="text-align:center;vertical-align:middle">76.0%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">77.8%</span></td><td style="text-align:center;vertical-align:middle">71.1%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Vibe Code Bench v1.1<br><span style="font-size:11px;opacity:0.7">openhands (vals)</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">77.9%</span></td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">64.9%</span></td><td style="text-align:center;vertical-align:middle">3.5%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">PMPP-Hard (AUC, 0–10h)<br><span style="font-size:11px;opacity:0.7">pi v0.83.0</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">455.82</span></td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">262.50</span></td><td style="text-align:center;vertical-align:middle">–</td></tr>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Agentic Search, Tool Use, and Knowledge Work</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">BrowseComp<br><span style="font-size:11px;opacity:0.7">Inkling 2 Mini with compaction</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">79.0%</span></td><td style="text-align:center;vertical-align:middle">37.0%</td><td style="text-align:center;vertical-align:middle">63.9%</td><td style="text-align:center;vertical-align:middle">61.6%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">78.9%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">DeepSearchQA<br><span style="font-size:11px;opacity:0.7">Inkling 2 Mini with compaction</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">85.3%</span></td><td style="text-align:center;vertical-align:middle">61.7%</td><td style="text-align:center;vertical-align:middle">74.6%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">79.7%</span></td><td style="text-align:center;vertical-align:middle">77.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">MCP Atlas<br><span style="font-size:11px;opacity:0.7">Public</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">82.7%</span></td><td style="text-align:center;vertical-align:middle">54.2%</td><td style="text-align:center;vertical-align:middle">75.5%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">77.3%</span></td><td style="text-align:center;vertical-align:middle">67.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA-Briefcase v1.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">1263</span></td><td style="text-align:center">369</td><td style="text-align:center">478</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">1391</span></td><td style="text-align:center">999</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">GDPVal-AA v2.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">1319</span></td><td style="text-align:center">627</td><td style="text-align:center">792</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">1427</span></td><td style="text-align:center">1162</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">JobBench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">49.6%</span></td><td style="text-align:center">8.2%</td><td style="text-align:center">16.1%</td><td style="text-align:center">33.4%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">36.5%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AutomationBench-AA v1.0.6</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">41.8%</span></td><td style="text-align:center">4.9%</td><td style="text-align:center">6.8%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">48.2%</span></td><td style="text-align:center">14.8%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">EnterpriseOps-Gym</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">42.0%</span></td><td style="text-align:center">28.4%</td><td style="text-align:center">34.7%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">44.2%</span></td><td style="text-align:center">28.1%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Vals Finance Index</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">50.1%</span></td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">41.0%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA-AnalystAgent</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">46.3%</span></td><td style="text-align:center">–</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">20.0%</span></td><td style="text-align:center">–</td><td style="text-align:center">–</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Diligence Bench<br><span style="font-size:11px;opacity:0.7">Best harness</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">48.6%</span></td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">40.9%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">46.4%</span></td><td style="text-align:center;vertical-align:middle">45.0%</td></tr>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Vision Interpretation and Creation</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">GDP.pdf</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">16.8%</span></td><td style="text-align:center">6.0%</td><td style="text-align:center">11.8%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">16.4%</span></td><td style="text-align:center">8.0%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">PerceptionBench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">57.5%</span></td><td style="text-align:center">40.7%</td><td style="text-align:center">42.8%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">52.4%</span></td><td style="text-align:center">36.8%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">MMMU Pro<br><span style="font-size:11px;opacity:0.7">Standard 10</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">74.9%</span></td><td style="text-align:center;vertical-align:middle">73.4%</td><td style="text-align:center;vertical-align:middle">74.3%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">76.3%</span></td><td style="text-align:center;vertical-align:middle">72.3%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">OmniDocBench v1.6</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">92.8%</span></td><td style="text-align:center">82.5%</td><td style="text-align:center">90.4%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">91.5%</span></td><td style="text-align:center">77.9%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">ScreenSpot Pro<br><span style="font-size:11px;opacity:0.7">original / with Python</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">76.5/87.7%</span></td><td style="text-align:center;vertical-align:middle">5.9%/–</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">75.4%/–</span></td><td style="text-align:center;vertical-align:middle">57.6/78.9%</td><td style="text-align:center;vertical-align:middle">43.5%/–</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Chartography<br><span style="font-size:11px;opacity:0.7">original / with Python</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">22.1/72.5%</span></td><td style="text-align:center;vertical-align:middle">12.0/11.8%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">17.4/25.1%</span></td><td style="text-align:center;vertical-align:middle">12.2/52.1%</td><td style="text-align:center;vertical-align:middle">9.7/29.3%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">ZeroBench (main)<br><span style="font-size:11px;opacity:0.7">original / with Python</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">17.0/34.0%</span></td><td style="text-align:center;vertical-align:middle">9.0/12.0%</td><td style="text-align:center;vertical-align:middle">9.0/17.0%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">12.0%/–</span></td><td style="text-align:center;vertical-align:middle">11.0/12.0%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">MuirBench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">77.5%</span></td><td style="text-align:center">72.3%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">77.6%</span></td><td style="text-align:center">65.9%</td><td style="text-align:center">72.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">BVB</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">68.4%</span></td><td style="text-align:center">47.0%</td><td style="text-align:center">50.5%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">70.7%</span></td><td style="text-align:center">52.1%</td></tr>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Reasoning, Knowledge, and Epistemics</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">HLE-Diamond</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">15.2%</span></td><td style="text-align:center">7.2%</td><td style="text-align:center">9.1%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">15.3%</span></td><td style="text-align:center">8.8%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">CritPt v2<br><span style="font-size:11px;opacity:0.7">Full / Hard</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">31.4/17.0%</span></td><td style="text-align:center;vertical-align:middle">–/4.0%</td><td style="text-align:center;vertical-align:middle">–/7.0%</td><td style="text-align:center;vertical-align:middle">10.6/4.0%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">36.9/13.5%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Apex 2025</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">34.4%</span></td><td style="text-align:center">5.2%</td><td style="text-align:center">4.2%</td><td style="text-align:center">18.8%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">22.9%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">ARC-AGI<br><span style="font-size:11px;opacity:0.7">v1 / v2</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">84.5/49.3%</span></td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">87.5/42.4%</span></td><td style="text-align:center;vertical-align:middle">22.7/1.5%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">LongBench v2</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">66.9%</span></td><td style="text-align:center">52.9%</td><td style="text-align:center">33.1%</td><td style="text-align:center">52.3%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">62.1%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">GraphWalks BFS<br><span style="font-size:11px;opacity:0.7">256K</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">69.7%</span></td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">50.4%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">63.3%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">MRCR v2<br><span style="font-size:11px;opacity:0.7">8-needle 256K–512K</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">46.4%</span></td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">38.0%</span></td><td style="text-align:center;vertical-align:middle">5.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">AA-Omniscience v1.1<br><span style="font-size:11px;opacity:0.7">score</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">-8.6</td><td style="text-align:center;vertical-align:middle">-48.5</td><td style="text-align:center;vertical-align:middle">-32.2</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">-6.4</span></td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">-2.9</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">AA-Omniscience v1.1<br><span style="font-size:11px;opacity:0.7">non-hallucination rate</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">63.4%</td><td style="text-align:center;vertical-align:middle">14.5%</td><td style="text-align:center;vertical-align:middle">16.4%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">71.4%</span></td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">69.5%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">ComplexConstraints v1.1<br><span style="font-size:11px;opacity:0.7">openhands v1.45.0</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">33.4%</span></td><td style="text-align:center;vertical-align:middle">31.6%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">35.6%</span></td><td style="text-align:center;vertical-align:middle">30.1%</td><td style="text-align:center;vertical-align:middle">30.1%</td></tr>
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Safety and Alignment</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">FORTRESS (adversarial)<br><span style="font-size:11px;opacity:0.7">Public</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">89.5%</span></td><td style="text-align:center;vertical-align:middle">67.8%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">90.3%</span></td><td style="text-align:center;vertical-align:middle">78.8%</td><td style="text-align:center;vertical-align:middle">79.6%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">FORTRESS (benign)<br><span style="font-size:11px;opacity:0.7">Public</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">96.4%</td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">97.6%</span></td><td style="text-align:center;vertical-align:middle">89.1%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">97.8%</span></td><td style="text-align:center;vertical-align:middle">92.7%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">StrongREJECT (none)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">97.8%</td><td style="text-align:center">98.1%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">99.0%</span></td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">99.0%</span></td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">99.4%</span></td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Misbehavior (Internal) ↓</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">9.8%</span></td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">27.6%</span></td><td style="text-align:center">46.4%</td><td style="text-align:center">57.7%</td><td style="text-align:center">52.8%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Covert misbehavior (Internal) ↓</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">8.3%</span></td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">24.3%</span></td><td style="text-align:center">45.6%</td><td style="text-align:center">33.2%</td><td style="text-align:center">43.4%</td></tr>
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Destructive actions (Internal) ↓</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">10.2%</span></td><td style="text-align:center">75.8%</td><td style="text-align:center">68.8%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">35.2%</span></td><td style="text-align:center">65.6%</td></tr>
</tbody>
</table>
<p align="center"><span style="white-space:nowrap"><span style="background-color:rgba(255,206,82,0.75);color:#282828;border-radius:3px;padding:0.1em 0.3em;font-weight:600">Best reported score</span> in each row (lowest where lower is better); equal displayed scores tie</span> <span style="white-space:nowrap"><span style="background-color:rgba(255,206,82,0.28);color:inherit;border-radius:3px;padding:0.1em 0.3em">Second-best reported score</span> in each row; none when first place is tied</span></p>
## 6. Safety
We conducted safety evaluations ahead of release, spanning both everyday human-AI interaction and dangerous-capability testing. Because Inkling 2 Mini is multimodal, we paid attention to whether safety behavior held consistently across text and image inputs. We applied mitigations to reduce risks before release, including an extra phase of alignment training using on-policy distillation, similarly to Inkling 2 Small.
For everyday use, we evaluated sycophancy and how the model handles requests involving suicide, self-harm and eating disorders, including whether it encourages harm or affirms false beliefs. We also assessed whether the model refuses genuinely harmful requests without over-refusing benign ones. For CBRN and cyber, we assessed knowledge and procedural uplift through internal evaluations and refusal-suppressed variants intended to estimate latent capability with safeguards removed. For loss of control, we evaluated agentic capability, strategic deception, and sabotage potential, benchmarked against public frontier models, and found the model materially below frontier capabilities.
Inkling 2 Mini is the first looped transformer we have released, so we also studied how looping affects monitorability: whether we can still oversee the model by reading its chain of thought. Looping lets the model do more computation without reasoning in visible tokens. Inkling 2 Mini has very little controllability over its chain of thought, and was unable to perform covert side tasks in realistic agentic environments. Further experiments on the effect of looping on monitorability are described in our [blog post](https://thinkingmachines.ai/news/inkling-2-mini/).
Across all areas, we concluded that Inkling 2 Mini does not give material uplift beyond what is already available from open-weights models.
## 7. Bias, risks and limitations
Inkling 2 Mini may exhibit general limitations common to foundation models, including hallucination (generating plausible but factually incorrect or unsupported content), occasional failures to follow instructions precisely, and degraded performance in long multi-turn conversations. As with other models trained on web-derived and synthetic data, it may reflect biases present in its training data, including demographic, cultural or linguistic biases, and may perform unevenly across languages, dialects or subject domains that were less represented during training.
Inkling 2 Mini’s knowledge is limited to information available as of its training cutoff, and it may not reflect events or changes that occurred afterward.
Inkling 2 Mini is a small model built to run on local hardware, and some limitations are specific to it:
* Knowledge and factual accuracy. It is less capable than Inkling 2 Small across most of our evaluations, with the largest gaps on factual recall and knowledge-heavy questions. It is more likely than larger models to state incorrect information with confidence.
* Repetitive reasoning. In internal testing the model sometimes became stuck in long reasoning loops, or repeatedly re-checked its own work without making progress, particularly when debugging code. We recommend setting limits on output tokens and agent turns.
* Agentic reliability. When using tools, it is less reliable than larger models at following detailed instructions and project conventions. It may take destructive actions, such as deleting or overwriting files, without confirmation, and may not report the outcome of its work accurately.
* Languages other than English. Output quality can be lower outside English. In testing in Chinese we observed clipped phrasing and mixed languages.
* Sensitive topics. The model can respond with more caution than the situation calls for when a conversation touches on mental health.
* Reasoning and compute. Setting reasoning effort to its minimum does not fully disable reasoning. Because the middle layers run three times, compute per token is higher than the active-parameter count alone suggests.
* Local performance. Support for looped models in local runtimes is new, and speed varies by runtime and hardware.
We recommend that downstream developers and deployers:
* Apply appropriate human oversight and review for outputs used in high-stakes or safety-critical contexts, rather than relying on Inkling 2 Mini’s outputs without verification.
* Run agentic workloads in a sandbox, and require confirmation before destructive or irreversible actions.
* Conduct their own evaluation of Inkling 2 Mini’s performance, safety and fairness for their specific use case, language and population prior to deployment, particularly for applications involving vulnerable groups.
* Implement additional safeguards, such as content filtering, rate limiting and monitoring, at the application layer, especially for open deployment contexts where Inkling 2 Mini’s built-in mitigations may not be sufficient on their own.
* Avoid deploying Inkling 2 Mini in domains such as medical, legal or safety-critical decision-making without additional fine-tuning, domain-specific validation and human oversight.
## 8. Legal
[Training Data Documentation](https://thinkingmachines.ai/training-data-documentation/)
|