AutoMedBench Lite
Leaderboard
Average model performance across all seven released Lite tracks.
7-track average
Model score comparison
Compare every model on the same 0–100 scale. Model-colored columns are generated directly from the leaderboard values below.
Scroll horizontally to compare every model.
Overall = 1/2 Agentic + 1/2 Task
Muted model colors · Higher is better · 0–100
7-track average
Overall score vs. Cost per Task
28 of 28 models
Most attractive quadrant
Pareto line
Leaderboard
7-track average
Overall = 1/2 Agentic + 1/2 Task
| Rank | Model | Parameter size | Release | Overall | Agentic | Task | Avg cost | Avg time | Avg turns |
|---|---|---|---|---|---|---|---|---|---|
| 1 |
Claude Opus 4.8Anthropic
|
undisclosed |
2026-05-28 | 81.94 | 88.10 | 75.77 | $2.14 | 14.5 min | 18.7 |
| 2 |
Claude Opus 4.6Anthropic
|
undisclosed |
2026-02-05 | 77.23 | 89.10 | 65.36 | $6.57 | 18.2 min | 18.3 |
| 3 |
Gemini 3.1 ProGoogle
|
undisclosed |
2026-02-19 | 75.21 | 83.49 | 66.93 | $3.85 | 18.0 min | 18.1 |
| 4 |
GPT-5.6 SolOpenAI
|
undisclosed |
2026-07-09 | 75.05 | 80.09 | 70.01 | $0.95 | 10.8 min | 11.7 |
| 5 |
Claude Sonnet 5Anthropic
|
undisclosed |
2026-06-30 | 75.04 | 80.53 | 69.53 | $2.29 | 15.1 min | 30.5 |
| 6 |
GPT-5.5OpenAI
|
undisclosed |
2026-04-23 | 74.14 | 85.38 | 62.90 | $1.90 | 20.5 min | 15.4 |
| 7 |
GLM-5Z.ai
|
744B | 2026-02-12 | 72.86 | 89.35 | 56.38 | $1.29 | 17.8 min | 31.3 |
| 8 |
DeepSeek V4 ProDeepSeek
|
1.6T | 2026-04-24 | 72.43 | 81.34 | 63.51 | $0.30 | 18.3 min | 24.6 |
| 9 |
Qwen3.5-397BAlibaba Cloud
|
397B | 2026-02-15 | 69.87 | 80.46 | 59.27 | $1.01 | 20.8 min | 25.1 |
| 10 |
GPT-5.6 TerraOpenAI
|
undisclosed |
2026-07-09 | 68.29 | 74.05 | 62.53 | $0.45 | 10.7 min | 10.2 |
| 11 |
MiniMax M2.5MiniMax
|
230B | 2026-02-12 | 66.31 | 77.71 | 54.91 | $1.36 | 20.6 min | 28.5 |
| 12 |
GPT-5.4OpenAI
|
undisclosed |
2026-03-05 | 66.09 | 76.76 | 55.43 | $2.36 | 16.7 min | 10.4 |
| 13 |
Qwen3.6-27BAlibaba Cloud
|
27B | 2026-04-21 | 65.29 | 73.27 | 57.30 | free |
23.0 min | 32.0 |
| 14 |
GPT-5.6 LunaOpenAI
|
undisclosed |
2026-07-09 | 63.98 | 70.77 | 57.20 | $0.21 | 8.9 min | 13.4 |
| 15 |
Qwen3.5-35B-A3BAlibaba Cloud
|
35B | 2026-02-24 | 57.94 | 68.91 | 46.97 | free |
28.4 min | 30.5 |
| 16 |
GLM-5.2Z.ai
|
753B | 2026-06-16 | 56.32 | 64.39 | 48.24 | $0.73 | 15.7 min | 28.6 |
| 17 |
Nemotron 3.5 Lightning 30B-A3BNVIDIA
|
30B | 2026-08-11 | 39.65 | 46.93 | 32.38 | $0.09 | 14.3 min | 42.5 |
| 18 |
Gemma 4 26B-A4BGoogle
|
26B | 2026-03-11 | 33.89 | 39.63 | 33.97 | $0.05 | 17.3 min | 14.8 |
| 19 |
Gemma 4 31BGoogle
|
31B | 2026-03-11 | 30.43 | 35.14 | 31.79 | $0.02 | 27.9 min | 8.1 |
| 20 |
GPT-OSS-120BOpenAI
|
117B | 2025-08-05 | 24.33 | 35.35 | 13.31 | $0.00 | 4.4 min | 9.4 |
| 21 |
Gemma 4 12BGoogle
|
12B | 2026-05-23 | 22.43 | 20.19 | 36.78 | $0.02 | 24.0 min | 17.1 |
| 22 |
Gemma 4 E4BGoogle
|
8B | 2026-03-02 | 21.94 | 26.17 | 19.61 | $0.01 | 7.4 min | 6.8 |
| 23 |
Qwen3.5-9BAlibaba Cloud
|
9B | 2026-02-27 | 19.87 | 11.33 | 28.41 | free |
19.3 min | 52.4 |
| 24 |
Nemotron Nano 9B v2NVIDIA
|
9B | 2025-08-18 | 18.68 | 24.52 | 12.84 | free |
12.0 min | 22.8 |
| 25 |
Nemotron 3 Nano 30B-A3BNVIDIA
|
30B | 2025-12-04 | 18.01 | 25.54 | 13.46 | $0.02 | 2.5 min | 16.3 |
| 26 |
GPT-OSS-20BOpenAI
|
21B | 2025-08-05 | 17.63 | 25.33 | 9.93 | $0.00 | 2.7 min | 5.1 |
| 27 |
Nemotron 3 Nano 4BNVIDIA
|
4B | 2026-03-07 | 9.06 | 18.12 | 0.00 | $0.01 | 5.9 min | 9.7 |
| 28 |
Gemma 4 E2BGoogle
|
2B | 2026-03-02 | 9.02 | 18.04 | 0.00 | $0.02 | 4.3 min | 18.9 |
| 29 |
Mistral 7B Instruct v0.3Mistral AI
|
7B | 2024-05-22 | 6.80 | 13.50 | 0.00 | $0.00 | 1.5 min | 2.7 |
| 30 |
Qwen3.5-4BAlibaba Cloud
|
4B | 2026-02-27 | 6.05 | 12.10 | 0.00 | free |
13.6 min | 72.6 |