MitakaKuma commited on
Commit
d1d7d7d
·
verified ·
1 Parent(s): 4c945b2

Add provider logos and aggregate operational metrics

Browse files
README.md CHANGED
@@ -16,3 +16,7 @@ Simple aggregate leaderboard for the seven tracks in the public
16
  [AutoMedBench Lite release](https://huggingface.co/datasets/MitakaKuma/AutoMedBench-Lite-release).
17
 
18
  Scores are sourced from the [official AutoMedBench results](https://automedbench.github.io/).
 
 
 
 
 
16
  [AutoMedBench Lite release](https://huggingface.co/datasets/MitakaKuma/AutoMedBench-Lite-release).
17
 
18
  Scores are sourced from the [official AutoMedBench results](https://automedbench.github.io/).
19
+
20
+ Provider icons are vendored from
21
+ [`@lobehub/icons-static-svg` 1.91.0](https://github.com/lobehub/lobe-icons)
22
+ under the MIT license; see `THIRD_PARTY_NOTICES.md`.
THIRD_PARTY_NOTICES.md ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Third-party notices
2
+
3
+ The provider SVG icons in `assets/` are from
4
+ [`@lobehub/icons-static-svg` 1.91.0](https://github.com/lobehub/lobe-icons).
5
+
6
+ MIT License
7
+
8
+ Copyright (c) 2023 LobeHub
9
+
10
+ Permission is hereby granted, free of charge, to any person obtaining a copy
11
+ of this software and associated documentation files (the "Software"), to deal
12
+ in the Software without restriction, including without limitation the rights
13
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
14
+ copies of the Software, and to permit persons to whom the Software is
15
+ furnished to do so, subject to the following conditions:
16
+
17
+ The above copyright notice and this permission notice shall be included in all
18
+ copies or substantial portions of the Software.
19
+
20
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
21
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
22
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
23
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
24
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
25
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
26
+ SOFTWARE.
assets/anthropic.svg ADDED
assets/gemini.svg ADDED
assets/minimax.svg ADDED
assets/openai.svg ADDED
assets/qwen.svg ADDED
assets/zai.svg ADDED
index.html CHANGED
@@ -19,6 +19,8 @@
19
  --primary: #4a7355;
20
  --primary-dark: #3e6248;
21
  --primary-soft: #e6ede5;
 
 
22
  --gold: #c9a35a;
23
  --silver: #a9afb7;
24
  --bronze: #b88556;
@@ -45,7 +47,7 @@
45
  a { color: inherit; }
46
 
47
  .page {
48
- width: min(100% - 32px, 980px);
49
  margin: 0 auto;
50
  padding: 44px 0 36px;
51
  }
@@ -207,14 +209,14 @@
207
 
208
  table {
209
  width: 100%;
210
- min-width: 620px;
211
  border-collapse: collapse;
212
  font-variant-numeric: tabular-nums;
213
  }
214
 
215
  th {
216
  height: 38px;
217
- padding: 8px 16px;
218
  border-bottom: 1px solid var(--edge);
219
  color: var(--ink-soft);
220
  background: #f9f5ec;
@@ -226,13 +228,19 @@
226
  }
227
 
228
  th.rank,
229
- td.rank { width: 64px; text-align: center; }
230
 
231
  th.model,
232
- td.model { text-align: left; }
233
 
234
  th.score,
235
- td.score { width: 150px; }
 
 
 
 
 
 
236
 
237
  tbody tr:nth-child(odd) { background: var(--panel-strong); }
238
  tbody tr:nth-child(even) { background: #f8f5ed; }
@@ -247,24 +255,82 @@
247
  tbody tr:hover { background: #edf3eb; }
248
 
249
  td {
250
- height: 54px;
251
- padding: 10px 16px;
252
  text-align: right;
253
  }
254
 
255
- td.model {
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
256
  font-weight: 680;
257
  letter-spacing: -0.01em;
 
 
 
 
 
 
 
 
 
258
  }
259
 
260
- td.score {
261
- color: #485044;
 
 
262
  font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace;
263
- font-size: 13px;
264
- font-weight: 650;
 
 
 
 
 
 
 
265
  }
266
 
267
- td.overall { color: var(--primary-dark); font-weight: 800; }
 
 
 
 
 
 
 
 
 
 
268
 
269
  .rank-mark {
270
  display: inline-grid;
@@ -315,7 +381,7 @@
315
  }
316
 
317
  @media (max-width: 720px) {
318
- .page { width: min(100% - 20px, 980px); padding-top: 26px; }
319
  .hero { display: block; }
320
  .actions { margin: 20px 0 0 56px; }
321
  .leaderboard-head { align-items: flex-start; }
@@ -372,53 +438,111 @@
372
  <tr>
373
  <th class="rank" scope="col">Rank</th>
374
  <th class="model" scope="col">Model</th>
 
375
  <th class="score" scope="col">Overall</th>
376
  <th class="score" scope="col">Agentic</th>
377
  <th class="score" scope="col">Task</th>
 
 
 
378
  </tr>
379
  </thead>
380
  <tbody>
381
  <tr>
382
  <td class="rank"><span class="rank-mark">1</span></td>
383
- <td class="model">Claude Opus 4.6</td>
 
 
 
 
 
 
384
  <td class="score overall">77.23</td>
385
  <td class="score">89.10</td>
386
  <td class="score">65.36</td>
 
 
 
387
  </tr>
388
  <tr>
389
  <td class="rank"><span class="rank-mark">2</span></td>
390
- <td class="model">Gemini 3.1 Pro</td>
 
 
 
 
 
 
391
  <td class="score overall">75.21</td>
392
  <td class="score">83.49</td>
393
  <td class="score">66.93</td>
 
 
 
394
  </tr>
395
  <tr>
396
  <td class="rank"><span class="rank-mark">3</span></td>
397
- <td class="model">GLM-5</td>
 
 
 
 
 
 
398
  <td class="score overall">72.86</td>
399
  <td class="score">89.35</td>
400
  <td class="score">56.38</td>
 
 
 
401
  </tr>
402
  <tr>
403
  <td class="rank"><span class="rank-mark">4</span></td>
404
- <td class="model">Qwen3.5</td>
 
 
 
 
 
 
405
  <td class="score overall">69.87</td>
406
  <td class="score">80.46</td>
407
  <td class="score">59.27</td>
 
 
 
408
  </tr>
409
  <tr>
410
  <td class="rank"><span class="rank-mark">5</span></td>
411
- <td class="model">MiniMax M2.5</td>
 
 
 
 
 
 
412
  <td class="score overall">66.31</td>
413
  <td class="score">77.71</td>
414
  <td class="score">54.91</td>
 
 
 
415
  </tr>
416
  <tr>
417
  <td class="rank"><span class="rank-mark">6</span></td>
418
- <td class="model">ChatGPT 5.4</td>
 
 
 
 
 
 
419
  <td class="score overall">66.09</td>
420
  <td class="score">76.76</td>
421
  <td class="score">55.43</td>
 
 
 
422
  </tr>
423
  </tbody>
424
  </table>
@@ -430,7 +554,7 @@
430
  </footer>
431
  </section>
432
 
433
- <p class="note">Macro-average across the seven Lite release tasks. Scores are shown on a 0-100 scale.</p>
434
  </main>
435
  </body>
436
  </html>
 
19
  --primary: #4a7355;
20
  --primary-dark: #3e6248;
21
  --primary-soft: #e6ede5;
22
+ --overall: #111411;
23
+ --metric: #7f877b;
24
  --gold: #c9a35a;
25
  --silver: #a9afb7;
26
  --bronze: #b88556;
 
47
  a { color: inherit; }
48
 
49
  .page {
50
+ width: min(100% - 32px, 1180px);
51
  margin: 0 auto;
52
  padding: 44px 0 36px;
53
  }
 
209
 
210
  table {
211
  width: 100%;
212
+ min-width: 1040px;
213
  border-collapse: collapse;
214
  font-variant-numeric: tabular-nums;
215
  }
216
 
217
  th {
218
  height: 38px;
219
+ padding: 8px 12px;
220
  border-bottom: 1px solid var(--edge);
221
  color: var(--ink-soft);
222
  background: #f9f5ec;
 
228
  }
229
 
230
  th.rank,
231
+ td.rank { width: 58px; text-align: center; }
232
 
233
  th.model,
234
+ td.model { width: 230px; text-align: left; }
235
 
236
  th.score,
237
+ td.score { width: 84px; }
238
+
239
+ th.release,
240
+ td.release { width: 112px; }
241
+
242
+ th.operational,
243
+ td.operational { width: 88px; }
244
 
245
  tbody tr:nth-child(odd) { background: var(--panel-strong); }
246
  tbody tr:nth-child(even) { background: #f8f5ed; }
 
255
  tbody tr:hover { background: #edf3eb; }
256
 
257
  td {
258
+ height: 62px;
259
+ padding: 10px 12px;
260
  text-align: right;
261
  }
262
 
263
+ .model-cell {
264
+ display: flex;
265
+ align-items: center;
266
+ gap: 10px;
267
+ min-width: 0;
268
+ }
269
+
270
+ .logo-wrap {
271
+ display: grid;
272
+ width: 32px;
273
+ height: 32px;
274
+ flex: 0 0 auto;
275
+ place-items: center;
276
+ border: 1px solid var(--edge);
277
+ border-radius: 8px;
278
+ background: #f4f0e7;
279
+ }
280
+
281
+ .company-logo {
282
+ display: block;
283
+ width: 19px;
284
+ height: 19px;
285
+ opacity: 0.86;
286
+ }
287
+
288
+ .model-copy { min-width: 0; }
289
+
290
+ .model-name {
291
+ display: block;
292
+ overflow: hidden;
293
+ color: var(--ink);
294
  font-weight: 680;
295
  letter-spacing: -0.01em;
296
+ text-overflow: ellipsis;
297
+ white-space: nowrap;
298
+ }
299
+
300
+ .provider {
301
+ display: block;
302
+ margin-top: 1px;
303
+ color: var(--metric);
304
+ font-size: 11px;
305
  }
306
 
307
+ td.score,
308
+ td.release,
309
+ td.operational {
310
+ color: var(--metric);
311
  font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace;
312
+ font-size: 12px;
313
+ font-weight: 600;
314
+ white-space: nowrap;
315
+ }
316
+
317
+ td.overall {
318
+ color: var(--overall);
319
+ font-size: 14px;
320
+ font-weight: 850;
321
  }
322
 
323
+ .release-link {
324
+ color: var(--metric);
325
+ text-decoration: none;
326
+ }
327
+
328
+ .release-link:hover,
329
+ .release-link:focus-visible {
330
+ color: var(--primary-dark);
331
+ text-decoration: underline;
332
+ outline: none;
333
+ }
334
 
335
  .rank-mark {
336
  display: inline-grid;
 
381
  }
382
 
383
  @media (max-width: 720px) {
384
+ .page { width: min(100% - 20px, 1180px); padding-top: 26px; }
385
  .hero { display: block; }
386
  .actions { margin: 20px 0 0 56px; }
387
  .leaderboard-head { align-items: flex-start; }
 
438
  <tr>
439
  <th class="rank" scope="col">Rank</th>
440
  <th class="model" scope="col">Model</th>
441
+ <th class="release" scope="col">Release</th>
442
  <th class="score" scope="col">Overall</th>
443
  <th class="score" scope="col">Agentic</th>
444
  <th class="score" scope="col">Task</th>
445
+ <th class="operational" scope="col">Avg cost</th>
446
+ <th class="operational" scope="col">Avg time</th>
447
+ <th class="operational" scope="col">Avg turns</th>
448
  </tr>
449
  </thead>
450
  <tbody>
451
  <tr>
452
  <td class="rank"><span class="rank-mark">1</span></td>
453
+ <td class="model">
454
+ <div class="model-cell">
455
+ <span class="logo-wrap"><img class="company-logo" src="assets/anthropic.svg" alt="Anthropic logo"></span>
456
+ <span class="model-copy"><span class="model-name">Claude Opus 4.6</span><span class="provider">Anthropic</span></span>
457
+ </div>
458
+ </td>
459
+ <td class="release"><a class="release-link" href="https://www.anthropic.com/news/claude-opus-4-6" target="_blank" rel="noreferrer">2026-02-05</a></td>
460
  <td class="score overall">77.23</td>
461
  <td class="score">89.10</td>
462
  <td class="score">65.36</td>
463
+ <td class="operational">$6.57</td>
464
+ <td class="operational">18.2 min</td>
465
+ <td class="operational">18.3</td>
466
  </tr>
467
  <tr>
468
  <td class="rank"><span class="rank-mark">2</span></td>
469
+ <td class="model">
470
+ <div class="model-cell">
471
+ <span class="logo-wrap"><img class="company-logo" src="assets/gemini.svg" alt="Google Gemini logo"></span>
472
+ <span class="model-copy"><span class="model-name">Gemini 3.1 Pro</span><span class="provider">Google</span></span>
473
+ </div>
474
+ </td>
475
+ <td class="release"><a class="release-link" href="https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-pro/" target="_blank" rel="noreferrer">2026-02-19</a></td>
476
  <td class="score overall">75.21</td>
477
  <td class="score">83.49</td>
478
  <td class="score">66.93</td>
479
+ <td class="operational">$3.85</td>
480
+ <td class="operational">18.0 min</td>
481
+ <td class="operational">18.1</td>
482
  </tr>
483
  <tr>
484
  <td class="rank"><span class="rank-mark">3</span></td>
485
+ <td class="model">
486
+ <div class="model-cell">
487
+ <span class="logo-wrap"><img class="company-logo" src="assets/zai.svg" alt="Z.ai logo"></span>
488
+ <span class="model-copy"><span class="model-name">GLM-5</span><span class="provider">Z.ai</span></span>
489
+ </div>
490
+ </td>
491
+ <td class="release"><a class="release-link" href="https://z.ai/blog/glm-5?id=GLM5" target="_blank" rel="noreferrer">2026-02-12</a></td>
492
  <td class="score overall">72.86</td>
493
  <td class="score">89.35</td>
494
  <td class="score">56.38</td>
495
+ <td class="operational">$1.29</td>
496
+ <td class="operational">17.8 min</td>
497
+ <td class="operational">31.3</td>
498
  </tr>
499
  <tr>
500
  <td class="rank"><span class="rank-mark">4</span></td>
501
+ <td class="model">
502
+ <div class="model-cell">
503
+ <span class="logo-wrap"><img class="company-logo" src="assets/qwen.svg" alt="Qwen logo"></span>
504
+ <span class="model-copy"><span class="model-name">Qwen3.5</span><span class="provider">Alibaba Cloud</span></span>
505
+ </div>
506
+ </td>
507
+ <td class="release"><a class="release-link" href="https://qwen.ai/blog?id=qwen3.5" target="_blank" rel="noreferrer">2026-02-15</a></td>
508
  <td class="score overall">69.87</td>
509
  <td class="score">80.46</td>
510
  <td class="score">59.27</td>
511
+ <td class="operational">$1.01</td>
512
+ <td class="operational">20.8 min</td>
513
+ <td class="operational">25.1</td>
514
  </tr>
515
  <tr>
516
  <td class="rank"><span class="rank-mark">5</span></td>
517
+ <td class="model">
518
+ <div class="model-cell">
519
+ <span class="logo-wrap"><img class="company-logo" src="assets/minimax.svg" alt="MiniMax logo"></span>
520
+ <span class="model-copy"><span class="model-name">MiniMax M2.5</span><span class="provider">MiniMax</span></span>
521
+ </div>
522
+ </td>
523
+ <td class="release"><a class="release-link" href="https://www.minimax.io/news/minimax-m25" target="_blank" rel="noreferrer">2026-02-12</a></td>
524
  <td class="score overall">66.31</td>
525
  <td class="score">77.71</td>
526
  <td class="score">54.91</td>
527
+ <td class="operational">$1.36</td>
528
+ <td class="operational">20.6 min</td>
529
+ <td class="operational">28.5</td>
530
  </tr>
531
  <tr>
532
  <td class="rank"><span class="rank-mark">6</span></td>
533
+ <td class="model">
534
+ <div class="model-cell">
535
+ <span class="logo-wrap"><img class="company-logo" src="assets/openai.svg" alt="OpenAI logo"></span>
536
+ <span class="model-copy"><span class="model-name">ChatGPT 5.4</span><span class="provider">OpenAI</span></span>
537
+ </div>
538
+ </td>
539
+ <td class="release"><a class="release-link" href="https://openai.com/index/introducing-gpt-5-4/" target="_blank" rel="noreferrer">2026-03-05</a></td>
540
  <td class="score overall">66.09</td>
541
  <td class="score">76.76</td>
542
  <td class="score">55.43</td>
543
+ <td class="operational">$2.36</td>
544
+ <td class="operational">16.7 min</td>
545
+ <td class="operational">10.4</td>
546
  </tr>
547
  </tbody>
548
  </table>
 
554
  </footer>
555
  </section>
556
 
557
+ <p class="note">All metrics are equal-weight averages across the seven Lite release tasks. Cost is USD; time is wall-clock.</p>
558
  </main>
559
  </body>
560
  </html>