Add evaluation results

#4
by SaylorTwift HF Staff - opened
Files changed (1) hide show
  1. .eval_results/Inkling-2-Small.yaml +88 -0
.eval_results/Inkling-2-Small.yaml ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: datacurve/deep-swe
3
+ task_id: deep_swe
4
+ value: 72.1
5
+ date: "2026-11-13"
6
+ source:
7
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
8
+ name: "Inkling-2-Small model card"
9
+ notes: "DeepSWE v1.1, mini-swe-agent v2.4.6."
10
+ - dataset:
11
+ id: harborframework/terminal-bench-science
12
+ task_id: terminalbenchscience_0_1
13
+ value: 5.7
14
+ date: "2026-11-13"
15
+ source:
16
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
17
+ name: "Inkling-2-Small model card"
18
+ notes: "mini-swe-agent v2.4.6."
19
+ - dataset:
20
+ id: ScaleAI/SWE-bench_Pro
21
+ task_id: SWE_Bench_Pro_hard
22
+ value: 86.3
23
+ date: "2026-11-13"
24
+ source:
25
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
26
+ name: "Inkling-2-Small model card"
27
+ notes: "mini-swe-agent v2.4.6."
28
+ - dataset:
29
+ id: likaixin/ScreenSpot-Pro
30
+ task_id: overall
31
+ value: 83.8
32
+ date: "2026-11-13"
33
+ source:
34
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
35
+ name: "Inkling-2-Small model card"
36
+ notes: "Images with a long side over 4,096 px downsampled to that limit (aspect ratio kept, ground truth rescaled)."
37
+ - dataset:
38
+ id: likaixin/ScreenSpot-Pro
39
+ task_id: overall
40
+ value: 92.0
41
+ date: "2026-11-13"
42
+ source:
43
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
44
+ name: "Inkling-2-Small model card"
45
+ notes: "With Python tool. Images with a long side over 4,096 px downsampled to that limit (aspect ratio kept, ground truth rescaled)."
46
+ - dataset:
47
+ id: MMMU/MMMU_Pro
48
+ task_id: mmmu_pro_standard_10_options
49
+ value: 81.6
50
+ date: "2026-11-13"
51
+ source:
52
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
53
+ name: "Inkling-2-Small model card"
54
+ - dataset:
55
+ id: cais/hle
56
+ task_id: hle
57
+ value: 44.1
58
+ date: "2026-11-13"
59
+ source:
60
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
61
+ name: "Inkling-2-Small model card"
62
+ notes: "Text-only subset."
63
+ - dataset:
64
+ id: Delores-Lin/MDPBench
65
+ task_id: overall
66
+ value: 81.4
67
+ date: "2026-11-13"
68
+ source:
69
+ url: https://huggingface.co/thinkingmachines/Inkling-2-Small
70
+ name: "Inkling-2-Small model card"
71
+ - dataset:
72
+ id: ScaleAI/SWE-bench_Pro
73
+ task_id: SWE_Bench_Pro
74
+ value: 97.7
75
+ date: "2026-11-13"
76
+ source:
77
+ url: https://thinkingmachines.ai/model-card/inkling-2-small/
78
+ name: "Inkling 2 Small model card (thinkingmachines.ai)"
79
+ notes: "SWE-Bench Pro v2 (Full), mini-swe-agent v2.4.6. As reported on the card; higher than the same model's Hard-subset score (86.3), pending confirmation."
80
+ - dataset:
81
+ id: SWE-bench/SWE-bench_Multimodal
82
+ task_id: swe_bench_multimodal_%_resolved
83
+ value: 43.1
84
+ date: "2026-11-13"
85
+ source:
86
+ url: https://thinkingmachines.ai/model-card/inkling-2-small/
87
+ name: "Inkling 2 Small model card (thinkingmachines.ai)"
88
+ notes: "SWE-bench Multimodal v2, mini-swe-agent (Vals harness)."