Hugging Face's logo

inclusionAI
/
Ling-3.0-tiny-base

Safetensors
bailing_hybrid
custom_code
Model card Files
xet
 Community
yuxintian commited on
Commit
f141aa7
·
verified ·
1 Parent(s): 0799d31

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +253 -1
README.md CHANGED
@@ -54,7 +54,259 @@ These checkpoints are released to support continued pretraining, fine-tuning, an
54
  ## Base Model Evaluation
55
  To systematically assess the capabilities of the base model, we use a self-built comprehensive benchmark suite covering several key domains, including knowledge, coding, mathematics, reasoning, multilingual understanding, and long-context comprehension. The performance of the pretrained base checkpoint, i.e., `Ling-3.0-tiny-base`, is compared below:
56
 
57
- <img src="https://intranetproxy.alipay.com/skylark/lark/0/2026/png/62256938/1787145422924-5c03d4b9-ee56-4d84-8927-58edb24b24c1.png" width="1578.5" title="" crop="0,0,1,1" id="u9090be40" class="ne-image">
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
  ## **Intended Use**
60
  Recommended use cases:
 
54
  ## Base Model Evaluation
55
  To systematically assess the capabilities of the base model, we use a self-built comprehensive benchmark suite covering several key domains, including knowledge, coding, mathematics, reasoning, multilingual understanding, and long-context comprehension. The performance of the pretrained base checkpoint, i.e., `Ling-3.0-tiny-base`, is compared below:
56
 
57
+ <style>
58
+ .benchmark-table {
59
+ border-collapse: collapse;
60
+ margin: 0 auto;
61
+ }
62
+
63
+ .benchmark-table th,
64
+ .benchmark-table td {
65
+ padding: 0.55rem 0.8rem;
66
+ text-align: center !important;
67
+ vertical-align: middle;
68
+ }
69
+
70
+ .benchmark-table .ling {
71
+ background-color: #e7f6ff;
72
+ }
73
+
74
+ .benchmark-table thead .ling {
75
+ color: #1677ff;
76
+ }
77
+
78
+ .benchmark-table thead tr:first-child .ling {
79
+ border-radius: 28px 28px 0 0;
80
+ }
81
+
82
+ .benchmark-table tbody tr:last-child .ling {
83
+ border-radius: 0 0 28px 28px;
84
+ }
85
+
86
+ .benchmark-table .section-start > * {
87
+ border-top: 2px solid #d0d7de;
88
+ }
89
+
90
+ .benchmark-table ins {
91
+ text-decoration-thickness: 1px;
92
+ text-underline-offset: 0.12em;
93
+ }
94
+
95
+ .benchmark-table .table-footnote {
96
+ padding-top: 0.85rem;
97
+ text-align: left !important;
98
+ color: #57606a;
99
+ font-size: 0.9em;
100
+ }
101
+ </style>
102
+
103
+ <table class="benchmark-table" align="center">
104
+ <thead>
105
+ <tr align="center">
106
+ <th align="center" rowspan="2">Domain</th>
107
+ <th align="center" rowspan="2">Benchmark</th>
108
+ <th align="center" rowspan="2">Shot Config</th>
109
+ <th align="center" class="ling" bgcolor="#e7f6ff">Ling-3.0-tiny-base</th>
110
+ <th align="center">Ling-2.5-mini-base</th>
111
+ <th align="center">Qwen3.5-9B-base</th>
112
+ <th align="center">Qwen3.5-4B-base</th>
113
+ </tr>
114
+ <tr align="center">
115
+ <th align="center" class="ling" bgcolor="#e7f6ff">7.9B A1.3B</th>
116
+ <th align="center">16B A1.4B</th>
117
+ <th align="center">9B</th>
118
+ <th align="center">4B</th>
119
+ </tr>
120
+ </thead>
121
+ <tbody>
122
+ <tr align="center">
123
+ <th align="center" rowspan="6">Knowledge</th>
124
+ <th align="center">CCPM<sub>(EM)</sub></th>
125
+ <td align="center">0-shot</td>
126
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>84.01</ins></td>
127
+ <td align="center">80.77</td>
128
+ <td align="center"><strong>88.31</strong></td>
129
+ <td align="center">81.88</td>
130
+ </tr>
131
+ <tr align="center">
132
+ <th align="center">ARC-C<sub>(EM)</sub></th>
133
+ <td align="center">0-shot</td>
134
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>92.20</ins></td>
135
+ <td align="center">91.19</td>
136
+ <td align="center"><strong>94.58</strong></td>
137
+ <td align="center"><ins>92.20</ins></td>
138
+ </tr>
139
+ <tr align="center">
140
+ <th align="center">AGIEval<sub>(Acc)</sub></th>
141
+ <td align="center">0-shot</td>
142
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>64.49</ins></td>
143
+ <td align="center">63.38</td>
144
+ <td align="center"><strong>66.17</strong></td>
145
+ <td align="center">61.45</td>
146
+ </tr>
147
+ <tr align="center">
148
+ <th align="center">SimpleQA-Verified<sub>(Acc)</sub></th>
149
+ <td align="center">5-shot</td>
150
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>7.20</ins></td>
151
+ <td align="center">7.10</td>
152
+ <td align="center"><strong>10.40</strong></td>
153
+ <td align="center">6.40</td>
154
+ </tr>
155
+ <tr align="center">
156
+ <th align="center">MMLU-Pro<sub>(EM)</sub></th>
157
+ <td align="center">5-shot</td>
158
+ <td align="center" class="ling" bgcolor="#e7f6ff">51.83</td>
159
+ <td align="center">49.89</td>
160
+ <td align="center"><strong>57.76</strong></td>
161
+ <td align="center"><ins>52.06</ins></td>
162
+ </tr>
163
+ <tr align="center">
164
+ <th align="center">CEval<sub>(EM)</sub></th>
165
+ <td align="center">5-shot</td>
166
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>80.59</ins></td>
167
+ <td align="center">80.16</td>
168
+ <td align="center"><strong>81.52</strong></td>
169
+ <td align="center">76.67</td>
170
+ </tr>
171
+ <tr align="center" class="section-start">
172
+ <th align="center" rowspan="7">Code</th>
173
+ <th align="center">HumanEval-Plus<sub>(Pass@1)</sub></th>
174
+ <td align="center">0-shot</td>
175
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>79.27</strong></td>
176
+ <td align="center"><ins>76.22</ins></td>
177
+ <td align="center">52.44</td>
178
+ <td align="center">49.39</td>
179
+ </tr>
180
+ <tr align="center">
181
+ <th align="center">CruxEval<sub>(Pass@1)</sub></th>
182
+ <td align="center">1-shot</td>
183
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>67.44</ins></td>
184
+ <td align="center">63.19</td>
185
+ <td align="center"><strong>68.12</strong></td>
186
+ <td align="center">64.31</td>
187
+ </tr>
188
+ <tr align="center">
189
+ <th align="center">MultiPL-E<sub>(Pass@1)</sub></th>
190
+ <td align="center">1-shot</td>
191
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>64.38</strong></td>
192
+ <td align="center"><ins>64.34</ins></td>
193
+ <td align="center">52.89</td>
194
+ <td align="center">45.47</td>
195
+ </tr>
196
+ <tr align="center">
197
+ <th align="center">LiveCodeBench<sup>1</sup><sub>(Pass@1)</sub></th>
198
+ <td align="center">1-shot</td>
199
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>24.23</strong></td>
200
+ <td align="center"><ins>22.91</ins></td>
201
+ <td align="center">18.06</td>
202
+ <td align="center">13.88</td>
203
+ </tr>
204
+ <tr align="center">
205
+ <th align="center">BigCodeBench<sub>(Pass@1)</sub></th>
206
+ <td align="center">0-shot</td>
207
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>42.89</strong></td>
208
+ <td align="center"><ins>42.54</ins></td>
209
+ <td align="center">29.39</td>
210
+ <td align="center">24.39</td>
211
+ </tr>
212
+ <tr align="center">
213
+ <th align="center">FullStackBench<sub>(Pass@1)</sub></th>
214
+ <td align="center">3-shot</td>
215
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>39.48</strong></td>
216
+ <td align="center"><ins>38.59</ins></td>
217
+ <td align="center">37.23</td>
218
+ <td align="center">33.14</td>
219
+ </tr>
220
+ <tr align="center">
221
+ <th align="center">LCBench<sup>2</sup><sub>(Pass@1)</sub></th>
222
+ <td align="center">3-shot</td>
223
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>41.59</ins></td>
224
+ <td align="center"><strong>46.16</strong></td>
225
+ <td align="center">33.47</td>
226
+ <td align="center">19.46</td>
227
+ </tr>
228
+ <tr align="center" class="section-start">
229
+ <th align="center" rowspan="4">Math</th>
230
+ <th align="center">MATH500<sub>(Acc)</sub></th>
231
+ <td align="center">4-shot</td>
232
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>65.60</ins></td>
233
+ <td align="center"><strong>68.40</strong></td>
234
+ <td align="center">55.20</td>
235
+ <td align="center">49.60</td>
236
+ </tr>
237
+ <tr align="center">
238
+ <th align="center">OlympiadBench<sub>(Acc)</sub></th>
239
+ <td align="center">3-shot</td>
240
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>25.90</strong></td>
241
+ <td align="center"><ins>24.10</ins></td>
242
+ <td align="center">21.84</td>
243
+ <td align="center">17.02</td>
244
+ </tr>
245
+ <tr align="center">
246
+ <th align="center">TheoremQA<sub>(Acc)</sub></th>
247
+ <td align="center">5-shot</td>
248
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>51.64</ins></td>
249
+ <td align="center">50.68</td>
250
+ <td align="center"><strong>52.88</strong></td>
251
+ <td align="center">47.40</td>
252
+ </tr>
253
+ <tr align="center">
254
+ <th align="center">OmniMath<sub>(Acc)</sub></th>
255
+ <td align="center">3-shot</td>
256
+ <td align="center" class="ling" bgcolor="#e7f6ff"><strong>29.70</strong></td>
257
+ <td align="center"><ins>29.02</ins></td>
258
+ <td align="center">22.99</td>
259
+ <td align="center">20.89</td>
260
+ </tr>
261
+ <tr align="center" class="section-start">
262
+ <th align="center" rowspan="2">Reasoning</th>
263
+ <th align="center">CommonSenseQA<sub>(EM)</sub></th>
264
+ <td align="center">5-shot</td>
265
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>83.46</ins></td>
266
+ <td align="center">81.16</td>
267
+ <td align="center"><strong>83.62</strong></td>
268
+ <td align="center">80.18</td>
269
+ </tr>
270
+ <tr align="center">
271
+ <th align="center">BBH<sub>(EM)</sub></th>
272
+ <td align="center">3-shot</td>
273
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>80.64</ins></td>
274
+ <td align="center">77.37</td>
275
+ <td align="center"><strong>84.52</strong></td>
276
+ <td align="center">80.11</td>
277
+ </tr>
278
+ <tr align="center" class="section-start">
279
+ <th align="center" rowspan="2">Long-context</th>
280
+ <th align="center">LongBench<sub>(Acc)</sub></th>
281
+ <td align="center">0-shot</td>
282
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>43.85</ins></td>
283
+ <td align="center">29.93</td>
284
+ <td align="center"><strong>51.87</strong></td>
285
+ <td align="center">39.96</td>
286
+ </tr>
287
+ <tr align="center">
288
+ <th align="center">LEval<sub>(Acc)</sub></th>
289
+ <td align="center">0-shot</td>
290
+ <td align="center" class="ling" bgcolor="#e7f6ff"><ins>68.37</ins></td>
291
+ <td align="center">62.72</td>
292
+ <td align="center"><strong>77.24</strong></td>
293
+ <td align="center">63.09</td>
294
+ </tr>
295
+ </tbody>
296
+ <tfoot>
297
+ <tr>
298
+ <td class="table-footnote" colspan="7" align="left">
299
+ <strong>Note:</strong><br>
300
+ <sup>1</sup> LiveCodeBench (2408-2505)<br>
301
+ <sup>2</sup> LCBench (2301-2502)
302
+ </td>
303
+ </tr>
304
+ </tfoot>
305
+ </table>
306
+
307
+
308
+
309
+ <!-- <img src="https://intranetproxy.alipay.com/skylark/lark/0/2026/png/62256938/1787145422924-5c03d4b9-ee56-4d84-8927-58edb24b24c1.png" width="1578.5" title="" crop="0,0,1,1" id="u9090be40" class="ne-image"> -->
310
 
311
  ## **Intended Use**
312
  Recommended use cases: