Hugging Face's logo

169Pi
/
Alpie_seed

Image-Text-to-Text
Transformers
Safetensors
GGUF
qwen3_5
alpie
169pi
qwen3.5
vision-language
multimodal
reasoning
coding
video-understanding
multilingual
indic
long-context
9b
apache-2.0
conversational
Model card Files
xet
 Community
deepanshupillm commited on
Commit
a27ac02
·
verified ·
1 Parent(s): 2ae5733

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +178 -330
README.md CHANGED
@@ -106,120 +106,184 @@ Training used eight NVIDIA H100 GPUs, with 3,034 GRPO steps across five phases.
106
  >
107
  > Alpie Seed scores are from the FP16 model and do not apply to the 2-bit, 3-bit, or 4-bit variants. **Bold** indicates the best score in each row. Comparison-model scores are taken from published model cards and reports (including the [Qwen3.5-9B model card](https://huggingface.co/Qwen/Qwen3.5-9B)).
108
 
109
- ### Language
110
-
111
- <table style="width: 100%; border-collapse: collapse; text-align: center; font-size: 14px;">
112
- <thead>
113
- <tr style="border-bottom: 2.5px solid #7C3AED; font-weight: 600;">
114
- <th style="text-align: left; padding: 10px; color: #7C3AED;">Benchmark</th>
115
- <th style="padding: 10px; color: #7C3AED;">Qwen3.5<br>9B</th>
116
- <th style="padding: 10px; color: #7C3AED;">Gemma 4<br>12B</th>
117
- <th style="padding: 10px; color: #7C3AED;">Ministral<br>3 8B</th>
118
- <th style="padding: 10px; color: #7C3AED;">GPT-OSS<br>20B</th>
119
- <th style="padding: 10px; color: #7C3AED;">Sarvam<br>30B</th>
120
- <th style="padding: 10px; color: #7C3AED;">Nemotron 3.5<br>Lightning</th>
121
- <th style="padding: 10px; color: #7C3AED;"><b>Alpie Seed<br>9B</b></th>
122
- </tr>
123
- </thead>
124
- <tbody>
125
- <tr style="background-color: #F5F3FF; color: #6D28D9; font-weight: bold; text-align: left;">
126
- <td colspan="8" style="padding: 8px 12px;">Knowledge &amp; STEM</td>
127
- </tr>
128
- <tr style="border-bottom: 1px solid #F1F5F9;">
129
- <td style="text-align: left; padding: 8px 12px;">MMLU</td>
130
- <td>--</td><td>--</td><td>76.1</td><td>85.3</td><td>85.1</td><td>78.63</td>
131
- <td><b>87.30</b></td>
132
- </tr>
133
- <tr style="border-bottom: 1px solid #F1F5F9;">
134
- <td style="text-align: left; padding: 8px 12px;">MMLU-Pro</td>
135
- <td>82.5</td><td>77.2</td><td>75.00</td><td>77.2</td><td>80.0</td><td>81.94</td>
136
- <td><b>82.54</b></td>
137
- </tr>
138
- <tr style="border-bottom: 1px solid #F1F5F9;">
139
- <td style="text-align: left; padding: 8px 12px;">GPQA Diamond</td>
140
- <td>81.7</td><td>78.8</td><td>38.6</td><td>71.5</td><td>66.5</td><td>75.44</td>
141
- <td><b>84.32</b></td>
142
- </tr>
143
- <tr style="background-color: #F5F3FF; color: #6D28D9; font-weight: bold; text-align: left;">
144
- <td colspan="8" style="padding: 8px 12px;">Mathematics</td>
145
- </tr>
146
- <tr style="border-bottom: 1px solid #F1F5F9;">
147
- <td style="text-align: left; padding: 8px 12px;">AIME 2026</td>
148
- <td>69.4</td><td>77.5</td><td>--</td><td>--</td><td>--</td><td>--</td>
149
- <td><b>83.33</b></td>
150
- </tr>
151
- <tr style="background-color: #F5F3FF; color: #6D28D9; font-weight: bold; text-align: left;">
152
- <td colspan="8" style="padding: 8px 12px;">Coding &amp; Software Engineering</td>
153
- </tr>
154
- <tr style="border-bottom: 1px solid #F1F5F9;">
155
- <td style="text-align: left; padding: 8px 12px;">LiveCodeBench v6</td>
156
- <td>65.6</td><td>72.0</td><td>61.6</td><td>61.0</td><td>70.0</td><td>--</td>
157
- <td><b>72.99</b></td>
158
- </tr>
159
- <tr style="border-bottom: 1px solid #F1F5F9;">
160
- <td style="text-align: left; padding: 8px 12px;">SWE-Bench Verified</td>
161
- <td>60.0</td><td>52.0</td><td>--</td><td>60.7</td><td>34.0</td><td>51.56</td>
162
- <td><b>61.20</b></td>
163
- </tr>
164
- </tbody>
165
- </table>
166
-
167
- ### Vision Language
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
168
 
169
- <table style="width: 100%; border-collapse: collapse; text-align: center; font-size: 14px;">
170
- <thead>
171
- <tr style="border-bottom: 2.5px solid #7C3AED; font-weight: 600;">
172
- <th style="text-align: left; padding: 10px; color: #7C3AED;">Benchmark</th>
173
- <th style="padding: 10px; color: #7C3AED;">GPT-5-Nano<br>2025-08-07</th>
174
- <th style="padding: 10px; color: #7C3AED;">Gemini-2.5<br>Flash-Lite</th>
175
- <th style="padding: 10px; color: #7C3AED;">Qwen3-VL<br>30B-A3B</th>
176
- <th style="padding: 10px; color: #7C3AED;">Qwen3.5<br>9B</th>
177
- <th style="padding: 10px; color: #7C3AED;">Gemma 4<br>12B</th>
178
- <th style="padding: 10px; color: #7C3AED;"><b>Alpie Seed<br>9B</b></th>
179
- </tr>
180
- </thead>
181
- <tbody>
182
- <tr style="background-color: #F5F3FF; color: #6D28D9; font-weight: bold; text-align: left;">
183
- <td colspan="7" style="padding: 8px 12px;">STEM and Puzzle</td>
184
- </tr>
185
- <tr style="border-bottom: 1px solid #F1F5F9;">
186
- <td style="text-align: left; padding: 8px 12px;">MMMU-Pro</td>
187
- <td>57.2</td><td>59.7</td><td>63.0</td><td><b>70.1</b></td><td>69.1</td>
188
- <td>68.70</td>
189
- </tr>
190
- <tr style="border-bottom: 1px solid #F1F5F9;">
191
- <td style="text-align: left; padding: 8px 12px;">MathVision</td>
192
- <td>62.2</td><td>52.1</td><td>65.7</td><td>78.9</td><td><b>79.7</b></td>
193
- <td>76.76</td>
194
- </tr>
195
- <tr style="background-color: #F5F3FF; color: #6D28D9; font-weight: bold; text-align: left;">
196
- <td colspan="7" style="padding: 8px 12px;">General VQA</td>
197
- </tr>
198
- <tr style="border-bottom: 1px solid #F1F5F9;">
199
- <td style="text-align: left; padding: 8px 12px;">RealWorldQA</td>
200
- <td>71.8</td><td>72.2</td><td>77.4</td><td>80.3</td><td>--</td>
201
- <td><b>80.39</b></td>
202
- </tr>
203
- <tr style="border-bottom: 1px solid #F1F5F9;">
204
- <td style="text-align: left; padding: 8px 12px;">MMStar</td>
205
- <td>68.6</td><td>69.1</td><td>75.5</td><td><b>79.7</b></td><td>--</td>
206
- <td>78.87</td>
207
- </tr>
208
- <tr style="border-bottom: 1px solid #F1F5F9;">
209
- <td style="text-align: left; padding: 8px 12px;">HallusionBench</td>
210
- <td>58.4</td><td>64.5</td><td>66.0</td><td>69.3</td><td>--</td>
211
- <td><b>79.88</b></td>
212
- </tr>
213
- <tr style="background-color: #F5F3FF; color: #6D28D9; font-weight: bold; text-align: left;">
214
- <td colspan="7" style="padding: 8px 12px;">Text Recognition and Document Understanding</td>
215
- </tr>
216
- <tr style="border-bottom: 1px solid #F1F5F9;">
217
- <td style="text-align: left; padding: 8px 12px;">OCRBench</td>
218
- <td>75.3</td><td>82.5</td><td>83.9</td><td>89.2</td><td>--</td>
219
- <td><b>90.30</b></td>
220
- </tr>
221
- </tbody>
222
- </table>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
223
 
224
  - SWE-Bench Verified: resolved rate using Mini-SWE-agent, with retries disabled.
225
  - Comparison-model scores were not re-run by 169Pi, and harnesses, prompts, and benchmark versions differ, so cross-model comparisons are indicative rather than strictly like-for-like. Scores vary with dataset versions, prompts, evaluation harnesses, and inference settings.
@@ -235,220 +299,4 @@ Training used eight NVIDIA H100 GPUs, with 3,034 GRPO steps across five phases.
235
  Qwen3.5 support requires a recent Transformers release (Transformers 5 or later).
236
 
237
  ```bash
238
- pip install -U "transformers>=5.2.0" torch torchvision pillow accelerate
239
- ```
240
-
241
- ### Serve with vLLM (OpenAI-compatible API)
242
-
243
- Start the standard Alpie Seed 9B server:
244
-
245
- ```bash
246
- vllm serve 169Pi/Alpie_seed \
247
- --port 8000 \
248
- --max-model-len 262144 \
249
- --reasoning-parser qwen3
250
- ```
251
-
252
- To enable automatic tool-call parsing, use this command instead:
253
-
254
- ```bash
255
- vllm serve 169Pi/Alpie_seed \
256
- --port 8000 \
257
- --max-model-len 262144 \
258
- --reasoning-parser qwen3 \
259
- --enable-auto-tool-choice \
260
- --tool-call-parser qwen3_coder
261
- ```
262
-
263
- The API is available at `http://localhost:8000/v1`. The tool-calling options enable vLLM to parse model-generated tool calls; your application is still responsible for executing the requested tools and returning their results.
264
-
265
- ### Run with Transformers
266
-
267
- The following examples load the full vision-language checkpoint using `AutoModelForImageTextToText` and `AutoProcessor`. They retain the model's multimodal architecture. The `enable_thinking` argument is passed to the chat template to select the response mode.
268
-
269
- #### 1. Thinking mode (`enable_thinking=True`)
270
-
271
- ```python
272
- import torch
273
- from transformers import AutoModelForImageTextToText, AutoProcessor
274
-
275
- MODEL_ID = "169Pi/Alpie_seed"
276
-
277
- processor = AutoProcessor.from_pretrained(MODEL_ID)
278
- model = AutoModelForImageTextToText.from_pretrained(
279
- MODEL_ID,
280
- torch_dtype="auto",
281
- device_map="auto",
282
- )
283
- model.eval()
284
-
285
- messages = [
286
- {
287
- "role": "user",
288
- "content": [
289
- {"type": "text", "text": "Explain why the sky appears blue."}
290
- ],
291
- }
292
- ]
293
-
294
- inputs = processor.apply_chat_template(
295
- messages,
296
- tokenize=True,
297
- add_generation_prompt=True,
298
- enable_thinking=True,
299
- return_dict=True,
300
- return_tensors="pt",
301
- ).to(model.device)
302
-
303
- with torch.inference_mode():
304
- output_ids = model.generate(
305
- **inputs,
306
- max_new_tokens=4096,
307
- do_sample=True,
308
- temperature=0.7,
309
- top_p=0.8,
310
- )
311
-
312
- new_tokens = output_ids[0, inputs["input_ids"].shape[1]:]
313
- response = processor.decode(new_tokens, skip_special_tokens=True)
314
- print(response)
315
- ```
316
-
317
- #### 2. Non-thinking mode (`enable_thinking=False`)
318
-
319
- ```python
320
- import torch
321
- from transformers import AutoModelForImageTextToText, AutoProcessor
322
-
323
- MODEL_ID = "169Pi/Alpie_seed"
324
-
325
- processor = AutoProcessor.from_pretrained(MODEL_ID)
326
- model = AutoModelForImageTextToText.from_pretrained(
327
- MODEL_ID,
328
- torch_dtype="auto",
329
- device_map="auto",
330
- )
331
- model.eval()
332
-
333
- messages = [
334
- {
335
- "role": "user",
336
- "content": [
337
- {"type": "text", "text": "Explain why the sky appears blue in two sentences."}
338
- ],
339
- }
340
- ]
341
-
342
- inputs = processor.apply_chat_template(
343
- messages,
344
- tokenize=True,
345
- add_generation_prompt=True,
346
- enable_thinking=False,
347
- return_dict=True,
348
- return_tensors="pt",
349
- ).to(model.device)
350
-
351
- with torch.inference_mode():
352
- output_ids = model.generate(
353
- **inputs,
354
- max_new_tokens=512,
355
- do_sample=True,
356
- temperature=0.7,
357
- top_p=0.8,
358
- )
359
-
360
- new_tokens = output_ids[0, inputs["input_ids"].shape[1]:]
361
- response = processor.decode(new_tokens, skip_special_tokens=True)
362
- print(response)
363
- ```
364
-
365
- #### 3. Tool calling with Transformers
366
-
367
- Transformers can format the available tool definitions into the prompt. The model generates a tool-call request; your application must parse the request, execute the corresponding function, append the tool result, and ask the model to continue. This example demonstrates generating and inspecting a tool call; it does not execute external tools automatically.
368
-
369
- ```python
370
- import json
371
- import torch
372
- from transformers import AutoModelForImageTextToText, AutoProcessor
373
-
374
- MODEL_ID = "169Pi/Alpie_seed"
375
-
376
- processor = AutoProcessor.from_pretrained(MODEL_ID)
377
- model = AutoModelForImageTextToText.from_pretrained(
378
- MODEL_ID,
379
- torch_dtype="auto",
380
- device_map="auto",
381
- )
382
- model.eval()
383
-
384
- tools = [
385
- {
386
- "type": "function",
387
- "function": {
388
- "name": "get_weather",
389
- "description": "Get the current weather for a city.",
390
- "parameters": {
391
- "type": "object",
392
- "properties": {
393
- "city": {
394
- "type": "string",
395
- "description": "The city to check.",
396
- }
397
- },
398
- "required": ["city"],
399
- },
400
- },
401
- }
402
- ]
403
-
404
- messages = [
405
- {
406
- "role": "user",
407
- "content": [
408
- {"type": "text", "text": "What is the weather like in Patna?"}
409
- ],
410
- }
411
- ]
412
-
413
- inputs = processor.apply_chat_template(
414
- messages,
415
- tools=tools,
416
- tokenize=True,
417
- add_generation_prompt=True,
418
- enable_thinking=True,
419
- return_dict=True,
420
- return_tensors="pt",
421
- ).to(model.device)
422
-
423
- with torch.inference_mode():
424
- output_ids = model.generate(
425
- **inputs,
426
- max_new_tokens=1024,
427
- do_sample=True,
428
- temperature=0.7,
429
- top_p=0.8,
430
- )
431
-
432
- new_tokens = output_ids[0, inputs["input_ids"].shape[1]:]
433
- generated_text = processor.decode(new_tokens, skip_special_tokens=False)
434
- print("Model output:")
435
- print(generated_text)
436
-
437
- # Tool execution is application-side. Inspect the generated tool-call
438
- # structure, validate its arguments, and dispatch only to tools you trust.
439
- ```
440
-
441
- For image or video inputs, provide the corresponding image/video content items in the user message and use the same `AutoProcessor` + `AutoModelForImageTextToText` loading pattern. See the upstream Qwen3.5 documentation for multimodal message examples and framework-specific details.
442
-
443
- ## Citation
444
-
445
- If you use Alpie Seed 9B in your work, please cite:
446
-
447
- ```bibtex
448
- @misc{alpie_seed_9b_2026,
449
- title = {Alpie Seed 9B},
450
- author = {{169Pi AI}},
451
- year = {2026},
452
- howpublished = {\url{https://huggingface.co/169Pi/Alpie_seed}}
453
- }
454
- ```
 
106
  >
107
  > Alpie Seed scores are from the FP16 model and do not apply to the 2-bit, 3-bit, or 4-bit variants. **Bold** indicates the best score in each row. Comparison-model scores are taken from published model cards and reports (including the [Qwen3.5-9B model card](https://huggingface.co/Qwen/Qwen3.5-9B)).
108
 
109
+ ### 🧮 Language
110
+
111
+ <div style="background-color: #FAF5FF; border: 1.5px solid #C4B5FD; border-radius: 14px; padding: 12px; margin-bottom: 24px; box-shadow: 0 4px 12px rgba(88, 37, 179, 0.05); overflow-x: auto;">
112
+ <table style="width: 100%; border-collapse: separate; border-spacing: 0; text-align: center; font-size: 13.5px; font-family: inherit;">
113
+ <thead>
114
+ <tr style="color: #1E1B4B; font-weight: 700; background-color: #EDE9FE;">
115
+ <th style="text-align: left; padding: 10px 14px; border-top-left-radius: 8px; border-bottom: 1.5px solid #C4B5FD;">Benchmark</th>
116
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Qwen3.5<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">9B</span></th>
117
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Gemma 4<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">12B</span></th>
118
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Ministral 3<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">8B</span></th>
119
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">GPT-OSS<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">20B</span></th>
120
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Sarvam<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">30B</span></th>
121
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Nemotron 3.5<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">Lightning</span></th>
122
+ <th style="padding: 10px 10px; background-color: #5825B3; color: #FFFFFF; border-top-right-radius: 8px; font-weight: 800; border-bottom: 1.5px solid #431A9E;">Alpie Seed<br><span style="font-size: 12px; color: #DDD6FE;">9B</span></th>
123
+ </tr>
124
+ </thead>
125
+ <tbody>
126
+ <!-- Section: Knowledge & STEM -->
127
+ <tr style="background-color: #E0E7FF; color: #3730A3; font-weight: 800; text-align: left;">
128
+ <td colspan="8" style="padding: 8px 14px; font-size: 13px; letter-spacing: 0.3px;">🧠 Knowledge &amp; STEM</td>
129
+ </tr>
130
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
131
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">MMLU</td>
132
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
133
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
134
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">76.1*</td>
135
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">85.3</td>
136
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">85.1</td>
137
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">78.63*</td>
138
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">87.30</td>
139
+ </tr>
140
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FAFAFE;">
141
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">MMLU-Pro</td>
142
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">82.5</td>
143
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">77.2</td>
144
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">75.00</td>
145
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">77.2</td>
146
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">80.0</td>
147
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">81.94</td>
148
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">82.54</td>
149
+ </tr>
150
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
151
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">GPQA Diamond</td>
152
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">81.7</td>
153
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">78.8</td>
154
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">38.6</td>
155
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">71.5</td>
156
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">66.5</td>
157
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">75.44</td>
158
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">84.32</td>
159
+ </tr>
160
+ <!-- Section: Mathematics -->
161
+ <tr style="background-color: #E0E7FF; color: #3730A3; font-weight: 800; text-align: left;">
162
+ <td colspan="8" style="padding: 8px 14px; font-size: 13px; letter-spacing: 0.3px;">📐 Mathematics</td>
163
+ </tr>
164
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
165
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">AIME 2026</td>
166
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">69.4</td>
167
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">77.5</td>
168
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
169
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
170
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
171
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
172
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">83.33</td>
173
+ </tr>
174
+ <!-- Section: Coding & Software Engineering -->
175
+ <tr style="background-color: #E0E7FF; color: #3730A3; font-weight: 800; text-align: left;">
176
+ <td colspan="8" style="padding: 8px 14px; font-size: 13px; letter-spacing: 0.3px;">💻 Coding &amp; Software Engineering</td>
177
+ </tr>
178
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
179
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">LiveCodeBench v6</td>
180
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">65.6</td>
181
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">72.0</td>
182
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">61.6</td>
183
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">61.0</td>
184
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">70.0</td>
185
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
186
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">72.99</td>
187
+ </tr>
188
+ <tr style="background-color: #FAFAFE;">
189
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom-left-radius: 8px;">SWE-Bench Verified</td>
190
+ <td style="padding: 9px 8px; color: #1E1B4B;">60.0</td>
191
+ <td style="padding: 9px 8px; color: #1E1B4B;">52.0</td>
192
+ <td style="padding: 9px 8px; color: #64748B;">--</td>
193
+ <td style="padding: 9px 8px; color: #1E1B4B;">60.7</td>
194
+ <td style="padding: 9px 8px; color: #1E1B4B;">34.0</td>
195
+ <td style="padding: 9px 8px; color: #1E1B4B;">51.56</td>
196
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom-right-radius: 8px; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">61.20</td>
197
+ </tr>
198
+ </tbody>
199
+ </table>
200
+ </div>
201
 
202
+ ### 👁️ Vision Language
203
+
204
+ <div style="background-color: #FAF5FF; border: 1.5px solid #C4B5FD; border-radius: 14px; padding: 12px; margin-bottom: 24px; box-shadow: 0 4px 12px rgba(88, 37, 179, 0.05); overflow-x: auto;">
205
+ <table style="width: 100%; border-collapse: separate; border-spacing: 0; text-align: center; font-size: 13.5px; font-family: inherit;">
206
+ <thead>
207
+ <tr style="color: #1E1B4B; font-weight: 700; background-color: #EDE9FE;">
208
+ <th style="text-align: left; padding: 10px 14px; border-top-left-radius: 8px; border-bottom: 1.5px solid #C4B5FD;">Benchmark</th>
209
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">GPT-5-Nano<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">2025-08-07</span></th>
210
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Gemini-2.5<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">Flash-Lite</span></th>
211
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Qwen3-VL<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">30B-A3B</span></th>
212
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Qwen3.5<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">9B</span></th>
213
+ <th style="padding: 10px 8px; border-bottom: 1.5px solid #C4B5FD;">Gemma 4<br><span style="font-size: 12px; font-weight: normal; color: #4338CA;">12B</span></th>
214
+ <th style="padding: 10px 10px; background-color: #5825B3; color: #FFFFFF; border-top-right-radius: 8px; font-weight: 800; border-bottom: 1.5px solid #431A9E;">Alpie Seed<br><span style="font-size: 12px; color: #DDD6FE;">9B</span></th>
215
+ </tr>
216
+ </thead>
217
+ <tbody>
218
+ <!-- Section: STEM and Puzzle -->
219
+ <tr style="background-color: #E0E7FF; color: #3730A3; font-weight: 800; text-align: left;">
220
+ <td colspan="7" style="padding: 8px 14px; font-size: 13px; letter-spacing: 0.3px;">🧩 STEM and Puzzle</td>
221
+ </tr>
222
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
223
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">MMMU-Pro</td>
224
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">57.2</td>
225
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">59.7</td>
226
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">63.0</td>
227
+ <td style="padding: 9px 8px; color: #1E1B4B; font-weight: 800; border-bottom: 1px solid #EDE9FE;">70.1</td>
228
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">69.1</td>
229
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">68.70</td>
230
+ </tr>
231
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FAFAFE;">
232
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">MathVision</td>
233
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">62.2</td>
234
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">52.1</td>
235
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">65.7</td>
236
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">78.9</td>
237
+ <td style="padding: 9px 8px; color: #1E1B4B; font-weight: 800; border-bottom: 1px solid #EDE9FE;">79.7</td>
238
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">76.76</td>
239
+ </tr>
240
+ <!-- Section: General VQA -->
241
+ <tr style="background-color: #E0E7FF; color: #3730A3; font-weight: 800; text-align: left;">
242
+ <td colspan="7" style="padding: 8px 14px; font-size: 13px; letter-spacing: 0.3px;">👁️ General VQA</td>
243
+ </tr>
244
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
245
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">RealWorldQA</td>
246
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">71.8</td>
247
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">72.2</td>
248
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">77.4</td>
249
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">80.3</td>
250
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
251
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">80.39</td>
252
+ </tr>
253
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FAFAFE;">
254
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">MMStar</td>
255
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">68.6</td>
256
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">69.1</td>
257
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">75.5</td>
258
+ <td style="padding: 9px 8px; color: #1E1B4B; font-weight: 800; border-bottom: 1px solid #EDE9FE;">79.7</td>
259
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
260
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">78.87</td>
261
+ </tr>
262
+ <tr style="border-bottom: 1px solid #DDD6FE; background-color: #FFFFFF;">
263
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">HallusionBench</td>
264
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">58.4</td>
265
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">64.5</td>
266
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">66.0</td>
267
+ <td style="padding: 9px 8px; color: #1E1B4B; border-bottom: 1px solid #EDE9FE;">69.3</td>
268
+ <td style="padding: 9px 8px; color: #64748B; border-bottom: 1px solid #EDE9FE;">--</td>
269
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom: 1px solid #DDD6FE; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">79.88</td>
270
+ </tr>
271
+ <!-- Section: Text Recognition and Document Understanding -->
272
+ <tr style="background-color: #E0E7FF; color: #3730A3; font-weight: 800; text-align: left;">
273
+ <td colspan="7" style="padding: 8px 14px; font-size: 13px; letter-spacing: 0.3px;">📄 Text Recognition &amp; Document Understanding</td>
274
+ </tr>
275
+ <tr style="background-color: #FAFAFE;">
276
+ <td style="text-align: left; padding: 9px 14px; font-weight: 600; color: #1E1B4B; border-bottom-left-radius: 8px;">OCRBench</td>
277
+ <td style="padding: 9px 8px; color: #1E1B4B;">75.3</td>
278
+ <td style="padding: 9px 8px; color: #1E1B4B;">82.5</td>
279
+ <td style="padding: 9px 8px; color: #1E1B4B;">83.9</td>
280
+ <td style="padding: 9px 8px; color: #1E1B4B;">89.2</td>
281
+ <td style="padding: 9px 8px; color: #64748B;">--</td>
282
+ <td style="padding: 9px 8px; background-color: #F3E8FF; color: #581C87; font-weight: 800; border-bottom-right-radius: 8px; border-left: 1.5px solid #C4B5FD; border-right: 1.5px solid #C4B5FD;">90.30</td>
283
+ </tr>
284
+ </tbody>
285
+ </table>
286
+ </div>
287
 
288
  - SWE-Bench Verified: resolved rate using Mini-SWE-agent, with retries disabled.
289
  - Comparison-model scores were not re-run by 169Pi, and harnesses, prompts, and benchmark versions differ, so cross-model comparisons are indicative rather than strictly like-for-like. Scores vary with dataset versions, prompts, evaluation harnesses, and inference settings.
 
299
  Qwen3.5 support requires a recent Transformers release (Transformers 5 or later).
300
 
301
  ```bash
302
+ pip install -U "transformers>=5.2.0" torch torchvision pillow accelerate