Hugging Face's logo

Skywork
/
Skywork-R1V2-38B

Image-Text-to-Text
Transformers
Safetensors
internvl_chat
conversational
custom_code
Model card Files
xet
Community
6
OrlandoHugBot commited on
Commit
896a5d4
·
verified ·
1 Parent(s): 834e577

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +295 -3
README.md CHANGED
@@ -1,3 +1,295 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-text-to-text
3
+ library_name: transformers
4
+ license: mit
5
+ ---
6
+
7
+ # Skywork-R1V
8
+
9
+ <div align="center">
10
+ <img src="skywork-logo.png" alt="Introduction Image" width="500" height="400">
11
+ </div>
12
+
13
+ ## 📖 [Technical Report](https://github.com/SkyworkAI/Skywork-R1V/blob/main/Skywork_R1V.pdf) | 💻 [GitHub](https://github.com/SkyworkAI/Skywork-R1V) | 🌐 [ModelScope](https://modelscope.cn/models/Skywork/Skywork-R1V-38B)
14
+
15
+ <div align="center">
16
+
17
+ [![GitHub Stars](https://img.shields.io/github/stars/SkyworkAI/Skywork-R1V)](https://github.com/SkyworkAI/Skywork-R1V/stargazers) [![GitHub Forks](https://img.shields.io/github/forks/SkyworkAI/Skywork-R1V)](https://github.com/SkyworkAI/Skywork-R1V/fork)
18
+
19
+ </div>
20
+
21
+
22
+ ## 1. Model Introduction
23
+ | Model Name | Vision Encoder | Language Model | HF Link |
24
+ | ---------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------ |
25
+ | Skywork-R1V-38B | [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5) | [deepseek-ai/DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) | [🤗 Link](https://huggingface.co/Skywork/Skywork-R1V-38B) |
26
+ | Skywork-R1V-38B-qwq | [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5) | [Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B) | [🤗 Link](https://huggingface.co/Skywork/Skywork-R1V2-38B) |
27
+
28
+
29
+ ## 2. Feature
30
+ - **Visual Chain-of-Thought**: Enables multi-step logical reasoning on visual inputs, breaking down complex image-based problems into manageable steps.
31
+ - **Mathematical & Scientific Analysis**: Capable of solving visual math problems and interpreting scientific/medical imagery with high precision.
32
+ - **Cross-Modal Understanding**: Seamlessly integrates text and images for richer, context-aware comprehension.
33
+
34
+
35
+ ## 3. Evaluation
36
+ <br>
37
+ <br>
38
+ <div align="center">
39
+ <b>Comparison with Larger-Scale Open-Source and Closed-Source Models</b>
40
+ </div>
41
+
42
+ <table align="center">
43
+ <thead>
44
+ <tr>
45
+ <th></th>
46
+ <th align="center"><strong>Benchmark</strong></th>
47
+ <th align="center"><strong>LLM</strong></th>
48
+ <th align="center" colspan="4"><strong>VLM</strong></th>
49
+ </tr>
50
+ <tr>
51
+ <th></th>
52
+ <th></th>
53
+ <th align="center"><strong>QwQ-32B-Preview</strong></th>
54
+ <th align="center"><strong>InternVL-2.5-38B</strong></th>
55
+ <th align="center"><strong>VILA 1.5-40B</strong></th>
56
+ <th align="center"><strong>InternVL2-40B</strong></th>
57
+ <th align="center"><strong>Skywork-R1V-38B</strong></th>
58
+ </tr>
59
+ </thead>
60
+ <tbody>
61
+ <tr>
62
+ <td rowspan="3">Reasoning</td>
63
+ <td>MATH-500</td>
64
+ <td align="center">90.6</td>
65
+ <td align="center">-</td>
66
+ <td align="center">-</td>
67
+ <td align="center">-</td>
68
+ <td align="center"><strong>94.0</strong></td>
69
+ </tr>
70
+ <tr>
71
+ <td>AIME 2024</td>
72
+ <td align="center">50.0</td>
73
+ <td align="center">-</td>
74
+ <td align="center">-</td>
75
+ <td align="center">-</td>
76
+ <td align="center"><strong>72.0</strong></td>
77
+ </tr>
78
+ <tr>
79
+ <td>GPQA</td>
80
+ <td align="center">54.5</td>
81
+ <td align="center">-</td>
82
+ <td align="center">-</td>
83
+ <td align="center">-</td>
84
+ <td align="center"><strong>61.6</strong></td>
85
+ </tr>
86
+ <tr>
87
+ <td rowspan="3">Vision</td>
88
+ <td>MathVista(mini)</td>
89
+ <td align="center">-</td>
90
+ <td align="center">71.9</td>
91
+ <td align="center">49.5</td>
92
+ <td align="center">63.7</td>
93
+ <td align="center">67.5</td>
94
+ </tr>
95
+ <tr>
96
+ <td>MMMU(Val)</td>
97
+ <td align="center">-</td>
98
+ <td align="center">63.9</td>
99
+ <td align="center">55.1</td>
100
+ <td align="center">55.2</td>
101
+ <td align="center"><strong>69.0</strong></td>
102
+ </tr>
103
+ </tbody>
104
+ </table>
105
+
106
+
107
+ <br>
108
+ <br>
109
+ <div align="center">
110
+ <b>Evaluation results of state-of-the-art LLMs and VLMs</b>
111
+ </div>
112
+ <table>
113
+ <thead>
114
+ <tr>
115
+ <th></th>
116
+ <th align="center"><strong>Vision</strong></th>
117
+ <th align="center" colspan="3"><strong>Reasoning</strong></th>
118
+ <th align="center" colspan="3"><strong>Vision</strong></th>
119
+ </tr>
120
+ <tr>
121
+ <th></th>
122
+ <th></th>
123
+ <th align="center"><strong>MATH-500</strong></th>
124
+ <th align="center"><strong>AIME 2024</strong></th>
125
+ <th align="center"><strong>GPQA</strong></th>
126
+ <th align="center"><strong>MathVista(mini)</strong></th>
127
+ <th align="center"><strong>MMMU(Val)</strong></th>
128
+ </tr>
129
+ <tr>
130
+ <th></th>
131
+ <th></th>
132
+ <th align="center">pass@1</th>
133
+ <th align="center">pass@1</th>
134
+ <th align="center">pass@1</th>
135
+ <th align="center">pass@1</th>
136
+ <th align="center">pass@1</th>
137
+ </tr>
138
+ </thead>
139
+ <tbody>
140
+ <tr>
141
+ <td>Qwen2.5-72B-Instruct</td>
142
+ <td align="center">❌</td>
143
+ <td align="center">80.0</td>
144
+ <td align="center">23.3</td>
145
+ <td align="center">49.0</td>
146
+ <td align="center">-</td>
147
+ <td align="center">-</td>
148
+ </tr>
149
+ <tr>
150
+ <td>Deepseek V3</td>
151
+ <td align="center">❌</td>
152
+ <td align="center">90.2</td>
153
+ <td align="center">39.2</td>
154
+ <td align="center">59.1</td>
155
+ <td align="center">-</td>
156
+ <td align="center">-</td>
157
+ </tr>
158
+ <tr>
159
+ <td>Deepseek R1</td>
160
+ <td align="center">❌</td>
161
+ <td align="center">97.3</td>
162
+ <td align="center">79.8</td>
163
+ <td align="center">71.5</td>
164
+ <td align="center">-</td>
165
+ <td align="center">-</td>
166
+ </tr>
167
+ <tr>
168
+ <td>Claude 3.5 Sonnet</td>
169
+ <td align="center">✅</td>
170
+ <td align="center">78.3</td>
171
+ <td align="center">16.0</td>
172
+ <td align="center">65.0</td>
173
+ <td align="center">65.3</td>
174
+ <td align="center">66.4</td>
175
+ </tr>
176
+ <tr>
177
+ <td>GPT-4o</td>
178
+ <td align="center">✅</td>
179
+ <td align="center">74.6</td>
180
+ <td align="center">9.3</td>
181
+ <td align="center">49.9</td>
182
+ <td align="center">63.8</td>
183
+ <td align="center">69.1</td>
184
+ </tr>
185
+ <tr>
186
+ <td>Kimi k1.5</td>
187
+ <td align="center">✅</td>
188
+ <td align="center">96.2</td>
189
+ <td align="center">77.5</td>
190
+ <td align="center">-</td>
191
+ <td align="center">74.9</td>
192
+ <td align="center">70.0</td>
193
+ </tr>
194
+ <tr>
195
+ <td>Qwen2.5-VL-72B-Instruct</td>
196
+ <td align="center">✅</td>
197
+ <td align="center">-</td>
198
+ <td align="center">-</td>
199
+ <td align="center">-</td>
200
+ <td align="center">74.8</td>
201
+ <td align="center">70.2</td>
202
+ </tr>
203
+ <tr>
204
+ <td>LLaVA-Onevision-72B</td>
205
+ <td align="center">✅</td>
206
+ <td align="center">-</td>
207
+ <td align="center">-</td>
208
+ <td align="center">-</td>
209
+ <td align="center">67.5</td>
210
+ <td align="center">56.8</td>
211
+ </tr>
212
+ <tr>
213
+ <td>InternVL2-Llama3-76B</td>
214
+ <td align="center">✅</td>
215
+ <td align="center">-</td>
216
+ <td align="center">-</td>
217
+ <td align="center">-</td>
218
+ <td align="center">65.5</td>
219
+ <td align="center">62.7</td>
220
+ </tr>
221
+ <tr>
222
+ <td>InternVL2.5-78B</td>
223
+ <td align="center">✅</td>
224
+ <td align="center">-</td>
225
+ <td align="center">-</td>
226
+ <td align="center">-</td>
227
+ <td align="center">72.3</td>
228
+ <td align="center">70.1</td>
229
+ </tr>
230
+ <tr>
231
+ <td>Skywork-R1V-38B</td>
232
+ <td align="center">✅</td>
233
+ <td align="center">94.0</td>
234
+ <td align="center">72.0</td>
235
+ <td align="center">61.6</td>
236
+ <td align="center">67.5</td>
237
+ <td align="center">69.0</td>
238
+ </tr>
239
+ </tbody>
240
+ </table>
241
+
242
+ <div align="center">
243
+ <img src="eval.jpeg" width="90%" height="90%" alt="skywork_r1v_eval" />
244
+ </div>
245
+
246
+ ---
247
+
248
+
249
+ ## 4. Usage
250
+
251
+ ### 1. Clone the Repository
252
+
253
+ ```shell
254
+ git clone https://github.com/SkyworkAI/Skywork-R1V.git
255
+ cd skywork-r1v/inference
256
+ ```
257
+ ### 2. Set Up the Environment
258
+
259
+ ```shell
260
+ conda create -n r1-v python=3.10
261
+ conda activate r1-v
262
+ bash setup.sh
263
+ ```
264
+
265
+ ### 3. Run the Inference Script
266
+
267
+ ```shell
268
+ CUDA_VISIBLE_DEVICES="0,1" python inference_with_transformers.py \
269
+ --model_path path \
270
+ --image_paths image1_path \
271
+ --question "your question"
272
+ ```
273
+
274
+ ---
275
+
276
+ ## 5. Citation
277
+ If you use Skywork-R1V in your research, please cite:
278
+
279
+ ```
280
+ @misc{peng2025skyworkr1vpioneeringmultimodal,
281
+ title={Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought},
282
+ author={Yi Peng and Chris and Xiaokun Wang and Yichen Wei and Jiangbo Pei and Weijie Qiu and Ai Jian and Yunzhuo Hao and Jiachun Pan and Tianyidan Xie and Li Ge and Rongxian Zhuang and Xuchen Song and Yang Liu and Yahui Zhou},
283
+ year={2025},
284
+ eprint={2504.05599},
285
+ archivePrefix={arXiv},
286
+ primaryClass={cs.CV},
287
+ url={https://arxiv.org/abs/2504.05599},
288
+ }
289
+ ```
290
+
291
+ *This project is released under an open-source license.*
292
+
293
+ ## Star History
294
+ [![Star History Chart](https://api.star-history.com/svg?repos=SkyworkAI/Skywork-R1V&type=Date)](https://www.star-history.com/#SkyworkAI/Skywork-R1V&Date)
295
+ ```