| | --- |
| | pipeline_tag: image-text-to-text |
| | library_name: transformers |
| | license: mit |
| | --- |
| | |
| | # Skywork-R1V2 |
| |
|
| | <div align="center"> |
| | <img src="skywork-logo.png" alt="Introduction Image" width="500" height="400"> |
| | </div> |
| |
|
| | ## 📖 [Technical Report](https://github.com/SkyworkAI/Skywork-R1V/blob/main/Skywork_R1V.pdf) | 💻 [GitHub](https://github.com/SkyworkAI/Skywork-R1V) | 🌐 [ModelScope](https://modelscope.cn/models/Skywork/Skywork-R1V-38B) |
| |
|
| | <div align="center"> |
| |
|
| | [](https://github.com/SkyworkAI/Skywork-R1V/stargazers) [](https://github.com/SkyworkAI/Skywork-R1V/fork) |
| |
|
| | </div> |
| |
|
| |
|
| | ## 1. Model Introduction |
| | | Model Name | Vision Encoder | Language Model | HF Link | |
| | | ---------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------ | |
| | | Skywork-R1V2-38B | [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5) | [Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B) | [🤗 Link](https://huggingface.co/Skywork/Skywork-R1V2-38B) | |
| |
|
| |
|
| |
|
| | ## 2. Evaluation |
| |
|
| |
|
| | <div align="center"> |
| | <img src="open_source.png" width="100%" height="100%" alt="skywork_r1v2_eval_open" /> |
| | </div> |
| | <div align="center"> |
| | <b>Comparison with Larger-Scale Open-Source Models</b> |
| | </div> |
| |
|
| |
|
| |
|
| |
|
| | <div align="center"> |
| | <img src="properitary.png" width="100%" height="100%" alt="skywork_r1v2_eval_properitary" /> |
| | </div> |
| | <div align="center"> |
| | <b>Comparison with Properitary Models</b> |
| | </div> |
| |
|
| |
|
| |
|
| | <table> |
| | <thead> |
| | <tr> |
| | <th>Model</th> |
| | <th align="center"><strong>Supports Vision</strong></th> |
| | <th align="center" colspan="8"><strong>Text Reasoning (%)</strong></th> |
| | <th align="center" colspan="5"><strong>Multimodal Reasoning (%)</strong></th> |
| | </tr> |
| | <tr> |
| | <th></th> |
| | <th></th> |
| | <th align="center">AIME24</th> |
| | <th align="center">LiveCodebench</th> |
| | <th align="center">liveBench</th> |
| | <th align="center">IFEVAL</th> |
| | <th align="center">BFCL</th> |
| | <th align="center">MATH‑500</th> |
| | <th align="center">AIME 2024</th> |
| | <th align="center">GPQA</th> |
| | <th align="center">MMMU(val)</th> |
| | <th align="center">MathVista(mini)</th> |
| | <th align="center">MathVision(mini)</th> |
| | <th align="center">OlympiadBench</th> |
| | <th align="center">mmmu‑pro</th> |
| | </tr> |
| | <tr> |
| | <th></th> |
| | <th></th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">pass@1</th> |
| | <th align="center">pass@1</th> |
| | <th align="center">pass@1</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | <th align="center">%</th> |
| | </tr> |
| | </thead> |
| | <tbody> |
| | <tr> |
| | <td>R1V2‑38B</td> |
| | <td align="center">✅</td> |
| | <td align="center">78.9</td> |
| | <td align="center">63.6</td> |
| | <td align="center">73.2</td> |
| | <td align="center">82.9</td> |
| | <td align="center">66.3</td> |
| | <td align="center">94.0</td> |
| | <td align="center">72.0</td> |
| | <td align="center">61.6</td> |
| | <td align="center">73.6</td> |
| | <td align="center">74.0</td> |
| | <td align="center">49.0</td> |
| | <td align="center">62.6</td> |
| | <td align="center">52.0</td> |
| | </tr> |
| | <tr> |
| | <td>R1V1‑38B</td> |
| | <td align="center">✅</td> |
| | <td align="center">72.0</td> |
| | <td align="center">57.2</td> |
| | <td align="center">54.6</td> |
| | <td align="center">72.5</td> |
| | <td align="center">53.5</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">68.0</td> |
| | <td align="center">67.0</td> |
| | <td align="center">–</td> |
| | <td align="center">40.4</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>Deepseek‑R1‑671B</td> |
| | <td align="center">❌</td> |
| | <td align="center">74.3</td> |
| | <td align="center">65.9</td> |
| | <td align="center">71.6</td> |
| | <td align="center">83.3</td> |
| | <td align="center">60.3</td> |
| | <td align="center">97.3</td> |
| | <td align="center">79.8</td> |
| | <td align="center">71.5</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>GPT‑o1</td> |
| | <td align="center">❌</td> |
| | <td align="center">79.8</td> |
| | <td align="center">63.4</td> |
| | <td align="center">72.2</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>GPT‑o4‑mini</td> |
| | <td align="center">✅</td> |
| | <td align="center">93.4</td> |
| | <td align="center">74.6</td> |
| | <td align="center">78.1</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">74.6</td> |
| | <td align="center">9.3</td> |
| | <td align="center">49.9</td> |
| | <td align="center">81.6</td> |
| | <td align="center">84.3</td> |
| | <td align="center">58.0</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>Claude 3.5 Sonnet</td> |
| | <td align="center">✅</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">78.3</td> |
| | <td align="center">16.0</td> |
| | <td align="center">65.0</td> |
| | <td align="center">66.4</td> |
| | <td align="center">65.3</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>Kimi k1.5 longcot</td> |
| | <td align="center">✅</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">96.2</td> |
| | <td align="center">77.5</td> |
| | <td align="center">–</td> |
| | <td align="center">70.0</td> |
| | <td align="center">74.9</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>Qwen2.5‑VL‑72B‑Instruct</td> |
| | <td align="center">✅</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">70.2</td> |
| | <td align="center">74.8</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | </tr> |
| | <tr> |
| | <td>InternVL2.5‑78B</td> |
| | <td align="center">✅</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">–</td> |
| | <td align="center">70.1</td> |
| | <td align="center">72.3</td> |
| | <td align="center">–</td> |
| | <td align="center">33.2</td> |
| | <td align="center">–</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | <div align="center"> |
| | <b>Evaluation results of state-of-the-art LLMs and VLMs</b> |
| | </div> |
| | |
| |
|
| | --- |
| |
|
| |
|
| | ## 3. Usage |
| |
|
| | ### 1. Clone the Repository |
| |
|
| | ```shell |
| | git clone https://github.com/SkyworkAI/Skywork-R1V.git |
| | cd skywork-r1v/inference |
| | ``` |
| | ### 2. Set Up the Environment |
| |
|
| | ```shell |
| | conda create -n r1-v python=3.10 |
| | conda activate r1-v |
| | bash setup.sh |
| | ``` |
| |
|
| | ### 3. Run the Inference Script |
| |
|
| | ```shell |
| | CUDA_VISIBLE_DEVICES="0,1" python inference_with_transformers.py \ |
| | --model_path path \ |
| | --image_paths image1_path \ |
| | --question "your question" |
| | ``` |
| |
|
| | --- |
| |
|
| | ## 4. Citation |
| | If you use Skywork-R1V in your research, please cite: |
| |
|
| | ``` |
| | @misc{peng2025skyworkr1vpioneeringmultimodal, |
| | title={Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought}, |
| | author={Yi Peng and Chris and Xiaokun Wang and Yichen Wei and Jiangbo Pei and Weijie Qiu and Ai Jian and Yunzhuo Hao and Jiachun Pan and Tianyidan Xie and Li Ge and Rongxian Zhuang and Xuchen Song and Yang Liu and Yahui Zhou}, |
| | year={2025}, |
| | eprint={2504.05599}, |
| | archivePrefix={arXiv}, |
| | primaryClass={cs.CV}, |
| | url={https://arxiv.org/abs/2504.05599}, |
| | } |
| | ``` |
| |
|
| | *This project is released under an open-source license.* |
| |
|
| | ## Star History |
| | [](https://www.star-history.com/#SkyworkAI/Skywork-R1V&Date) |
| | ``` |
