README.md

---
license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
---

# Vision-SR1: Self-Rewarding Vision-Language Model via Reasoning Decomposition

This repository hosts the **Vision-SR1-7B** model, a self-rewarding Vision-Language Model (VLM) for improved visual reasoning, as presented in the paper [Self-Rewarding Vision-Language Model via Reasoning Decomposition](https://huggingface.co/papers/2508.19652).

📖 [Paper](https://huggingface.co/papers/2508.19652) | 💻 [Code](https://github.com/zli12321/Vision-SR1)

## About Vision-SR1

Vision-SR1 is a novel self-rewarding method designed to enhance visual reasoning in Vision-Language Models (VLMs) through a reinforcement learning framework, without relying on external visual supervisions. Traditional VLMs often struggle with visual hallucinations and language shortcuts due to sparse visual signals, tending to prioritize language over visual perception.

Vision-SR1 addresses these limitations by decomposing VLM reasoning into two distinct stages: visual perception and language reasoning. The model is first prompted to generate self-contained visual perceptions that are sufficient to answer a given question without referring back to the input image. Subsequently, the same VLM model is re-prompted to perform language reasoning using only the generated perception as input, which then computes a self-reward. This self-reward, combined with supervision on the final outputs, delivers a balanced training signal that strengthens both visual perception and language reasoning. Experimental results demonstrate that Vision-SR1 effectively improves visual reasoning, mitigates visual hallucinations, and reduces reliance on language shortcuts across various vision-language tasks.

The framework for Vision-SR1 is illustrated below:

<p align="center">
    <img src="https://github.com/zli12321/Vision-SR1/raw/main/assets/method.png" width="80%">
</p>

### Datasets

The training dataset for Vision-SR1 is compiled from 23 diverse sources, evenly distributed across three primary areas: general visual understanding, science knowledge, and multimodal mathematical reasoning.

<p align="center">
    <img src="https://github.com/zli12321/Vision-SR1/raw/main/assets/data.png" width="80%">
</p>

Related models and datasets can be found on the Hugging Face Hub:

*   **Models:**
    *   🤗 [Vision-SR1-7B](https://huggingface.co/LMMs-Lab-Turtle/SelfRewarded-R1-7B)
    *   🤗 [Vision-SR1-7B-Cold-Start](https://huggingface.co/LMMs-Lab-Turtle/Qwen-2.5VL-7B-Cold-Start)
    *   🤗 [Vision-SR1-3B-Cold-Start](https://huggingface.co/LMMs-Lab-Turtle/Qwen-2.5VL-3B-Cold-Start)
*   **Datasets:**
    *   📊 [Vision-SR1-Cold-Start-9K](https://huggingface.co/datasets/LMMs-Lab-Turtle/Vision-SR1-Cold-9K)
    *   📊 [Vision-SR1-47K](https://huggingface.co/datasets/LMMs-Lab-Turtle/Vision-SR1-47K)

## Installation and Training

The codebase for Vision-SR1 is adapted from [verl](https://github.com/volcengine/verl) and [EasyR1](https://github.com/hiyouga/EasyR1), and requires `transformers=4.49.0`.

To set up the environment and for detailed training instructions, please refer to the [official GitHub repository](https://github.com/zli12321/Vision-SR1).

### Software Requirements

*   Python 3.9+
*   transformers=4.49.0

### RL Training Setup

```bash
git clone https://github.com/zli12321/Vision-SR1.git
cd Vision-SR1
conda create -n Vision-SR1 python=3.11
bash setup.sh
```

### GRPO Training

```bash
### Self-Reward Vision-SR1 GRPO Training
bash ./train_examples/2-7b_selfReward_train.sh

### Vision-SR1 regular training
bash ./train_examples/1-7b_visionR1_train.sh
```

### Supervised Finetuning

The supervised finetuning code is adopted from [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory).

#### Setup
```bash
conda create -n SFT python=3.11
cd LLaMA-Factory-Cold-Start
pip install -e ".[torch,metrics]" --no-build-isolation

pip install --upgrade huggingface_hub
huggingface-cli login
```

#### Training
```bash
FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/Vision-SR1-Cold-Start.yaml
```

## Citation

If you find our work helpful, please cite the paper:

**Self-Rewarding Vision-Language Model via Reasoning Decomposition**
[https://huggingface.co/papers/2508.19652](https://huggingface.co/papers/2508.19652)

*(Note: The BibTeX citation for `luo2024semievol` provided in the original GitHub README appears to be for a different paper. Please ensure to update your local citation with the correct BibTeX for "Self-Rewarding Vision-Language Model via Reasoning Decomposition".)*

We also recommend citing the source code work, `EasyR1`, from which parts of this project were adapted:

```bibtex
@misc{zheng2025easyr1,
  title        = {EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework},
  author       = {Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, Yuwen Xiong},
  howpublished = {\url{https://github.com/hiyouga/EasyR1}},
  year         = {2025}
}
```