license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
Vision-SR1: Self-Rewarding Vision-Language Model via Reasoning Decomposition
This repository hosts the Vision-SR1-7B model, a self-rewarding Vision-Language Model (VLM) for improved visual reasoning, as presented in the paper Self-Rewarding Vision-Language Model via Reasoning Decomposition.
About Vision-SR1
Vision-SR1 is a novel self-rewarding method designed to enhance visual reasoning in Vision-Language Models (VLMs) through a reinforcement learning framework, without relying on external visual supervisions. Traditional VLMs often struggle with visual hallucinations and language shortcuts due to sparse visual signals, tending to prioritize language over visual perception.
Vision-SR1 addresses these limitations by decomposing VLM reasoning into two distinct stages: visual perception and language reasoning. The model is first prompted to generate self-contained visual perceptions that are sufficient to answer a given question without referring back to the input image. Subsequently, the same VLM model is re-prompted to perform language reasoning using only the generated perception as input, which then computes a self-reward. This self-reward, combined with supervision on the final outputs, delivers a balanced training signal that strengthens both visual perception and language reasoning. Experimental results demonstrate that Vision-SR1 effectively improves visual reasoning, mitigates visual hallucinations, and reduces reliance on language shortcuts across various vision-language tasks.
The framework for Vision-SR1 is illustrated below:
Datasets
The training dataset for Vision-SR1 is compiled from 23 diverse sources, evenly distributed across three primary areas: general visual understanding, science knowledge, and multimodal mathematical reasoning.
Related models and datasets can be found on the Hugging Face Hub:
- Models:
- Datasets:
Installation and Training
The codebase for Vision-SR1 is adapted from verl and EasyR1, and requires transformers=4.49.0.
To set up the environment and for detailed training instructions, please refer to the official GitHub repository.
Software Requirements
- Python 3.9+
- transformers=4.49.0
RL Training Setup
git clone https://github.com/zli12321/Vision-SR1.git
cd Vision-SR1
conda create -n Vision-SR1 python=3.11
bash setup.sh
GRPO Training
### Self-Reward Vision-SR1 GRPO Training
bash ./train_examples/2-7b_selfReward_train.sh
### Vision-SR1 regular training
bash ./train_examples/1-7b_visionR1_train.sh
Supervised Finetuning
The supervised finetuning code is adopted from LLaMA-Factory.
Setup
conda create -n SFT python=3.11
cd LLaMA-Factory-Cold-Start
pip install -e ".[torch,metrics]" --no-build-isolation
pip install --upgrade huggingface_hub
huggingface-cli login
Training
FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/Vision-SR1-Cold-Start.yaml
Citation
If you find our work helpful, please cite the paper:
Self-Rewarding Vision-Language Model via Reasoning Decomposition https://huggingface.co/papers/2508.19652
(Note: The BibTeX citation for luo2024semievol provided in the original GitHub README appears to be for a different paper. Please ensure to update your local citation with the correct BibTeX for "Self-Rewarding Vision-Language Model via Reasoning Decomposition".)
We also recommend citing the source code work, EasyR1, from which parts of this project were adapted:
@misc{zheng2025easyr1,
title = {EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework},
author = {Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, Yuwen Xiong},
howpublished = {\url{https://github.com/hiyouga/EasyR1}},
year = {2025}
}