README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-VL-8B-Instruct
tags:
  - reward-model
  - image-editing
  - reinforcement-learning
  - spatial-reasoning
  - vision-language-model
  - icml2026
datasets:
  - SpatialReward/SpatialReward-Train
pipeline_tag: image-text-to-text
language:
  - en
---

<p align="center">
  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/logo.png" width="65%">
</p>

<p align="center">
  <a href="https://lorangan-ddup.github.io/SpatialReward/"><img src="https://img.shields.io/badge/Project%20Page-SpatialReward-yellow" alt="project page"></a>
  <a href="https://arxiv.org/abs/2602.07458"><img src="https://img.shields.io/badge/arXiv-2602.07458-b31b1b.svg" alt="arxiv"></a>
  <a href="https://github.com/lorangan-ddup/SpatialReward"><img src="https://img.shields.io/badge/GitHub-Code-black?logo=github" alt="github"></a>
  <a href="https://huggingface.co/datasets/SpatialReward/MER-Bench"><img src="https://img.shields.io/badge/MER--Bench-🤗-yellow" alt="dataset"></a>
  <a href="https://huggingface.co/datasets/SpatialReward/SpatialReward-Train"><img src="https://img.shields.io/badge/Training--Data-🤗-yellow" alt="dataset"></a>
</p>

<h4 align="center">
    <p>
        <a href=#-news>News</a> |
        <a href=#-introduction>Introduction</a> |
        <a href=#-quick-start>Quick Start</a> |
        <a href=#-benchmark-evaluation>Benchmark Evaluation</a> |
        <a href=#️-citing-us>Citation</a>
    </p>
</h4>

**SpatialReward** is a state-of-the-art reward model for instruction-guided image editing that addresses the critical "Attention Collapse" problem through explicit spatial reasoning. By anchoring semantic judgments to predicted edit regions via bounding boxes, SpatialReward achieves unprecedented accuracy and reliability as both an evaluator and RL training signal.

<p align="center">
  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/attention_visualization.png" width="95%">
  <br>
  <em>Visualizing the Attention Collapse problem vs. SpatialReward's spatial grounding.</em>
</p>

## 🔥 News

- **2026-05-05**: 🎉 We have open-sourced the **SpatialReward-8B** model weights, **[MER-Bench](https://huggingface.co/datasets/SpatialReward/MER-Bench)** benchmark, and **[SpatialReward-Train](https://huggingface.co/datasets/SpatialReward/SpatialReward-Train)** (260k spatial-aware training data)!
- **2026-05-01**: 🎉 **SpatialReward** has been accepted to **ICML 2026**!
- **2026-02-12**: We have released the **inference code**, **reward server**, and **training configurations**!
- **2026-02-07**: The paper is available on [arXiv](https://arxiv.org/abs/2602.07458).

## 📌 Introduction

Online Reinforcement Learning (RL) holds immense potential for advancing instruction-guided image editing, but its progress has been severely hindered by a critical perception gap we term **"Attention Collapse"**. Existing reward models frequently neglect cross-image comparisons and fail to capture fine-grained editing details, leading to inaccurate evaluations and unstable RL training.

To overcome this, we propose **SpatialReward**, which:
- **Introduces MER-Bench**: A new benchmark featuring multi-edit scenarios and expert human annotations for measuring reward model quality.
- **Enforces spatial reasoning**: Predicts bounding boxes for edit regions and anchors semantic judgments to pixel-level evidence.

<p align="center">
  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/performance_table.png" width="95%">
  <br>
  <em>Comprehensive benchmark results. SpatialReward achieves SOTA performance, outperforming GPT-4.1 and GPT-5 on MER-Bench.</em>
</p>

<p align="center">
  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/merbench_category_breakdown.png" width="70%">
  <br>
  <em>MER-Bench performance breakdown by editing category.</em>
</p>

## 🚀 Quick Start

### Installation

```bash
git clone https://github.com/Kwai-Keye/SpatialReward.git
cd SpatialReward

conda create -n spatialreward python=3.11 -y
conda activate spatialreward

pip install torch==2.8.0 torchvision --extra-index-url https://download.pytorch.org/whl/cu126
pip install -r requirements.txt
```

### Reward Server

```bash
# Start reward server
cd example/reward/server
bash start_servers.sh
bash start_proxy.sh

# Query from client
from example.reward.client.reward_client_edit import RewardClient

client = RewardClient(proxy_host="127.0.0.1", proxy_port=23456)
scores, rewards, reasoning, meta_data = client.evaluate(
    input_images=[input_img],
    output_image=[output_img],
    meta_datas=[{"instruction": "Remove the dog"}]
)
```

## 📊 Benchmark Evaluation

Model and data are loaded directly from HuggingFace by default.

```bash
# MER-Bench
bash eval/MERBench/run.sh

# MMRB2
bash eval/MMRB2/run.sh

# EditReward-Bench
bash eval/EditReward-Bench/run.sh
```

## 📚 Datasets

| Dataset | Description | Link |
|---|---|---|
| **SpatialReward-Train** | 260k spatial-aware training data (SFT + RL) | [🤗 Hub](https://huggingface.co/datasets/SpatialReward/SpatialReward-Train) |
| **MER-Bench** | MultiEditReward-Bench evaluation benchmark | [🤗 Hub](https://huggingface.co/datasets/SpatialReward/MER-Bench) |

## 🎯 Training

### SFT (LLaMA-Factory)
```bash
llamafactory-cli train example/SpatialReward-train/sft/qwen3vl_lora_spatial_reward.yaml
```

### RL (ms-swift / GRPO)

```bash
# Replace ORM first
cp example/SpatialReward-train/rl/orm.py <ms-swift>/swift/plugin/orm.py
bash example/SpatialReward-train/rl/run_mater.sh
```

### RL Results on OmniGen2

<p align="center">
  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/omnigen2_rl_results.png" width="85%">
  <br>
  <em>SpatialReward delivers +0.90 on GEdit-EN Overall, doubling GPT-4.1's gain (+0.45).</em>
</p>

<p align="center">
  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/rl_training_curves.png" width="95%">
  <br>
  <em>Stable RL training dynamics with SpatialReward as reward signal.</em>
</p>

## 🙏 Acknowledgements

We thank [EditScore](https://github.com/VectorSpaceLab/EditScore) and [EditReward](https://github.com/TIGER-AI-Lab/EditReward) for valuable references.

## ❤️ Citing Us

```bibtex
@article{long2026spatialreward,
  title={SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning},
  author={Long, Yancheng and Yang, Yankai and Wei, Hongyang and Chen, Wei and Zhang, Tianke and Fan, Haonan and Liu, Changyi and Jiang, Kaiyu and Chen, Jiankang and Tang, Kaiyu and Wen, Bin and Yang, Fan and Gao, Tingting and Li, Han and Yang, Shuo},
  journal={arXiv preprint arXiv:2602.07458},
  year={2026}
}
```

## 📄 License

Apache 2.0