Hugging Face's logo

MeiGen-AI
/
PosterReward_v1

Image-to-Text
Transformers
Safetensors
English
reward-model
poster
graphic-design
image-quality-assessment
preference-learning
qwen3-vl
Model card Files
xet
 Community
PosterReward_v1
File size: 4,565 Bytes
3ff8083
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
---
license: apache-2.0
language:
- en
tags:
- reward-model
- poster
- graphic-design
- image-quality-assessment
- preference-learning
- qwen3-vl
pipeline_tag: image-to-text
library_name: transformers
---

<div align="center">

<h1>PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation</h1>

<img src="assert/posterreward_logo_v1.png" alt="PosterReward Logo" width="200"/>

[![Paper](https://img.shields.io/badge/Paper-PDF-red)](https://alexlai2860.github.io/mypaper/posterreward/PosterReward_Arxiv_released.pdf)
[![Project](https://img.shields.io/badge/Project-Page-black)](https://alexlai2860.github.io/PosterReward/)
[![Code](https://img.shields.io/badge/Code-Repository-blue)](https://github.com/MeiGen-AI/PosterReward)
[![arXiv](https://img.shields.io/badge/arXiv-2603.29855-red)](https://arxiv.org/abs/2603.29855)


</div>

## Overview

PosterReward is a dedicated reward modeling framework for poster assessment. It builds a **70k poster preference dataset** from multi-MLLM consensus and introduces specialized models for poster quality evaluation across five dimensions:

1. **Foundational Visual Quality**
2. **AI Artifacts**
3. **Textual Accuracy**
4. **Prompt Fidelity**
5. **Aesthetic Value**

## Available Models

This repository contains three model variants, all built on **Qwen3-VL-8B**:

| Model | Path | Type | Description |
|-------|------|------|-------------|
| **PosterReward Analyser** | `PosterReward_analyser/` | Generative VLM | Generates detailed multi-dimensional analysis of poster images |
| **PosterReward Scorer** | `PosterReward_scorer/` | Scalar Reward Model | Takes analysis + image and produces a scalar reward score |
| **PosterReward-Lite** | `PosterReward-Lite/` | Scalar Reward Model | Simplified pointwise scorer that omits the analysis module for faster inference |

### PosterReward (Full Pipeline)

The full PosterReward pipeline is a two-stage `analysis -> scoring` process:
1. **PosterReward Analyser** generates a detailed textual analysis across five quality dimensions.
2. **PosterReward Scorer** takes the analysis together with the image and outputs a scalar reward score.

### PosterReward-Lite

A simplified variant that directly predicts a scalar reward from the image and prompt, without requiring a separate analysis step. Faster inference at the cost of slightly lower accuracy.

## Benchmark Data

This repository also hosts the **PosterRewardBench** benchmark images:

| File | Description | Size |
|------|-------------|------|
| `PRB_basic_images.tar.gz` | PosterRewardBench-Basic images (1,034 images from Flux, Flux-Krea, SD3.5-L) | ~1.1 GB |
| `PRB_advanced_images.tar.gz` | PosterRewardBench-Advanced images (2,446 images from Seedream-3.0, Seedream-4.0, Qwen-Image-Lightning) | ~736 MB |

Download and extract these archives into the `poster_reward_bench/` directory of the [code repository](https://github.com/MeiGen-AI/PosterReward).

## Quick Start

### Environment Setup

```bash
git clone https://github.com/MeiGen-AI/PosterReward.git
cd PosterReward

cd swift && pip install -e . && cd ..
pip install msgspec "qwen_vl_utils>=0.0.14" torchvision diffusers pillow
```

### PosterReward-Lite (Fast Pointwise Scoring)

```python
from swift.llm import PtEngine, InferRequest

model_path = "path/to/PosterReward-Lite"  # or download from this repo
engine = PtEngine(model_path, max_batch_size=64, task_type='seq_cls', num_labels=1)

messages = [
    {"role": "user", "content": "<image>Your poster description prompt here."},
    {"role": "assistant", "content": ""}
]
request = InferRequest(messages=messages, images=["path/to/poster.png"])

resp_list = engine.infer([request])
score = resp_list[0].choices[0].message.content
print(f"Reward Score: {score}")
```

### Full PosterReward (Two-Stage Pipeline)

```bash
# Edit model paths in inference_posterreward.sh, then:
bash inference_posterreward.sh
```

## Results

### Pointwise Reward Models on PosterRewardBench

| Model | MMRB2 ↑ | HPDv3 ↑ | PRB-Basic ↑ | PRB-Ad ↑ |
|-------|---------|---------|-------------|----------|
| ImageReward | 53.0 | 58.6 | 60.7 | 49.3 |
| PickScore | 57.6 | 65.6 | 66.7 | 44.1 |
| HPSv2 | 55.0 | 65.3 | 70.8 | 43.7 |
| HPSv3 | 58.5 | 76.9 | 72.9 | 41.2 |
| **PosterReward-Lite** | **60.5** | **77.1** | **83.9** | **85.0** |
| **PosterReward** | **59.6** | **77.8** | **86.7** | **86.0** |

## Citation

Coming Soon!

## Acknowledgments

- Thanks to our collaborators and affiliated institutions.
- Thanks to the open-source community and prior reward modeling research.