Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,130 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- reward-model
|
| 7 |
+
- poster
|
| 8 |
+
- graphic-design
|
| 9 |
+
- image-quality-assessment
|
| 10 |
+
- preference-learning
|
| 11 |
+
- qwen3-vl
|
| 12 |
+
pipeline_tag: image-to-text
|
| 13 |
+
library_name: transformers
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
<div align="center">
|
| 17 |
+
|
| 18 |
+
<h1>PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation</h1>
|
| 19 |
+
|
| 20 |
+
<img src="assert/posterreward_logo_v1.png" alt="PosterReward Logo" width="200"/>
|
| 21 |
+
|
| 22 |
+
[](https://alexlai2860.github.io/mypaper/posterreward/PosterReward_Arxiv_released.pdf)
|
| 23 |
+
[](https://alexlai2860.github.io/PosterReward/)
|
| 24 |
+
[](https://github.com/MeiGen-AI/PosterReward)
|
| 25 |
+
[](https://arxiv.org/abs/2603.29855)
|
| 26 |
+
|
| 27 |
+
**CVPR 2026**
|
| 28 |
+
|
| 29 |
+
</div>
|
| 30 |
+
|
| 31 |
+
## Overview
|
| 32 |
+
|
| 33 |
+
PosterReward is a dedicated reward modeling framework for poster assessment. It builds a **70k poster preference dataset** from multi-MLLM consensus and introduces specialized models for poster quality evaluation across five dimensions:
|
| 34 |
+
|
| 35 |
+
1. **Foundational Visual Quality**
|
| 36 |
+
2. **AI Artifacts**
|
| 37 |
+
3. **Textual Accuracy**
|
| 38 |
+
4. **Prompt Fidelity**
|
| 39 |
+
5. **Aesthetic Value**
|
| 40 |
+
|
| 41 |
+
## Available Models
|
| 42 |
+
|
| 43 |
+
This repository contains three model variants, all built on **Qwen3-VL-8B**:
|
| 44 |
+
|
| 45 |
+
| Model | Path | Type | Description |
|
| 46 |
+
|-------|------|------|-------------|
|
| 47 |
+
| **PosterReward Analyser** | `PosterReward_analyser/` | Generative VLM | Generates detailed multi-dimensional analysis of poster images |
|
| 48 |
+
| **PosterReward Scorer** | `PosterReward_scorer/` | Scalar Reward Model | Takes analysis + image and produces a scalar reward score |
|
| 49 |
+
| **PosterReward-Lite** | `PosterReward-Lite/` | Scalar Reward Model | Simplified pointwise scorer that omits the analysis module for faster inference |
|
| 50 |
+
|
| 51 |
+
### PosterReward (Full Pipeline)
|
| 52 |
+
|
| 53 |
+
The full PosterReward pipeline is a two-stage `analysis -> scoring` process:
|
| 54 |
+
1. **PosterReward Analyser** generates a detailed textual analysis across five quality dimensions.
|
| 55 |
+
2. **PosterReward Scorer** takes the analysis together with the image and outputs a scalar reward score.
|
| 56 |
+
|
| 57 |
+
### PosterReward-Lite
|
| 58 |
+
|
| 59 |
+
A simplified variant that directly predicts a scalar reward from the image and prompt, without requiring a separate analysis step. Faster inference at the cost of slightly lower accuracy.
|
| 60 |
+
|
| 61 |
+
## Benchmark Data
|
| 62 |
+
|
| 63 |
+
This repository also hosts the **PosterRewardBench** benchmark images:
|
| 64 |
+
|
| 65 |
+
| File | Description | Size |
|
| 66 |
+
|------|-------------|------|
|
| 67 |
+
| `PRB_basic_images.tar.gz` | PosterRewardBench-Basic images (1,034 images from Flux, Flux-Krea, SD3.5-L) | ~1.1 GB |
|
| 68 |
+
| `PRB_advanced_images.tar.gz` | PosterRewardBench-Advanced images (2,446 images from Seedream-3.0, Seedream-4.0, Qwen-Image-Lightning) | ~736 MB |
|
| 69 |
+
|
| 70 |
+
Download and extract these archives into the `poster_reward_bench/` directory of the [code repository](https://github.com/MeiGen-AI/PosterReward).
|
| 71 |
+
|
| 72 |
+
## Quick Start
|
| 73 |
+
|
| 74 |
+
### Environment Setup
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
git clone https://github.com/MeiGen-AI/PosterReward.git
|
| 78 |
+
cd PosterReward
|
| 79 |
+
|
| 80 |
+
cd swift && pip install -e . && cd ..
|
| 81 |
+
pip install msgspec "qwen_vl_utils>=0.0.14" torchvision diffusers pillow
|
| 82 |
+
```
|
| 83 |
+
|
| 84 |
+
### PosterReward-Lite (Fast Pointwise Scoring)
|
| 85 |
+
|
| 86 |
+
```python
|
| 87 |
+
from swift.llm import PtEngine, InferRequest
|
| 88 |
+
|
| 89 |
+
model_path = "path/to/PosterReward-Lite" # or download from this repo
|
| 90 |
+
engine = PtEngine(model_path, max_batch_size=64, task_type='seq_cls', num_labels=1)
|
| 91 |
+
|
| 92 |
+
messages = [
|
| 93 |
+
{"role": "user", "content": "<image>Your poster description prompt here."},
|
| 94 |
+
{"role": "assistant", "content": ""}
|
| 95 |
+
]
|
| 96 |
+
request = InferRequest(messages=messages, images=["path/to/poster.png"])
|
| 97 |
+
|
| 98 |
+
resp_list = engine.infer([request])
|
| 99 |
+
score = resp_list[0].choices[0].message.content
|
| 100 |
+
print(f"Reward Score: {score}")
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
### Full PosterReward (Two-Stage Pipeline)
|
| 104 |
+
|
| 105 |
+
```bash
|
| 106 |
+
# Edit model paths in inference_posterreward.sh, then:
|
| 107 |
+
bash inference_posterreward.sh
|
| 108 |
+
```
|
| 109 |
+
|
| 110 |
+
## Results
|
| 111 |
+
|
| 112 |
+
### Pointwise Reward Models on PosterRewardBench
|
| 113 |
+
|
| 114 |
+
| Model | MMRB2 ↑ | HPDv3 ↑ | PRB-Basic ↑ | PRB-Ad ↑ |
|
| 115 |
+
|-------|---------|---------|-------------|----------|
|
| 116 |
+
| ImageReward | 53.0 | 58.6 | 60.7 | 49.3 |
|
| 117 |
+
| PickScore | 57.6 | 65.6 | 66.7 | 44.1 |
|
| 118 |
+
| HPSv2 | 55.0 | 65.3 | 70.8 | 43.7 |
|
| 119 |
+
| HPSv3 | 58.5 | 76.9 | 72.9 | 41.2 |
|
| 120 |
+
| **PosterReward-Lite** | **60.5** | **77.1** | **83.9** | **85.0** |
|
| 121 |
+
| **PosterReward** | **59.6** | **77.8** | **86.7** | **86.0** |
|
| 122 |
+
|
| 123 |
+
## Citation
|
| 124 |
+
|
| 125 |
+
Coming Soon!
|
| 126 |
+
|
| 127 |
+
## Acknowledgments
|
| 128 |
+
|
| 129 |
+
- Thanks to our collaborators and affiliated institutions.
|
| 130 |
+
- Thanks to the open-source community and prior reward modeling research.
|