README.md

---
license: apache-2.0
library_name: peft
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3-VL-4B-Instruct
tags:
- image-editing
- video-editing
- reward-model
- evaluation
---

# UniEditBench Distilled Evaluator (4B LoRA)

This repository contains the distilled 4B MLLM judge (LoRA adapter) introduced in the paper [UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs](https://huggingface.co/papers/2604.15871).

UniEditBench is a unified benchmark for image and video editing that standardizes evaluation across different paradigms and modalities. To enable scalable and cost-effective evaluation, the authors distilled high-capacity MLLM judges (such as Qwen3-VL-235B) into lightweight 4B and 8B evaluators. These reward models provide multi-dimensional scoring across structural fidelity, text alignment, background consistency, naturalness, and (for videos) temporal-spatial consistency.

- **Paper:** [UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs](https://huggingface.co/papers/2604.15871)
- **GitHub Repository:** [wesar1/UniEditBench](https://github.com/wesar1/UniEditBench)
- **Dataset:** [UniEditBench Dataset](https://huggingface.co/datasets/wesar1/UniEditBench)

## Usage

### Deployment

The reward model can be deployed using the [swift](https://github.com/modelscope/swift) framework. Below is an example command to deploy the 4B image-based evaluator:

```bash
CUDA_VISIBLE_DEVICES=0 swift deploy \
    --model Qwen/Qwen3-VL-4B-Instruct \
    --adapters /path/to/sft_image_lora_4b \
    --device_map balanced \
    --vllm_tensor_parallel_size 1 \
    --attn_impl eager \
    --infer_backend vllm \
    --port 8003 \
    --vllm_max_lora_rank 32 \
    --max_new_tokens 2048 \
    --served_model_name Qwen3-VL-4B-SFT-Image
```

For video evaluation or 8B model scripts, please refer to the `scripts/deploy_qwen3vl.sh` file in the official GitHub repository.

## Citation

If you find UniEditBench helpful, please cite the following work:

```bibtex
@misc{jiang2026unieditbenchunifiedcosteffectivebenchmark,
      title={UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs}, 
      author={Lifan Jiang and Tianrun Wu and Yuhang Pei and Chenyang Wang and Boxi Wu and Deng Cai},
      year={2026},
      eprint={2604.15871},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2604.15871}, 
}
```