license: apache-2.0
library_name: peft
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3-VL-4B-Instruct
tags:
- image-editing
- video-editing
- reward-model
- evaluation
UniEditBench Distilled Evaluator (4B LoRA)
This repository contains the distilled 4B MLLM judge (LoRA adapter) introduced in the paper UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs.
UniEditBench is a unified benchmark for image and video editing that standardizes evaluation across different paradigms and modalities. To enable scalable and cost-effective evaluation, the authors distilled high-capacity MLLM judges (such as Qwen3-VL-235B) into lightweight 4B and 8B evaluators. These reward models provide multi-dimensional scoring across structural fidelity, text alignment, background consistency, naturalness, and (for videos) temporal-spatial consistency.
- Paper: UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs
- GitHub Repository: wesar1/UniEditBench
- Dataset: UniEditBench Dataset
Usage
Deployment
The reward model can be deployed using the swift framework. Below is an example command to deploy the 4B image-based evaluator:
CUDA_VISIBLE_DEVICES=0 swift deploy \
--model Qwen/Qwen3-VL-4B-Instruct \
--adapters /path/to/sft_image_lora_4b \
--device_map balanced \
--vllm_tensor_parallel_size 1 \
--attn_impl eager \
--infer_backend vllm \
--port 8003 \
--vllm_max_lora_rank 32 \
--max_new_tokens 2048 \
--served_model_name Qwen3-VL-4B-SFT-Image
For video evaluation or 8B model scripts, please refer to the scripts/deploy_qwen3vl.sh file in the official GitHub repository.
Citation
If you find UniEditBench helpful, please cite the following work:
@misc{jiang2026unieditbenchunifiedcosteffectivebenchmark,
title={UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs},
author={Lifan Jiang and Tianrun Wu and Yuhang Pei and Chenyang Wang and Boxi Wu and Deng Cai},
year={2026},
eprint={2604.15871},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2604.15871},
}