Unison-Judge is a fine-tuned Qwen3-VL-8B vision-language model that serves as the local automatic judge for the Unison benchmark. It scores UMMs' outputs across all four unified tasks (IC, UGG, GGU and ME) without requiring a hosted API.
Code: github.com/FudanCVL/Unison — if you use this model or find it helpful, please give it a star!
Judge Consistency Data
The Judge_Consistency/ directory contains 231 evaluation cases used to assess the scoring consistency of Unison-Judge across all four tasks.
| Field | Description |
|---|---|
id |
Item identifier |
task |
One of IC, UGG, GGU, ME |
family |
Question type |
model |
The UMM whose output is being evaluated |
questions |
List of sub-questions, each with the model's answer and judge-assigned score |
images |
Reference image(s) and the model-generated image |
Task distribution: IC (57), ME (62), GGU (56), UGG (56)
Models covered: BAGEL-7B-MoT, OmniGen2, SEED-X-17B, UniWorld-V1