EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment
Abstract
Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality Assessment: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence-grounded explanation. Without a supplied textual claim, the system must progressively refine factual questions during investigation and align external evidence with the specific situation depicted. We introduce EviLens, an expert-reviewed benchmark of 10,174 supported and refuted images from controlled construction and web collection, covering twelve factual error types and evaluating both verdict accuracy and evidence sufficiency. We develop EviScene, a multimodal ReAct agent that interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. We train a compact policy with supervised fine-tuning on reviewed teacher trajectories, followed by privileged on-policy self-distillation applied to verified decisions from the policy's own investigations. On the 1,527-image test set, the trained Qwen3.5-9B agent achieves 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate (SESR) of 52.26%, improving over the same-backbone base agent by 6.71 and 9.37 percentage points, respectively. Its SESR is the highest among the evaluated agents.Code, models, and data will be released at https://github.com/wangzian828/EVISCENE and https://huggingface.co/ZiAnwang/EviScene.
Community
Hi Community!
๐ We introduce EviScene: an approach where the model is given only an image, without pre-provided textual assertions to verify. The agent autonomously identifies issues and conducts investigations through visual inspection, web searches, image retrieval, and evidence comparison. This capability is cultivated via Supervised Fine-Tuning (SFT) and Privileged on-policy Self-Distillation (PSD).
๐ The accompanying benchmark, EviLens, comprises 10,174 images and covers 12 categories of factual errors. Evaluation assesses both the correctness of the judgment and whether the evidence substantiates the conclusion; success requires not only the right answer but also a solid evidentiary basis.
๐ Tested on 1,527 images, the trained Qwen3.5-9B agent achieved 79.05% BAcc and 52.26% SESRโimprovements of 6.71 and 9.37 points, respectively, over the baseline agent. It attained the highest SESR among all agents evaluated in the study. Code, model weights, and datasets have all been made publicly available.
๐ Paper: https://arxiv.org/abs/2610.14609
๐ป Code: https://github.com/wangzian828/EVISCENE
๐ค Models & Data: https://huggingface.co/ZiAnwang/EviScene
๐ Website: https://wangzian828.github.io/EVISCENE/
Get this paper in your agent:
hf papers read 2610.14609 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 2
Collections including this paper 0
No Collection including this paper