metadata
license: apache-2.0
base_model: Qwen/Qwen3-VL-8B-Instruct
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- multimodal
- hallucination-detection
- hallucination-classification
- hallucination-diagnosis
- mllm
- qwen3-vl
language:
- en
- zh
datasets:
- wkinglin/HalluScope-30K
HalluScope-8B
HalluScope-8B is a diagnostic model for fine-grained hallucination diagnosis in multimodal large language models (MLLMs). Given an image and a model-generated response, it detects hallucinated spans, classifies each into one of 12 fine-grained types, and returns span-level annotations in a single pass. It is the larger, higher-accuracy variant of the HalluScope family.
- Base model: Qwen3-VL-8B-Instruct
- Training data: HalluScope-30K
- Task: span-level hallucination detection + classification
Output Format
The model wraps hallucinated spans with typed <hallucination> tags:
<Tagged_Text>
The <hallucination type="Color_Attribute">bright red</hallucination>
<hallucination type="Object">sports</hallucination> car is
<hallucination type="Spatial_Attribute">parked near a lake</hallucination>.
</Tagged_Text>
Hallucination Taxonomy
12 fine-grained types across two categories:
| Category | Types |
|---|---|
| Perception | Object, OCR, Numerical_Attribute, Color_Attribute, Shape_Attribute, Spatial_Attribute |
| Reasoning | Logical_Error, Calculation_Error, Knowledge_Error, Query_Misunderstanding, Numerical_Relation, Spatial_Relation |
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image
model = AutoModelForImageTextToText.from_pretrained(
"wkinglin/HalluScope-8B", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("wkinglin/HalluScope-8B")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": Image.open("example.jpg")},
{"type": "text", "text": "Analyze the response and tag hallucinated spans:\n<response to diagnose>"},
],
}]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=2048)
print(processor.batch_decode(out, skip_special_tokens=True)[0])
For high-throughput inference, serve the model with vLLM and query it through the OpenAI-compatible API.
Related
- HalluScope-4B — the lightweight variant: wkinglin/HalluScope-4B
- Dataset — wkinglin/HalluScope-30K
Citation
@inproceedings{jin2026halluscope,
title = {HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models},
author = {Jin, Weilin and Wang, Mingyu and Li, Wenbo and Huang, Haoyang and Wu, Yifan and Li, Ying and Huang, Gang and Wu, Zhonghai},
booktitle = {Proceedings of the 34th ACM International Conference on Multimedia (MM '26)},
year = {2026}
}