README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-VL-4B-Instruct
pipeline_tag: image-text-to-text
library_name: transformers
tags:
  - multimodal
  - hallucination-detection
  - hallucination-classification
  - hallucination-diagnosis
  - mllm
  - qwen3-vl
language:
  - en
  - zh
datasets:
  - wkinglin/HalluScope-30K
---

# HalluScope-4B

**HalluScope-4B** is a lightweight diagnostic model for **fine-grained
hallucination diagnosis** in multimodal large language models (MLLMs). Given an
image and a model-generated response, it detects hallucinated spans, classifies
each into one of **12 fine-grained types**, and returns span-level annotations
in a single pass.

- **Base model:** Qwen3-VL-4B-Instruct
- **Training data:** [HalluScope-30K](https://huggingface.co/datasets/wkinglin/HalluScope-30K)
- **Task:** span-level hallucination detection + classification

## Output Format

The model wraps hallucinated spans with typed `<hallucination>` tags:

```xml
<Tagged_Text>
The <hallucination type="Color_Attribute">bright red</hallucination>
<hallucination type="Object">sports</hallucination> car is
<hallucination type="Spatial_Attribute">parked near a lake</hallucination>.
</Tagged_Text>
```

## Hallucination Taxonomy

12 fine-grained types across two categories:

| Category | Types |
|---|---|
| Perception | Object, OCR, Numerical_Attribute, Color_Attribute, Shape_Attribute, Spatial_Attribute |
| Reasoning | Logical_Error, Calculation_Error, Knowledge_Error, Query_Misunderstanding, Numerical_Relation, Spatial_Relation |

## Usage

```python
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model = AutoModelForImageTextToText.from_pretrained(
    "wkinglin/HalluScope-4B", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("wkinglin/HalluScope-4B")

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": Image.open("example.jpg")},
        {"type": "text", "text": "Analyze the response and tag hallucinated spans:\n<response to diagnose>"},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=2048)
print(processor.batch_decode(out, skip_special_tokens=True)[0])
```

For high-throughput inference, serve the model with vLLM and query it through
the OpenAI-compatible API.

## Related

- **HalluScope-8B** — the larger variant: [wkinglin/HalluScope-8B](https://huggingface.co/wkinglin/HalluScope-8B)
- **Dataset** — [wkinglin/HalluScope-30K](https://huggingface.co/datasets/wkinglin/HalluScope-30K)

## Citation

```bibtex
@inproceedings{jin2026halluscope,
  title     = {HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models},
  author    = {Jin, Weilin and Wang, Mingyu and Li, Wenbo and Huang, Haoyang and Wu, Yifan and Li, Ying and Huang, Gang and Wu, Zhonghai},
  booktitle = {Proceedings of the 34th ACM International Conference on Multimedia (MM '26)},
  year      = {2026}
}
```