README.md

---
license: bsd-3-clause
library_name: lavis
pipeline_tag: visual-question-answering
tags:
- explainable-ai
- deepfake-detection
- vlm
- instructblip
- forensic-explanation
- acl-2026
---

# DFF: InstructBLIP-based Explainable DeepFake Detection

## 📖 Model Description
This is the core **DFF (DeepFake Detection and Forensic Explanation Framework)** model as described in the ACL 2026 paper: 
*"Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline"*.

DFF is built upon the **InstructBLIP (Flan-T5 XL)** architecture. By integrating the Face-ViT auxiliary classifier, it achieves state-of-the-art performance in both **forgery localization (mask generation)** and **forensic explanation (captioning)**. 

## 🌟 Key Capabilities
1. **Forgery Localization**: Generates high-resolution binary masks highlighting manipulated facial regions.
2. **Natural Language Explanation**: Produces detailed text describing why a specific image is considered a forgery (e.g., "The texture around the eyes is unnatural due to GAN-based blending").

## 🛠️ Model Details
- **Base LLM**: Flan-T5 XL.
- **Visual Encoder**: EVA-ViT-G.
- **Auxiliary Module**: Face-ViT (Multi-label perception).
- **Task**: Explainable Detection & Multi-modal Attribution Reporting.

## 🚀 Links
- **Official Code**: [Generating-Attribution-Reports](https://github.com/NattyLianJc/Generating-Attribution-Reports)
- **Auxiliary Classifier**: [LianJC/Face-ViT-MultiLabel](https://huggingface.co/LianJC/Face-ViT-MultiLabel)
- **Dataset (MMTT)**: [LianJC/MMTT-Dataset](https://huggingface.co/datasets/LianJC/MMTT-Dataset)

## 📜 Citation
```bibtex
@inproceedings{lian2026generating,
  title={Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline},
  author={Lian, Jingchun and others},
  booktitle={Proceedings of ACL},
  year={2026},
  note={To appear}
}