README.md

license: bsd-3-clause
library_name: pytorch
pipeline_tag: image-classification
tags:
- facial-forgery-detection
- multi-label-classification
- vit
- deepfake
- acl-2026
---

# Face-ViT: Multi-Label Facial Forgery Region Classifier

## 📖 Model Description
This is the **Face-ViT** auxiliary perception module proposed in the ACL 2026 paper: 
*"Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline"*.

Face-ViT is a multi-label classifier based on the **ViT-H/14** architecture. It is specifically trained to recognize 21 different types of facial manipulations (e.g., eye modification, skin smoothing, mouth tampering). In the DFF framework, it provides fine-grained visual cues that guide the large language model to generate accurate forensic explanations.

## 🛠️ Model Details
- **Architecture**: ViT-H/14 with an additional CNN branch and max-pooling for multi-label support.
- **Input Size**: 224x224 RGB images.
- **Number of Classes**: 21 (Facial attributes/manipulation types).
- **Training Objective**: Joint loss including BCE, Focal, Dice, and Jaccard loss.

## 🚀 Links
- **Official Code**: [Generating-Attribution-Reports](https://github.com/NattyLianJc/Generating-Attribution-Reports)
- **Main Framework (DFF)**: [LianJC/DFF-InstructBLIP-Detection](https://huggingface.co/LianJC/DFF-InstructBLIP-Detection)
- **Dataset (MMTT)**: [LianJC/MMTT-Dataset](https://huggingface.co/datasets/LianJC/MMTT-Dataset)

## 📜 Citation
If you find this model useful, please cite:
```bibtex
@inproceedings{lian2026generating,
  title={Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline},
  author={Lian, Jingchun and others},
  booktitle={Proceedings of ACL},
  year={2026},
  note={To appear}
}