---
library_name: pytorch
language:
- en
tags:
- misinformation-detection
- social-media
- fakett
- multimodal
- pytorch
- transformers
---

# SigLIP Base Patch16-224 — Multimodal Misinformation Detection on FakeTT

**Authors:** Andrei-Gabriel Radu, Ciprian-Octavian Truică, Elena-Simona Apostol

**National University of Science and Technology POLITEHNICA Bucharest**

Supervised **multimodal misinformation classification** checkpoint based on **SigLIP Base Patch16-224**, trained and evaluated on FakeTT.

This model accompanies the bachelor thesis *Misinformation Detection in Social Media Videos*.


## Results

| Dataset | Modality | Macro-F1 |
|---|---|---:|
| FakeTT | Multimodal | 0.8798 |

## Model

- **Architecture:** SigLIP Base Patch16-224
- **Task:** Binary misinformation classification
- **Modality:** Text + video
- **Dataset:** FakeTT
- **Number of classes:** 2
- **Checkpoint:** `pytorch_model.bin`
- **Primary metric:** Macro-F1

## Saved Metadata

- **batch_size:** 8
- **dropout:** 0.1
- **early_stopping_patience:** 3
- **freeze_backbone:** False
- **fusion_hidden_dim:** 256
- **gradient_accumulation_steps:** 1
- **learning_rate:** 1e-05
- **lr_scheduler_type:** linear
- **max_grad_norm:** 1.0
- **model_name:** google/siglip-base-patch16-224
- **num_labels:** 2
- **num_train_epochs:** 10
- **text_max_length:** 64
- **warmup_ratio:** 0.0
- **weight_decay:** 0.01

The original machine-readable metadata is included as `model_metadata.json`.

## Usage

This repository contains a checkpoint for the custom multimodal classifier used in the thesis implementation. It is **not** a drop-in `AutoModel.from_pretrained()` repository.

Instantiate the matching custom classifier, load the included tokenizer and image/video processor, then load the state dictionary:

```python
import torch

state = torch.load("pytorch_model.bin", map_location="cpu")

# model = MatchingClassifier(...)
# model.load_state_dict(state)
# model.eval()
```

The constructor and preprocessing must match the thesis implementation and `model_metadata.json`.

## Intended Use

Research, benchmarking and reproducibility of supervised multimodal misinformation detection on short social-media videos.

## Limitations

The model performs classification rather than factual verification, can inherit biases from FakeTT and its pretrained encoders, and requires the matching custom implementation for reproducible inference.

## Citation

```bibtex
@thesis{radu2026misinformation,
    author = {Radu, Andrei-Gabriel and Truică, Ciprian-Octavian and Apostol, Elena-Simona},
    title  = {Misinformation Detection in Social Media Videos},
    school = {National University of Science and Technology POLITEHNICA Bucharest},
    year   = {2026}
}
```