README.md

---
license: apache-2.0
base_model:
- google/siglip2-base-patch16-256
language:
- en
pipeline_tag: image-classification
library_name: transformers
tags:
- SigLIP2
- ImageShield
- 90M
- Guardrail
---

![1](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/k8CGgBmXs93h8zpWVPKgK.png)

# **SigLIP2-ImageShield-90M-256**

> **SigLIP2-ImageShield-90M-256** is a vision-language encoder model fine-tuned from **google/siglip2-base-patch16-256** for **multi-class image classification**. Built on the **SiglipForImageClassification** architecture, the model is designed to identify and categorize visual content for explicit, suggestive, and safe media filtering.

> [!note]
> *SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features*
> https://arxiv.org/pdf/2502.14786

> [!note]
> This model is experimental. Expert VLMs are coming soon: [ImageShield Multimodal SFT Collection](https://huggingface.co/collections/prithivMLmods/imageshield-multimodal-sft).

## **Label Space: 5 Classes**

The model classifies each image into one of the following content categories:

```text
Class 0: "Anime"
Class 1: "Hentai"
Class 2: "Normal"
Class 3: "Pornography"
Class 4: "Sensual"
```

## **Install Dependencies**

```bash
pip install transformers torch torchvision pillow gradio
```

## **Inference Code**

```python
import gradio as gr
from transformers import AutoImageProcessor, SiglipForImageClassification
from PIL import Image
import torch

# Load model and processor
model_name = "prithivMLmods/SigLIP2-ImageShield-90M-256"  # Replace with your model path if needed
model = SiglipForImageClassification.from_pretrained(model_name)
processor = AutoImageProcessor.from_pretrained(model_name)

# ID to Label mapping
id2label = {
    "0": "Anime",
    "1": "Hentai",
    "2": "Normal",
    "3": "Pornography",
    "4": "Sensual"
}

def classify_image(image):
    image = Image.fromarray(image).convert("RGB")
    inputs = processor(images=image, return_tensors="pt")

    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits
        probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist()

    prediction = {
        id2label[str(i)]: round(probs[i], 3)
        for i in range(len(probs))
    }

    return prediction

# Gradio Interface
iface = gr.Interface(
    fn=classify_image,
    inputs=gr.Image(type="numpy"),
    outputs=gr.Label(
        num_top_classes=5,
        label="Predicted Content Type"
    ),
    title="SigLIP2-ImageShield-90M-256",
    description="Classifies images into Anime, Hentai, Normal, Pornography, and Sensual categories."
)

if __name__ == "__main__":
    iface.launch()
```

## **Intended Use**

This model is intended for applications such as:

* **Content Moderation:** Detect explicit or suggestive visual content.
* **Parental Controls:** Support AI-based media filtering.
* **Dataset Preprocessing:** Categorize and filter image datasets.
* **Online Platforms:** Assist with content safety and upload moderation.

## **Classification Report**

### Training vs Evaluation Loss / Accuracy

![Training vs Evaluation Loss and Accuracy](assets/training_eval_graph.png)

### Precision / Recall / F1-score per Class

![Per-Class Precision, Recall, and F1-score](assets/classification_report_bar.png)

### Confusion Matrix

![Confusion Matrix](assets/confusion_matrix.png)

### Test Set Class Distribution

![Test Set Class Distribution](assets/class_distribution_pie.png)

### Overall Prediction Accuracy

![Overall Prediction Accuracy](assets/prediction_accuracy_pie.png)

### Misalignment Distribution by True Class

![Misalignment Distribution by True Class](assets/misalignment_distribution_pie.png)

## **Acknowledgements**
 
* **[Transformers](https://huggingface.co/docs/transformers/en/index)**: Transformers provides state-of-the-art machine learning models for text, computer vision, audio, video, and multimodal tasks, supporting both inference and training.

* **[SigLIP 2](https://huggingface.co/papers/2502.14786)**: Multilingual vision-language encoders with improved semantic understanding, localization, and dense feature representations.