---
pipeline_tag: image-text-to-text
library_name: transformers
license: mit
---

# DiffCLIP: Differential Attention Meets CLIP

This repository contains the DiffCLIP model as presented in [DiffCLIP: Differential Attention Meets CLIP](https://huggingface.co/papers/2503.06626).

Project Page: https://hammoudhasan.github.io/DiffCLIP

Code: https://github.com/hammoudhasan/DiffCLIP

## How to Use

### Installation

```bash
# Clone the repository
git clone https://github.com/hammoudhasan/DiffCLIP.git
cd DiffCLIP

# Install dependencies
pip install -r requirements.txt
```

### Basic Usage

```python
import torch
from diff_clip import DiffCLIP_VITB16

# Create model
model = DiffCLIP_VITB16()

# Process image and text
image = torch.randn(1, 3, 224, 224)
text = torch.randint(0, 49408, (1, 77))  # Tokenized text

# Get embeddings
with torch.no_grad():
    outputs = model(image, text)

print(outputs["image_embed"].shape)  # Should be [1, 512]
print(outputs["text_embed"].shape)   # Should be [1, 512]
```

### Zero-Shot Classification

You can use the provided `test_models.py` script to perform zero-shot classification. See the [GitHub README](https://github.com/hammoudhasan/DiffCLIP) for details.