README.md

---
license: mit
library_name: transformers
pipeline_tag: image-feature-extraction
base_model: google/siglip2-so400m-patch16-512
tags:
- siglip
- siglip2
- vision-encoder
- mllm
- image-feature-extraction
---

# SigLIP-HD

SigLIP-HD is a vision encoder fine-tuned from
[SigLIP 2-So400m/16-512px](https://huggingface.co/google/siglip2-so400m-patch16-512)
with **fine-to-coarse supervision**.

- **Paper:** [SigLIP-HD by Fine-to-Coarse Supervision]()
- **Code:** https://github.com/LiheYoung/SigLIP-HD

SigLIP-HD exhibits better performance than SigLIP 2 in MLLMs, especially for OCR scenarios.

This repository contains **only the vision encoder** (no text tower). It is a **drop-in replacement**
for the SigLIP 2 vision tower: identical architecture and I/O. To use it in an MLLM, keep your existing SigLIP 2 pipeline and only change the
vision-tower path to this checkpoint.

## Usage

```python
import torch
from PIL import Image
from transformers import SiglipVisionModel, AutoImageProcessor

model = SiglipVisionModel.from_pretrained("LiheYoung/SigLIP-HD").eval()
processor = AutoImageProcessor.from_pretrained("LiheYoung/SigLIP-HD")

image = Image.open("example.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    features = model(**inputs, output_hidden_states=True).hidden_states[-1]  # (1, 1024, 1152)
```

## Citation

```bibtex
@inproceedings{sigliphd,
  title={SigLIP-HD by Fine-to-Coarse Supervision},
  author={Yang, Lihe and Zhao, Zhen and Zhao, Hengshuang},
  booktitle={ICLR},
  year={2026}
}
```

## Acknowledgement

This work is built upon [SigLIP 2](https://arxiv.org/abs/2502.14786). We sincerely thank the authors for
open-sourcing their excellent vision encoder.