metadata
license: mit
library_name: transformers
pipeline_tag: image-feature-extraction
base_model: google/siglip2-so400m-patch16-512
tags:
- siglip
- siglip2
- vision-encoder
- mllm
- image-feature-extraction
SigLIP-HD
SigLIP-HD is a vision encoder fine-tuned from SigLIP 2-So400m/16-512px with fine-to-coarse supervision.
SigLIP-HD exhibits better performance than SigLIP 2 in MLLMs, especially for OCR scenarios.
This repository contains only the vision encoder (no text tower). It is a drop-in replacement for the SigLIP 2 vision tower: identical architecture and I/O. To use it in an MLLM, keep your existing SigLIP 2 pipeline and only change the vision-tower path to this checkpoint.
Usage
import torch
from PIL import Image
from transformers import SiglipVisionModel, AutoImageProcessor
model = SiglipVisionModel.from_pretrained("LiheYoung/SigLIP-HD").eval()
processor = AutoImageProcessor.from_pretrained("LiheYoung/SigLIP-HD")
image = Image.open("example.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
features = model(**inputs, output_hidden_states=True).hidden_states[-1] # (1, 1024, 1152)
Citation
@inproceedings{sigliphd,
title={SigLIP-HD by Fine-to-Coarse Supervision},
author={Yang, Lihe and Zhao, Zhen and Zhao, Hengshuang},
booktitle={ICLR},
year={2026}
}
Acknowledgement
This work is built upon SigLIP 2. We sincerely thank the authors for open-sourcing their excellent vision encoder.