pipeline_tag: visual-document-retrieval
library_name: transformers
license: apache-2.0
ModernVBERT: Towards Smaller Visual Document Retrievers 👁️
This repository contains the ModernVBERT model, a compact 250M-parameter vision-language encoder designed for efficient Visual Document Retrieval (VDR). As presented in the paper "ModernVBERT: Towards Smaller Visual Document Retrievers", this model establishes a principled recipe for improving VDR models by revisiting the entire training pipeline. It outperforms models up to 10 times larger while enabling efficient inference on CPU hardware, significantly reducing latency and costs. Key factors measured for improvement include attention masking, image resolution, modality alignment data regimes, and late interaction-centered contrastive objectives.
Usage
A detailed tutorial for fine-tuning and using ModernVBERT, including all information required to launch a model post-training, is available in a Google Colab notebook:
Citation
If you use ModernVBERT in your research, please cite the paper as follows:
@misc{teiletche2025modernvbertsmallervisualdocument,
title={ModernVBERT: Towards Smaller Visual Document Retrievers},
author={Paul Teiletche and Quentin Macé and Max Conti and Antonio Loison and Gautier Viaud and Pierre Colombo and Manuel Faysse},
year={2025},
eprint={2510.01149},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2510.01149},
}