BiUltraVBERT

BiUltraVBERT is a 255M-parameter single-vector retriever for visual documents, fine-tuned from UltraVBERT. It encodes each query and each page as a single 768-d vector and ranks pages by cosine similarity. Truncating the vectors to 128, 256, 320 or 512 dimensions gives a smaller index.

Models

Model	Role	Status
UltraVBERT	Base encoder aligned to page images	Available
BiUltraVBERT	Single-vector retriever for visual documents	Available
ColUltraVBERT	Late-interaction retriever for visual documents	Available · Demo

Data

Dataset	Content
VDR Train 500k	499,704 query-page pairs; queries in English (73%), Spanish, German, French, Italian and Portuguese

Usage

Requires sentence-transformers>=6.1 and transformers>=5.20.

from PIL import Image
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("athrael-soju/BiUltraVBERT", trust_remote_code=True)

pages = [Image.open("page1.png"), Image.open("page2.png")]
queries = ["What was the 2023 revenue?", "Which table lists regional margins?"]

page_embeddings = model.encode_document(pages)
query_embeddings = model.encode_query(queries)

# One row per query, one column per page.
scores = model.similarity(query_embeddings, page_embeddings)

For a smaller index, load the model with truncate_dim set to 128, 256, 320 or 512.

Model

BiUltraVBERT mean-pools UltraVBERT's token states, projects the result to 768 dimensions with a linear layer and L2-normalizes it. Pages are resized to at most 3,072 px on the longest edge and split into up to 25 tiles of 512 px, plus a downscaled view of the whole page.

Training

The released weights are a per-block merge of four fine-tunes of UltraVBERT on VDR Train 500k that differ only in seed. Each fine-tune trains rank-32 LoRA adapters on the text encoder and the output layer for one epoch at batch size 256, with Matryoshka losses at 128, 256, 320, 512 and 768 dimensions; the vision encoder and connector stay frozen.

ViDoRe

BiUltraVBERT's scores are its official MTEB results (results#785): MTEB 2.26.10, 768-d vectors, bf16 and SDPA attention. Multilingual tasks report the mean over their language splits. The MTEB maintainers scored the private ViDoRe V3 tasks, Nuclear and Telecom (results#791). BiModernVBERT's V1 and V2 scores are our MTEB 2.26.3 runs of ModernVBERT/bimodernvbert at revision 09ed8566; its V3 scores are its MTEB results.

ViDoRe V1 (nDCG@5)	BiModernVBERT	BiUltraVBERT
ArxivQA	62.7	69.6
DocVQA	31.1	36.0
InfoVQA	72.5	81.0
TabFQuAD	64.9	80.4
TAT-DQA	49.2	42.4
ShiftProject	30.3	49.2
SyntheticDocQA AI	80.2	86.1
SyntheticDocQA Energy	76.2	86.2
SyntheticDocQA Government Reports	81.2	87.5
SyntheticDocQA Healthcare Industry	87.4	89.7
Mean	63.6	70.8

ViDoRe V2 (nDCG@5)	BiModernVBERT	BiUltraVBERT
ESG Reports	22.9	39.6
Economics Reports	29.5	42.3
BioMedical Lectures	19.8	39.5
ESG Reports HL	28.0	42.0
Mean	25.0	40.9

ViDoRe V3 (nDCG@10)	BiModernVBERT	BiUltraVBERT
Computer Science	28.0	49.1
Energy	16.4	32.0
Finance EN	14.2	34.2
Finance FR	6.3	16.3
HR	14.7	32.9
Industrial	8.0	18.7
Pharmaceuticals	21.4	43.9
Physics	14.3	26.3
Nuclear (private)	10.0	21.3
Telecom (private)	17.8	42.1
Mean, public	15.4	31.7
Mean, private	13.9	31.7
Mean, all	15.1	31.7