README.md

---
license: apache-2.0
language:
- en
tags:
- sparse-retrieval
- splade
- visual-document-retrieval
- multimodal
- information-retrieval
- inference-free
- sentence-transformers
- sparse-encoder
pipeline_tag: feature-extraction
library_name: sentence-transformers
---

<p align="center">
  <img src="v-splade-logo.png" alt="V-SPLADE" width="480"/>
</p>

# V-SPLADE: Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search

**Paper:** [arXiv:2605.30917](https://arxiv.org/abs/2605.30917) &nbsp;·&nbsp; **Code:** [github.com/naver/v-splade](https://github.com/naver/v-splade) &nbsp;·&nbsp; **Demo:** [🤗 Space](https://huggingface.co/spaces/hugging-apps/v-splade-document-retrieval)

> **This repository hosts the `Quality` variant** (higher retrieval quality). For the lower-FLOPs checkpoint, see [`naver/v-splade-efficient`](https://huggingface.co/naver/v-splade-efficient).
>
> 🚀 **Try it live:** an [interactive demo](https://huggingface.co/spaces/hugging-apps/v-splade-document-retrieval) of this model, kindly contributed by [Apolinário](https://huggingface.co/multimodalart) and the Hugging Face open-source team.

## Model Summary

**V-SPLADE** is a **0.25B (250M) inference-free sparse retriever** for visual-document retrieval — retrieving image-based document pages (rendered PDFs, slides, scanned reports) from a text query.

- **Inference-free** — queries are resolved by a learned Bag-of-Words lookup with **no neural query encoding at serving time**, so retrieval runs on a standard inverted index (Pyserini / PISA) without a GPU.
- **Direct visual embedding** — document pages are encoded directly into sparse vectors, building indexes **over 20× faster** than caption- or OCR-based text-extraction pipelines.

## Benchmark Performance

### Six visual-document benchmarks (NDCG@5)

| Model | Size | ViDoRe v1 | v2 | v3 | VisRAG | VisDoc OOD | IRPAPERS | Avg |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| BiModernVBERT (dense) | 0.25B | 67.6 | 35.7 | 28.9 | 60.5 | 53.4 | 31.8 | 46.3 |
| BM25 (caption, Qwen3-VL) | — | 67.5 | 44.1 | 38.3 | 76.5 | 58.0 | 38.4 | 53.8 |
| BM25 (unstructured OCR) | — | 68.2 | 41.7 | 38.7 | 61.1 | 51.2 | 65.7 | 54.4 |
| [**V-SPLADE Quality**](https://huggingface.co/naver/v-splade-quality) | 0.25B | **77.4** | **49.9** | **40.9** | 76.4 | **61.7** | 54.0 | **60.1** |
| [**V-SPLADE Efficient**](https://huggingface.co/naver/v-splade-efficient) | 0.25B | 74.6 | 46.6 | 37.6 | 73.0 | 59.5 | 47.1 | 56.4 |

V-SPLADE Quality improves average NDCG@5 by **+13.8pp** over the same-scale dense baseline (BiModernVBERT) and by up to **+6.3pp** over the OCR/caption BM25 baselines.

### Production-scale retrieval (18.7M-document corpus)

| Model | R@5 | R@100 | Query latency |
| --- | ---: | ---: | --- |
| BiModernVBERT (same-scale dense) | 0.090 | 0.299 | ~HNSW |
| **V-SPLADE** | **0.228** | **0.520** | ~HNSW approx |

V-SPLADE more than **doubles R@5** over the same-backbone dense retriever at production scale, and retains recall more robustly as the corpus grows from 500K to 18.7M pages.

### Document encoding throughput

| Method | Pages/sec |
| --- | ---: |
| **V-SPLADE (ours)** | **20.19** |
| Qwen3-VL-30B-A3B caption (vLLM, eff. 3B) | 0.83 |
| Unstructured OCR (Tesseract hi_res) | 0.90 |

Measured on a single H100 GPU with 4 CPU cores, using 1,000 sampled documents across the six benchmarks. V-SPLADE is **over 20× faster** than caption- or OCR-based text-extraction pipelines for index building.

## Quick Start

### Using Sentence Transformers

Install Sentence Transformers (v5.6.0 or later) with image support, and note that the ModernVBERT backbone requires `transformers>=5.3.0`:

```bash
pip install "sentence_transformers[image]>=5.6.0"
```

Queries are encoded with the inference-free Li-LSR lookup (no transformer forward pass), while document page images run through the full model:

```python
from sentence_transformers import SparseEncoder

model = SparseEncoder("naver/v-splade-quality", trust_remote_code=True)

queries = ["send signed forms", "records office"]
documents = ["https://raw.githubusercontent.com/naver/v-splade/main/examples/sample_page.png"]

query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# torch.Size([2, 50368]) torch.Size([1, 50368])

similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.9843],
#         [0.5935]], device='cuda:0')

# Inspect the top activated tokens of the page image
decoded = model.decode(document_embeddings[0], top_k=5)
print([(token.replace("Ġ", " ").strip(), round(weight, 3)) for token, weight in decoded])
# [('dog', 1.836), ('dogs', 1.664), ('puppy', 1.586), ('Records', 1.578), ('Bennett', 1.508)]
```

Images can be passed as PIL images, local paths, URLs (as above), or together with text as `{"image": ..., "text": ...}`. Plain text documents are also supported: `model.encode_document(["some passage text"])`. The model runs in bfloat16 by default. You can pass `model_kwargs={"torch_dtype": "float32"}` for full precision.

### Using the reference implementation

Install (see the [code repository](https://github.com/naver/v-splade) for full instructions):

```bash
git clone https://github.com/naver/v-splade.git
cd v-splade
python -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install torch torchvision torchaudio \
    --index-url https://download.pytorch.org/whl/cu128
grep -v -E '^(torch|flash-attn)==' requirements.txt > requirements_filtered.txt
pip install -r requirements_filtered.txt
pip install flash-attn==2.8.3 --no-build-isolation --no-cache-dir
```

#### Single-image inference (minimal example)

The shortest path to seeing V-SPLADE work on your own page image — encode one image into a sparse vocabulary vector, inspect the top-activated tokens, and score a text query against it:

```bash
python examples/quickstart.py \
    --hf_dir  naver/v-splade-quality \
    --image   examples/sample_page.png \
    --queries "send signed forms" "records office"
```

Expected output (against the sample page):

```
[2/3] Encoding image: examples/sample_page.png
      sparse vector shape=(50368,)  nnz=552  max=1.836
      Top-10 activated tokens:
          1.836   'dog'
          1.672   'dogs'
          1.586   'puppy'
          1.570   'Records'
          1.523   'Bennett'
          ...
[3/3] Query-image similarity scores
        score=  0.997   query='send signed forms'
          top matches: forms(0.438), send(0.403), signed(0.156)
        score=  0.594   query='records office'
          top matches: office(0.594)
```

## License

This model and the accompanying code are released under the **Apache License 2.0**. See [`LICENSE`](https://huggingface.co/naver/v-splade-quality/blob/main/LICENSE) in the repository for the full text.

Base model ([ModernVBERT/modernvbert](https://huggingface.co/ModernVBERT/modernvbert)) and caption generator ([Qwen3-VL-30B-A3B](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct)) are subject to their own licenses; please review them before redistribution or commercial use.

**Training data.** This model was trained on [vidore/colpali_train_set](https://huggingface.co/datasets/vidore/colpali_train_set) and [rlhn/rlhn-680K](https://huggingface.co/datasets/rlhn/rlhn-680K). `rlhn/rlhn-680K` is distributed under **CC BY-SA 4.0**. `vidore/colpali_train_set` is a collection of multiple source datasets, each of which remains under its own original license.

## Citation

```bibtex
@misc{cho2026vsplade,
  title         = {Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search},
  author        = {Cho, Gyu-Hwung and Lee, Youngjune and Jeong, Kiyoon and Lee, Siyoung and Han, Sanggyu and Dejean, Herv{\'e} and Clinchant, St{\'e}phane and Hwang, Seung-won},
  year          = {2026},
  eprint        = {2605.30917},
  archivePrefix = {arXiv},
  primaryClass  = {cs.IR}
}
```

## Authors

Gyu-Hwung Cho (NAVER Corp. & Seoul National University), Youngjune Lee, Kiyoon Jeong, Siyoung Lee, Sanggyu Han (NAVER Corp.), Hervé Dejean, Stéphane Clinchant (Naver Labs Europe), Seung-won Hwang (Seoul National University, corresponding).

## Contact

Issues and pull requests welcome at [github.com/naver/v-splade](https://github.com/naver/v-splade). For research questions, contact the author at `gyuhwung.cho@navercorp.com`.