EVIE-Preview-4.5B
Browse files- .gitattributes +0 -1
- .gitignore +7 -0
- README.md +153 -104
.gitattributes
CHANGED
|
@@ -1,2 +1 @@
|
|
| 1 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 2 |
-
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 1 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
|
|
.gitignore
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.safetensors
|
| 2 |
+
__pycache__/
|
| 3 |
+
*.pyc
|
| 4 |
+
local.env
|
| 5 |
+
results.json
|
| 6 |
+
reproduce_*.log
|
| 7 |
+
vidore/
|
README.md
CHANGED
|
@@ -7,6 +7,7 @@ language:
|
|
| 7 |
- es
|
| 8 |
- pt
|
| 9 |
- zh
|
|
|
|
| 10 |
- multilingual
|
| 11 |
license: apache-2.0
|
| 12 |
library_name: colpali-engine
|
|
@@ -22,6 +23,7 @@ tags:
|
|
| 22 |
- vidore
|
| 23 |
- document-retrieval
|
| 24 |
- multimodal
|
|
|
|
| 25 |
base_model:
|
| 26 |
- Qwen/Qwen3.5-4B
|
| 27 |
datasets:
|
|
@@ -32,116 +34,152 @@ inference: false
|
|
| 32 |
|
| 33 |
<div align="center">
|
| 34 |
|
| 35 |
-
# EVIE-Preview-4.5B
|
| 36 |
|
| 37 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
<p align="center">
|
| 40 |
-
<a href="LICENSE.txt"><img src="https://img.shields.io/badge/License-
|
| 41 |
-
<a href="https://huggingface.co/Qwen/Qwen3.5-4B"><img src="https://img.shields.io/badge/Base
|
| 42 |
-
<a href="
|
| 43 |
-
<a href="
|
| 44 |
-
<a href="https://github.com/
|
| 45 |
-
<a href="https://huggingface.co/tencent/EVIE-Preview-4.5B"><img src="https://img.shields.io/badge/Hugging%20Face-Model-FFD21E.svg" alt="Hugging Face"></a>
|
| 46 |
-
<a href="https://github.com/Tencent/EVIE-Preview-4.5B"><img src="https://img.shields.io/badge/GitHub-Source-181717.svg" alt="GitHub"></a>
|
| 47 |
</p>
|
| 48 |
|
| 49 |
-
[
|
| 50 |
|
| 51 |
</div>
|
| 52 |
|
| 53 |
---
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
-
|
| 62 |
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
-
|
| 66 |
-
- **🏆 SOTA on ViDoRe Benchmarks**: Outperforms larger 8B models on ViDoRe V3 (leading **7 of 8** public domains) and delivers top average accuracy on ViDoRe V1+V2 (**85.93** nDCG@5).
|
| 67 |
-
- **🌐 Robust Multilingual & Multi-Format**: Strong zero-shot generalization across diverse languages (EN, FR, DE, IT, ES, PT, ZH, etc.) and visual formats (charts, tables, scientific reports, financial filings).
|
| 68 |
-
- **⚡ Seamless ColPali Compatibility**: Fully integrated with the standard `colpali-engine` ecosystem and late-interaction scoring pipelines.
|
| 69 |
|
| 70 |
-
|
| 71 |
|
| 72 |
-
##
|
| 73 |
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
-
|
| 83 |
-
2. **Compact Projection**: Projects contextual token states directly into 128-dimensional representations.
|
| 84 |
-
3. **Late-Interaction Retrieval**: Calculates similarity via token-level MaxSim operator across query tokens and document visual tokens.
|
| 85 |
|
| 86 |
---
|
| 87 |
|
| 88 |
-
##
|
| 89 |
|
| 90 |
-
|
| 91 |
|
| 92 |
-
| Model |
|
| 93 |
-
| :--- | :---: | :---: |
|
| 94 |
-
| **EVIE-Preview-4.5B** | **
|
| 95 |
-
|
|
| 96 |
-
|
|
| 97 |
-
|
|
| 98 |
-
|
|
| 99 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
|
| 101 |
-
*
|
| 102 |
|
| 103 |
---
|
| 104 |
|
| 105 |
-
##
|
| 106 |
|
| 107 |
-
|
| 108 |
|
| 109 |
-
|
|
| 110 |
-
| :--- |
|
| 111 |
-
|
|
| 112 |
-
|
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
| colnomic-embed-multimodal-7b | 57.64 | 76.20 | 63.58 | 56.57 | 45.46 | 58.67 | 50.13 | 62.26 | 48.25 |
|
| 118 |
-
| jina-embeddings-v4 | 57.54 | 71.81 | 63.50 | 59.30 | 46.10 | 59.53 | 50.38 | 63.09 | 46.63 |
|
| 119 |
|
| 120 |
-
|
| 121 |
|
| 122 |
---
|
| 123 |
|
| 124 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
|
| 126 |
-
|
| 127 |
-
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 128 |
-
| **EVIE-Preview-4.5B** | **85.9** | 91.5 | 62.9 | 93.0 | **94.0** | **100.0** | **99.0** | **98.9** | 98.9 | **97.5** | 81.6 | **71.0** | **80.1** | **66.3** | **68.3** |
|
| 129 |
-
| Ops-Colqwen3-4B | 84.9 | 91.8 | 66.5 | 94.0 | 90.8 | 99.6 | 97.3 | 98.0 | 99.6 | 93.6 | 82.4 | 65.5 | 78.6 | 66.0 | 64.5 |
|
| 130 |
-
| nemotron-colembed-vl-8b-v2 | 84.8 | 93.1 | 68.1 | 94.6 | 93.3 | 100.0 | 97.9 | 98.9 | 99.6 | 97.7 | 83.4 | 66.2 | 73.2 | 60.6 | 60.8 |
|
| 131 |
-
| nemotron-colembed-vl-4b-v2 | 83.9 | 92.0 | 67.4 | 93.3 | 92.3 | 99.3 | 96.2 | 98.0 | 98.5 | 98.1 | 81.2 | 64.3 | 71.4 | 61.5 | 60.8 |
|
| 132 |
-
| colqwen3.5-4.5B-v3 | 83.7 | 91.9 | 66.6 | 93.6 | 90.2 | 100.0 | 97.1 | 97.3 | 98.9 | 95.9 | 84.0 | 65.3 | 73.8 | 58.0 | 59.9 |
|
| 133 |
-
| llama-nemotron-colembed-vl-3b-v2 | 83.6 | 90.4 | 67.2 | 94.7 | 92.0 | 100.0 | 98.0 | 98.0 | 98.9 | 97.3 | 81.0 | 63.2 | 73.1 | 58.6 | 58.6 |
|
| 134 |
-
| tomoro-colqwen3-embed-8b | 83.5 | 91.2 | 66.4 | 94.5 | 87.9 | 99.3 | 96.7 | 97.6 | 99.1 | 94.2 | 80.9 | 65.5 | 76.0 | 60.7 | 59.5 |
|
| 135 |
-
| EvoQwen2.5-VL-Retriever-7B-v1 | 83.4 | 91.5 | 65.1 | 94.1 | 88.8 | 99.6 | 96.6 | 96.3 | 98.9 | 93.6 | 82.3 | 65.2 | 77.0 | 59.7 | 59.1 |
|
| 136 |
-
| tomoro-colqwen3-embed-4b | 83.2 | 90.6 | 66.3 | 94.3 | 87.4 | 99.3 | 96.9 | 97.2 | 99.6 | 94.3 | 79.9 | 65.4 | 74.6 | 62.4 | 56.3 |
|
| 137 |
-
| llama-nemoretriever-colembed-3b-v1 | 83.1 | 88.4 | 66.2 | 94.9 | 90.7 | 99.6 | 96.6 | 97.8 | 99.3 | 95.9 | 80.6 | 62.7 | 75.4 | 57.4 | 57.8 |
|
| 138 |
-
| SauerkrautLM-ColQwen3-8b-v0.1 | 82.9 | 93.8 | 64.7 | 94.5 | 90.4 | 98.6 | 96.5 | 96.8 | 99.3 | 92.2 | 84.0 | 63.3 | 70.8 | 57.9 | 58.0 |
|
| 139 |
|
| 140 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
|
| 142 |
---
|
| 143 |
|
| 144 |
-
## Quick Start
|
| 145 |
|
| 146 |
### Installation
|
| 147 |
|
|
@@ -156,7 +194,6 @@ import torch
|
|
| 156 |
from PIL import Image
|
| 157 |
from colpali_engine.models import ColQwen3_5, ColQwen3_5Processor
|
| 158 |
|
| 159 |
-
# Use the Hugging Face model repository or local directory
|
| 160 |
model_id = "tencent/EVIE-Preview-4.5B"
|
| 161 |
|
| 162 |
# 1. Load model and enable bidirectional attention
|
|
@@ -188,74 +225,86 @@ scores = processor.score(query_embeddings, image_embeddings)
|
|
| 188 |
print("Late-interaction retrieval scores:", scores)
|
| 189 |
```
|
| 190 |
|
| 191 |
-
> ⚠️
|
| 192 |
-
|
| 193 |
-
### CLI Scoring Tool
|
| 194 |
|
| 195 |
-
|
| 196 |
|
| 197 |
```bash
|
| 198 |
-
python infer.py --query "Quarterly revenue report" --image
|
| 199 |
```
|
| 200 |
|
| 201 |
---
|
| 202 |
|
| 203 |
-
## Reproducing
|
| 204 |
|
| 205 |
-
|
| 206 |
|
| 207 |
```bash
|
| 208 |
bash reproduce.sh
|
| 209 |
```
|
| 210 |
|
| 211 |
-
|
| 212 |
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
- **Target Aggregates**:
|
| 219 |
|
| 220 |
```text
|
| 221 |
-
ViDoRe V1 nDCG@5 91.
|
| 222 |
-
ViDoRe V2 nDCG@5
|
| 223 |
-
ViDoRe V1+V2 nDCG@5 85.
|
| 224 |
-
ViDoRe V3 public nDCG@10 64.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 225 |
```
|
| 226 |
|
| 227 |
---
|
| 228 |
|
| 229 |
-
## Training Details
|
|
|
|
|
|
|
| 230 |
|
| 231 |
-
|
| 232 |
|
| 233 |
-
|
| 234 |
|
| 235 |
-
-
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
|
| 239 |
-
-
|
| 240 |
|
| 241 |
---
|
| 242 |
|
| 243 |
-
## Acknowledgements
|
| 244 |
|
| 245 |
-
- Built
|
| 246 |
- Powered by the [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) vision-language backbone.
|
| 247 |
- Evaluated on the [ViDoRe Benchmark](https://huggingface.co/vidore) family.
|
| 248 |
|
| 249 |
---
|
| 250 |
|
| 251 |
-
## Citation
|
| 252 |
|
| 253 |
```bibtex
|
| 254 |
@misc{tencent2026evie,
|
| 255 |
-
title = {EVIE-Preview-4.5B},
|
| 256 |
author = {{Tencent}},
|
| 257 |
year = {2026},
|
| 258 |
-
howpublished = {\url{https://huggingface.co/tencent/EVIE-Preview-4.5B}}
|
| 259 |
-
note = {Multilingual visual document retrieval with compact multi-vector embeddings}
|
| 260 |
}
|
| 261 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
- es
|
| 8 |
- pt
|
| 9 |
- zh
|
| 10 |
+
- ja
|
| 11 |
- multilingual
|
| 12 |
license: apache-2.0
|
| 13 |
library_name: colpali-engine
|
|
|
|
| 23 |
- vidore
|
| 24 |
- document-retrieval
|
| 25 |
- multimodal
|
| 26 |
+
- state-of-the-art
|
| 27 |
base_model:
|
| 28 |
- Qwen/Qwen3.5-4B
|
| 29 |
datasets:
|
|
|
|
| 34 |
|
| 35 |
<div align="center">
|
| 36 |
|
| 37 |
+
# 🏆 EVIE-Preview-4.5B
|
| 38 |
|
| 39 |
+
### **Rank #1 on ViDoRe V3 · Rank #1 on ViDoRe V1+V2**
|
| 40 |
+
|
| 41 |
+
**The most accurate visual document retriever, with native 128-dimensional token vectors.**
|
| 42 |
+
|
| 43 |
+
<p align="center">
|
| 44 |
+
<a href="#-vidore-v3--rank-1"><img src="https://img.shields.io/badge/🥇_ViDoRe_V3-65.36_·_Rank_%231-FFD700?style=for-the-badge&labelColor=1a1a2e" alt="ViDoRe V3 Rank 1"></a>
|
| 45 |
+
<a href="#-vidore-v1--v2--rank-1"><img src="https://img.shields.io/badge/🥇_ViDoRe_V1+V2-85.77_·_Rank_%231-FFD700?style=for-the-badge&labelColor=1a1a2e" alt="ViDoRe V1+V2 Rank 1"></a>
|
| 46 |
+
</p>
|
| 47 |
+
|
| 48 |
+
<p align="center">
|
| 49 |
+
<a href="#-index-cost"><img src="https://img.shields.io/badge/Token_Dim-128D_native-39d4bd?style=flat-square&logo=vectorworks&logoColor=white" alt="128D"></a>
|
| 50 |
+
<a href="#-index-cost"><img src="https://img.shields.io/badge/Index-420.5_GiB_%2F_1M_pages-45c8f5?style=flat-square&logo=databricks&logoColor=white" alt="Index Cost"></a>
|
| 51 |
+
<a href="#-model-footprint"><img src="https://img.shields.io/badge/Params-4.54B-b285f7?style=flat-square&logo=pytorch&logoColor=white" alt="4.54B"></a>
|
| 52 |
+
<a href="#-multilingual"><img src="https://img.shields.io/badge/Languages-7_query_langs-f4c45e?style=flat-square&logo=googletranslate&logoColor=white" alt="Multilingual"></a>
|
| 53 |
+
</p>
|
| 54 |
|
| 55 |
<p align="center">
|
| 56 |
+
<a href="LICENSE.txt"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg?style=flat-square" alt="License"></a>
|
| 57 |
+
<a href="https://huggingface.co/Qwen/Qwen3.5-4B"><img src="https://img.shields.io/badge/Base-Qwen3.5--4B-purple.svg?style=flat-square" alt="Base Model"></a>
|
| 58 |
+
<a href="https://github.com/illuin-tech/colpali"><img src="https://img.shields.io/badge/Framework-ColPali_Engine-orange.svg?style=flat-square" alt="Framework"></a>
|
| 59 |
+
<a href="https://huggingface.co/tencent/EVIE-Preview-4.5B"><img src="https://img.shields.io/badge/🤗_Hugging_Face-Model-FFD21E.svg?style=flat-square" alt="Hugging Face"></a>
|
| 60 |
+
<a href="https://github.com/Tencent/EVIE-Preview-4.5B"><img src="https://img.shields.io/badge/GitHub-Source-181717.svg?style=flat-square&logo=github" alt="GitHub"></a>
|
|
|
|
|
|
|
| 61 |
</p>
|
| 62 |
|
| 63 |
+
[🏆 Results](#-vidore-v3--rank-1) • [💾 Index Cost](#-index-cost) • [⚡ Quick Start](#-quick-start) • [🔬 Reproducing](#-reproducing) • [🧠 Architecture](#-architecture) • [📚 Citation](#-citation)
|
| 64 |
|
| 65 |
</div>
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
+
## 🥇 ViDoRe V3 — Rank #1
|
| 70 |
|
| 71 |
+
8 public domains × 6 query languages, nDCG@10.
|
| 72 |
|
| 73 |
+
| # | Model | Params | Token Dim | **V3 public** |
|
| 74 |
+
| :---: | :--- | ---: | ---: | ---: |
|
| 75 |
+
| 🥇 **1** | **EVIE-Preview-4.5B** | 4.54B | 128D | **65.36** |
|
| 76 |
+
| 🥈 2 | webAI-ColVec1.1-8b | 8.40B | 640D | 65.32 |
|
| 77 |
+
| 🥉 3 | webAI-ColVec1.1-4b | 4.54B | 640D | 63.90 |
|
| 78 |
+
| 4 | nemotron-colembed-vl-8b-v2 | 8B | — | 63.54 |
|
| 79 |
+
| 5 | tomoro-colqwen3-embed-8b | 8B | — | 61.60 |
|
| 80 |
+
| 6 | nemotron-colembed-vl-4b-v2 | 4B | — | 61.42 |
|
| 81 |
+
| 7 | tomoro-colqwen3-embed-4b | 4B | — | 60.16 |
|
| 82 |
+
| 8 | llama-nemotron-colembed-vl-3b-v2 | 3B | — | 59.70 |
|
| 83 |
+
| 9 | colnomic-embed-multimodal-7b | 7B | — | 57.64 |
|
| 84 |
+
| 10 | jina-embeddings-v4 | ~3.8B | — | 57.54 |
|
| 85 |
|
| 86 |
+
### Two deployment tiers, one checkpoint
|
| 87 |
|
| 88 |
+
| Visual tokens / page | **V3 public** | Vectors / page | Raw index / 1M pages (BF16) |
|
| 89 |
+
| :--- | ---: | ---: | ---: |
|
| 90 |
+
| 768 | 64.56 | **751.62** | **179.2 GiB** |
|
| 91 |
+
| 1,792 | **65.36** | 1,763.58 | 420.5 GiB |
|
| 92 |
|
| 93 |
+
Both rows are the same weights — no retraining, no re-export. Pick whichever tier fits your compute budget; 7 of the 8 domains keep improving as the page budget grows.
|
|
|
|
|
|
|
|
|
|
| 94 |
|
| 95 |
+
The lighter tier holds a million pages in under 180 GiB.
|
| 96 |
|
| 97 |
+
### Per-domain breakdown
|
| 98 |
|
| 99 |
+
| Model | **Avg** | CompSci | Energy | Finance EN | Finance FR | HR | Industrial | Pharma | Physics |
|
| 100 |
+
| :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 101 |
+
| 🥇 **EVIE-Preview-4.5B** | **65.36** | 80.65 | 71.36 | 70.50 | 54.44 | 67.34 | 58.76 | 69.20 | 50.62 |
|
| 102 |
+
| webAI-ColVec1.1-8b | 65.32 | 80.08 | 70.12 | 71.90 | 54.87 | 68.55 | 57.65 | 67.88 | 51.50 |
|
| 103 |
+
| webAI-ColVec1.1-4b | 63.90 | 80.34 | 69.50 | 69.18 | 53.13 | 66.90 | 56.36 | 67.25 | 51.24 |
|
| 104 |
+
| nemotron-colembed-vl-8b-v2 | 63.54 | 79.30 | 69.82 | 67.29 | 51.54 | 66.32 | 56.03 | 67.19 | 50.84 |
|
| 105 |
+
| tomoro-colqwen3-embed-8b | 61.60 | 75.35 | 68.41 | 65.08 | 49.10 | 63.98 | 54.41 | 66.36 | 50.13 |
|
| 106 |
+
| nemotron-colembed-vl-4b-v2 | 61.42 | 78.56 | 67.48 | 65.02 | 49.01 | 62.39 | 53.91 | 66.10 | 48.86 |
|
| 107 |
+
| llama-nemotron-colembed-vl-3b-v2 | 59.70 | 77.09 | 64.88 | 64.23 | 44.41 | 62.28 | 51.71 | 66.04 | 46.93 |
|
| 108 |
+
| colnomic-embed-multimodal-7b | 57.64 | 76.20 | 63.58 | 56.57 | 45.46 | 58.67 | 50.13 | 62.26 | 48.25 |
|
| 109 |
+
| jina-embeddings-v4 | 57.54 | 71.81 | 63.50 | 59.30 | 46.10 | 59.53 | 50.38 | 63.09 | 46.63 |
|
| 110 |
|
| 111 |
+
*EVIE rows measured with [`reproduce.sh`](reproduce.sh). Comparison rows are the vendors' published ViDoRe V3 public scores.*
|
|
|
|
|
|
|
| 112 |
|
| 113 |
---
|
| 114 |
|
| 115 |
+
## 🥇 ViDoRe V1 + V2 — Rank #1
|
| 116 |
|
| 117 |
+
14 tasks, nDCG@5. **First place on the classic boards too.**
|
| 118 |
|
| 119 |
+
| # | Model | **Avg** | ArxivQA | DocVQA | InfoVQA | ShiftProj | SynAI | SynEnergy | SynGov | SynHealth | Tabfquad | Tatdqa | BioMed | ESGHL | ESG | Econ |
|
| 120 |
+
| :---: | :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 121 |
+
| 🥇 **1** | **EVIE-Preview-4.5B** | **85.77** | 90.73 | 64.53 | 93.26 | 93.85 | 99.63 | 98.26 | 98.89 | 98.89 | 97.32 | 81.93 | 70.17 | 79.84 | 64.95 | 68.53 |
|
| 122 |
+
| 🥈 2 | Ops-Colqwen3-4B | 84.90 | 91.80 | 66.50 | 94.00 | 90.80 | 99.60 | 97.30 | 98.00 | 99.60 | 93.60 | 82.40 | 65.50 | 78.60 | 66.00 | 64.50 |
|
| 123 |
+
| 🥉 3 | nemotron-colembed-vl-8b-v2 | 84.80 | 93.10 | 68.10 | 94.60 | 93.30 | 100.0 | 97.90 | 98.90 | 99.60 | 97.70 | 83.40 | 66.20 | 73.20 | 60.60 | 60.80 |
|
| 124 |
+
| 4 | nemotron-colembed-vl-4b-v2 | 83.90 | 92.00 | 67.40 | 93.30 | 92.30 | 99.30 | 96.20 | 98.00 | 98.50 | 98.10 | 81.20 | 64.30 | 71.40 | 61.50 | 60.80 |
|
| 125 |
+
| 5 | colqwen3.5-4.5B-v3 | 83.70 | 91.90 | 66.60 | 93.60 | 90.20 | 100.0 | 97.10 | 97.30 | 98.90 | 95.90 | 84.00 | 65.30 | 73.80 | 58.00 | 59.90 |
|
| 126 |
+
| 6 | llama-nemotron-colembed-vl-3b-v2 | 83.60 | 90.40 | 67.20 | 94.70 | 92.00 | 100.0 | 98.00 | 98.00 | 98.90 | 97.30 | 81.00 | 63.20 | 73.10 | 58.60 | 58.60 |
|
| 127 |
+
| 7 | tomoro-colqwen3-embed-8b | 83.50 | 91.20 | 66.40 | 94.50 | 87.90 | 99.30 | 96.70 | 97.60 | 99.10 | 94.20 | 80.90 | 65.50 | 76.00 | 60.70 | 59.50 |
|
| 128 |
+
| 8 | EvoQwen2.5-VL-Retriever-7B-v1 | 83.40 | 91.50 | 65.10 | 94.10 | 88.80 | 99.60 | 96.60 | 96.30 | 98.90 | 93.60 | 82.30 | 65.20 | 77.00 | 59.70 | 59.10 |
|
| 129 |
+
| 9 | tomoro-colqwen3-embed-4b | 83.20 | 90.60 | 66.30 | 94.30 | 87.40 | 99.30 | 96.90 | 97.20 | 99.60 | 94.30 | 79.90 | 65.40 | 74.60 | 62.40 | 56.30 |
|
| 130 |
+
| 10 | SauerkrautLM-ColQwen3-8b-v0.1 | 82.90 | 93.80 | 64.70 | 94.50 | 90.40 | 98.60 | 96.50 | 96.80 | 99.30 | 92.20 | 84.00 | 63.30 | 70.80 | 57.90 | 58.00 |
|
| 131 |
|
| 132 |
+
*Tasks 1–10: ViDoRe V1. Tasks 11–14: ViDoRe V2. Board aggregates: **V1 91.73** · **V2 70.87**.*
|
| 133 |
|
| 134 |
---
|
| 135 |
|
| 136 |
+
## 💾 Index Cost
|
| 137 |
|
| 138 |
+
Index size is what decides whether multi-vector retrieval actually ships. EVIE emits native **128D** token vectors, so the index stays compact at both page budgets.
|
| 139 |
|
| 140 |
+
| Raw BF16 index | 768 tokens/page | 1,792 tokens/page |
|
| 141 |
+
| :--- | ---: | ---: |
|
| 142 |
+
| 1M pages | **179.2 GiB** | 420.5 GiB |
|
| 143 |
+
| 10M pages | **1.8 TB** | 4.1 TB |
|
| 144 |
+
|
| 145 |
+
```text
|
| 146 |
+
1,763.58 vectors/page × 128 dim × 2 bytes × 1,000,000 pages ÷ 2^30 = 420.5 GiB
|
| 147 |
+
```
|
|
|
|
|
|
|
| 148 |
|
| 149 |
+
Scoring stays cheap for the same reason: MaxSim is a late-interaction dot product over the token vectors, so a narrower vector cuts the scoring work exactly as it cuts storage.
|
| 150 |
|
| 151 |
---
|
| 152 |
|
| 153 |
+
## 🧠 Architecture
|
| 154 |
+
|
| 155 |
+
```text
|
| 156 |
+
Text Query ───────► ColQwen3_5 (BiDir Attn) ─────► Query Token Embeddings (128D)
|
| 157 |
+
│
|
| 158 |
+
Late Interaction (MaxSim) ──► Relevance Score
|
| 159 |
+
│
|
| 160 |
+
Document Image ─────► ColQwen3_5 (Dynamic Vision) ──► Doc Token Embeddings (128D)
|
| 161 |
+
```
|
| 162 |
+
|
| 163 |
+
1. **Vision-Language Backbone** — `Qwen3.5-4B` with interleaved GatedDeltaNet linear attention and full attention.
|
| 164 |
+
2. **Compact Projection** — contextual token states projected directly into native 128-dimensional representations.
|
| 165 |
+
3. **Late-Interaction Retrieval** — token-level MaxSim between query tokens and document visual tokens.
|
| 166 |
+
|
| 167 |
+
### 🌍 Multilingual
|
| 168 |
|
| 169 |
+
Queries in **English, French, German, Italian, Spanish, Portuguese and Chinese**, retrieving over charts, tables, scientific reports, financial filings and scanned forms — including Japanese-language pages.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 170 |
|
| 171 |
+
### 📦 Model Footprint
|
| 172 |
+
|
| 173 |
+
| | Value |
|
| 174 |
+
| :--- | ---: |
|
| 175 |
+
| Parameters | 4.54B |
|
| 176 |
+
| Checkpoint (BF16) | 8.5 GB |
|
| 177 |
+
| Token embedding | 128D |
|
| 178 |
+
| Max visual tokens | 768 / 1,792 |
|
| 179 |
|
| 180 |
---
|
| 181 |
|
| 182 |
+
## ⚡ Quick Start
|
| 183 |
|
| 184 |
### Installation
|
| 185 |
|
|
|
|
| 194 |
from PIL import Image
|
| 195 |
from colpali_engine.models import ColQwen3_5, ColQwen3_5Processor
|
| 196 |
|
|
|
|
| 197 |
model_id = "tencent/EVIE-Preview-4.5B"
|
| 198 |
|
| 199 |
# 1. Load model and enable bidirectional attention
|
|
|
|
| 225 |
print("Late-interaction retrieval scores:", scores)
|
| 226 |
```
|
| 227 |
|
| 228 |
+
> ⚠️ Call `model.enable_bidirectional_attention()` and reset `model.rope_deltas = None` before every query forward pass. Both are required to reach the scores above.
|
|
|
|
|
|
|
| 229 |
|
| 230 |
+
### CLI
|
| 231 |
|
| 232 |
```bash
|
| 233 |
+
python infer.py --query "Quarterly revenue report" --image page_1.png --image page_2.png
|
| 234 |
```
|
| 235 |
|
| 236 |
---
|
| 237 |
|
| 238 |
+
## 🔬 Reproducing
|
| 239 |
|
| 240 |
+
Every number on this card is reproducible with the shipped script across all visible GPUs:
|
| 241 |
|
| 242 |
```bash
|
| 243 |
bash reproduce.sh
|
| 244 |
```
|
| 245 |
|
| 246 |
+
On the first run, [`download_data.py`](download_data.py) fetches the 22 public ViDoRe datasets (~55 GB) from Hugging Face. To reuse an existing directory:
|
| 247 |
|
| 248 |
+
```bash
|
| 249 |
+
bash reproduce.sh /path/to/vidore
|
| 250 |
+
```
|
| 251 |
+
|
| 252 |
+
### Target aggregates
|
|
|
|
| 253 |
|
| 254 |
```text
|
| 255 |
+
ViDoRe V1 nDCG@5 91.73 (10 tasks)
|
| 256 |
+
ViDoRe V2 nDCG@5 70.87 (4 tasks)
|
| 257 |
+
ViDoRe V1+V2 nDCG@5 85.77 (14 tasks)
|
| 258 |
+
ViDoRe V3 public nDCG@10 64.56 (8 domains x 6 languages, 768 visual tokens)
|
| 259 |
+
ViDoRe V3 public nDCG@10 65.36 (8 domains x 6 languages, 1792 visual tokens)
|
| 260 |
+
```
|
| 261 |
+
|
| 262 |
+
To score the 1,792-token tier directly:
|
| 263 |
+
|
| 264 |
+
```bash
|
| 265 |
+
python -m torch.distributed.run --nproc_per_node=$(nvidia-smi -L | wc -l) reproduce.py \
|
| 266 |
+
--boards v3 --max-visual-tokens 1792 --data-root /path/to/vidore
|
| 267 |
```
|
| 268 |
|
| 269 |
---
|
| 270 |
|
| 271 |
+
## 🎓 Training Details
|
| 272 |
+
|
| 273 |
+
EVIE was trained on approximately **0.8 million high-quality image-query pairs** spanning multilingual documents, technical reports, complex financial tables, infographics and document visual QA.
|
| 274 |
|
| 275 |
+
### Hard Negative Mining & Evidence Judging
|
| 276 |
|
| 277 |
+
Every mined negative is re-judged by a large multimodal judge before it reaches the loss:
|
| 278 |
|
| 279 |
+
- 🟢 Candidates that actually answer the query are **promoted to positives**.
|
| 280 |
+
- 🟡 Partially relevant or ambiguous candidates are **masked out of the loss**.
|
| 281 |
+
- 🔴 Only strictly irrelevant pages survive as **true hard negatives**.
|
| 282 |
+
|
| 283 |
+
Multi-positive rows are group-aware weighted by `1/positive_count` so that positives from the same query never penalise each other in-batch. Rows with empty queries, corrupted images or degraded text are dropped.
|
| 284 |
|
| 285 |
---
|
| 286 |
|
| 287 |
+
## 🙏 Acknowledgements
|
| 288 |
|
| 289 |
+
- Built on the [ColPali Engine](https://github.com/illuin-tech/colpali) by Illuin Technology.
|
| 290 |
- Powered by the [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) vision-language backbone.
|
| 291 |
- Evaluated on the [ViDoRe Benchmark](https://huggingface.co/vidore) family.
|
| 292 |
|
| 293 |
---
|
| 294 |
|
| 295 |
+
## 📚 Citation
|
| 296 |
|
| 297 |
```bibtex
|
| 298 |
@misc{tencent2026evie,
|
| 299 |
+
title = {EVIE-Preview-4.5B: Rank-1 Multilingual Visual Document Retrieval with 128-Dimensional Multi-Vector Embeddings},
|
| 300 |
author = {{Tencent}},
|
| 301 |
year = {2026},
|
| 302 |
+
howpublished = {\url{https://huggingface.co/tencent/EVIE-Preview-4.5B}}
|
|
|
|
| 303 |
}
|
| 304 |
```
|
| 305 |
+
|
| 306 |
+
<div align="center">
|
| 307 |
+
|
| 308 |
+
**🏆 Rank #1 on ViDoRe V3 · 🏆 Rank #1 on ViDoRe V1+V2 · 💾 native 128D token vectors**
|
| 309 |
+
|
| 310 |
+
</div>
|