File size: 5,603 Bytes
785e185 e9046ac 8613884 e9046ac 8613884 e9046ac 8613884 4894b7d 8613884 4894b7d 8613884 4894b7d 8613884 4894b7d 8613884 4894b7d 8613884 e9046ac 8613884 e9046ac 8613884 e9046ac 8613884 e9046ac 8613884 e9046ac 8613884 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 |
---
license: apache-2.0
base_model:
- Qwen/Qwen3-VL-4B-Instruct
pipeline_tag: visual-document-retrieval
library_name: transformers
tags:
- transformers
- multimodal_embedding
- embedding
- colpali
- multilingual-embedding
- colqwen3
---
# OpenSearch-AI/Ops-Colqwen3-4B
**Ops-Colqwen3-4B** is a ColPali-style multimodal embedding model based on the **Qwen3-VL-4B-Instruct** architecture, developed and open-sourced by the Alibaba Cloud OpenSearch-AI team. It maps text queries and visual documents such as images and PDF pages into a unified, aligned **multi-vector embedding space**, enabling highly effective retrieval of visual documents.
The model is trained using a multi-stage strategy that combines large-scale text-based retrieval datasets with diverse visual document data. This hybrid training approach significantly enhances its capability to handle complex document understanding and retrieval tasks. On the Vidore v1–v3 benchmarks, **Ops-Colqwen3-4B** achieves **state-of-the-art results** among models of comparable size.
## Key Features
- **Model size**: 4 billion parameters
- **Multimodal alignment**: Enables fine-grained semantic alignment between text and images or PDF pages
- **Multi-vector embeddings**: Following the ColPali design, each input generates multiple context-aware embedding vectors; similarity is computed using **MaxSim**, enabling high-precision matching
- **Scalable embedding dimensions**: Supports embedding dimensions up to **2,560** during inference via an extended projection head, enabling **higher retrieval accuracy** through more expressive representations. Lower-dimensional prefixes (e.g., the first 128 or 320 dimensions) remain highly effective for lightweight applications.
- **Multilingual support**: Covers over 30 languages
- **Context length**: Supports up to **32,000 tokens**
- **Visual token capacity**: Handles up to **1,280 visual tokens** per page input.
## Usage
**Requirements**
```
pillow
transformers>=4.57.0
qwen-vl-utils>=0.0.14
torch==2.8.0
```
**Basic Usage**
```python
import torch
from PIL import Image
from scripts.ops_colqwen3_embedder import OpsColQwen3Embedder
images = [Image.new("RGB", (32, 32), color="white"), Image.new("RGB", (16, 16), color="black")]
queries = ["Is attention really all you need?", "What is the amount of bananas farmed in Salvador?"]
embedder = OpsColQwen3Embedder(
model_name="OpenSearch-AI/Ops-Colqwen3-4B",
dims=2560,
dtype=torch.float16,
attn_implementation="flash_attention_2",
)
query_embeddings = embedder.encode_queries(queries)
image_embeddings = embedder.encode_images(images)
print(query_embeddings[0].shape, image_embeddings[0].shape) # (23, 2560) (18, 2560)
scores = embedder.compute_scores(query_embeddings, image_embeddings)
print(f"Scores:\n{scores}")
```
## Model Performance
### Vidore v1 + v2 (NDCG@5)
| Model | Dim | Vidore v1+v2 | Vidore v2 | Vidore v1 |
|--------------------------------------------|------|--------------|-----------|-----------|
| **Ops-Colqwen3-4B** | 2560 | **84.87** | **68.7** | **91.4** |
| **Ops-Colqwen3-4B** | 1280 | 84.71 | 68.2 | 91.3 |
| **Ops-Colqwen3-4B** | 640 | 84.39 | 67.7 | 91.1 |
| **Ops-Colqwen3-4B** | 320 | 84.12 | 67.0 | 91.0 |
| **Ops-Colqwen3-4B** | 128 | 84.04 | 66.9 | 90.9 |
| tomoro-colqwen3-embed-8b | 320 | 83.52 | 65.4 | 90.8 |
| EvoQwen2.5-VL-Retriever-7B-v1 | 128 | 83.41 | 65.2 | 90.7 |
| tomoro-colqwen3-embed-4b | 320 | 83.18 | 64.7 | 90.6 |
| llama-nemoretriever-colembed-3b-v1 | 3072 | 83.10 | 63.3 | 91.0 |
| SauerkrautLM-ColQwen3-8b-v0.1 | 128 | 82.91 | 62.5 | 91.1 |
| EvoQwen2.5-VL-Retriever-3B-v1 | 128 | 82.76 | 63.0 | 90.7 |
| SauerkrautLM-ColQwen3-4b-v0.1 | 128 | 81.97 | 59.9 | 90.8 |
| jina-embedding-v4 | 128 | 81.17 | 58.2 | 90.4 |
### Vidore v3 (NDCG@10)
| Model | Dim | PUB AVG |
|--------------------------------------------|------|---------|
| **Ops-Colqwen3-4B** | 2560 | 61.27 |
| **Ops-Colqwen3-4B** | 1280 | **61.32** |
| **Ops-Colqwen3-4B** | 640 | 61.21 |
| **Ops-Colqwen3-4B** | 320 | 60.88 |
| **Ops-Colqwen3-4B** | 128 | 60.23 |
| tomoro-colqwen3-embed-4b | 320 | 60.19 |
| SauerkrautLM-ColQwen3-8b-v0.1 | 128 | 58.55 |
| jina-embedding-v4 | 128 | 57.54 |
| llama-nemoretriever-colembed-3b-v1 | 3072 | 57.07 |
| SauerkrautLM-ColQwen3-4b-v0.1 | 128 | 56.03 |
> With only **128 dimensions**, `Ops-Colqwen3-4B` outperforms other 4B-parameter models such as `tomoro-colqwen3-embed-4b`, making it well-suited for latency- and memory-constrained applications.
## Citation
If you use this model in your work, please cite:
```bibtex
@misc{ops_colqwen3_4b,
author = {{OpenSearch-AI}},
title = {{Ops-Colqwen3: State-of-the-Art Multimodal Embedding Model for Visual Document Retrieval}},
year = {2026},
howpublished = {\url{https://huggingface.co/OpenSearch-AI/Ops-Colqwen3-4B}},
}
``` |