Sentence Similarity
Safetensors
sentence-transformers
Italian
pylate
modernbert
colbert
late-interaction
italian
retrieval
information-retrieval
rag
multi-vector
text-embeddings-inference
Instructions to use enricollen/ItColBERT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use enricollen/ItColBERT with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("enricollen/ItColBERT") sentences = [ "Questa è una persona felice", "Questo è un cane felice", "Questa è una persona molto felice", "Oggi è una giornata di sole" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
Add new ColBERT model
Browse files
README.md
CHANGED
|
@@ -32,7 +32,8 @@ full development history: [github.com/enricollen/it-colbert](https://github.com/
|
|
| 32 |
- **What:** ColBERT-style multi-vector retriever, specialized on Italian.
|
| 33 |
- **Why it exists:** as far as I could find, there was no *Italian-only*
|
| 34 |
late-interaction retriever — see "Why I built this" below.
|
| 35 |
-
- **Size:** ModernBERT-base backbone (~
|
|
|
|
| 36 |
- **Best at:** short-to-medium Italian passages (search, RAG chunks, FAQ
|
| 37 |
retrieval). Weakest at long documents unless you use the chunking recipe
|
| 38 |
below.
|
|
@@ -165,7 +166,7 @@ this matters and how much it recovers.
|
|
| 165 |
| | |
|
| 166 |
|---|---|
|
| 167 |
| Base model | [`nickprock/Italian-ModernBERT-base-embed-mmarco-mnrl`](https://huggingface.co/nickprock/Italian-ModernBERT-base-embed-mmarco-mnrl) |
|
| 168 |
-
| Architecture | ModernBERT backbone (~
|
| 169 |
| Similarity | MaxSim (late interaction) |
|
| 170 |
| Query length | 32 tokens |
|
| 171 |
| Document length | 512 tokens (see the chunking note above for longer documents) |
|
|
@@ -295,6 +296,32 @@ it doesn't beat large multilingual dense embedders on most benchmarks —
|
|
| 295 |
matching those was never the goal; they're a different, much larger model
|
| 296 |
class.
|
| 297 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 298 |
**Protocol notes that matter for these numbers:**
|
| 299 |
|
| 300 |
- All late-interaction models above are indexed at the same document length
|
|
|
|
| 32 |
- **What:** ColBERT-style multi-vector retriever, specialized on Italian.
|
| 33 |
- **Why it exists:** as far as I could find, there was no *Italian-only*
|
| 34 |
late-interaction retriever — see "Why I built this" below.
|
| 35 |
+
- **Size:** ModernBERT-base backbone (~135M parameters), 128-dim token vectors —
|
| 36 |
+
the smallest model in the comparison table below, by a wide margin.
|
| 37 |
- **Best at:** short-to-medium Italian passages (search, RAG chunks, FAQ
|
| 38 |
retrieval). Weakest at long documents unless you use the chunking recipe
|
| 39 |
below.
|
|
|
|
| 166 |
| | |
|
| 167 |
|---|---|
|
| 168 |
| Base model | [`nickprock/Italian-ModernBERT-base-embed-mmarco-mnrl`](https://huggingface.co/nickprock/Italian-ModernBERT-base-embed-mmarco-mnrl) |
|
| 169 |
+
| Architecture | ModernBERT backbone (~135M params) → dense projection → 128-dim token vectors |
|
| 170 |
| Similarity | MaxSim (late interaction) |
|
| 171 |
| Query length | 32 tokens |
|
| 172 |
| Document length | 512 tokens (see the chunking note above for longer documents) |
|
|
|
|
| 296 |
matching those was never the goal; they're a different, much larger model
|
| 297 |
class.
|
| 298 |
|
| 299 |
+
### Quality per parameter
|
| 300 |
+
|
| 301 |
+
Worth stating plainly: this is also the smallest model in the whole
|
| 302 |
+
comparison, by a wide margin.
|
| 303 |
+
|
| 304 |
+
| Model | Parameters | MLDR-it (nDCG@10) |
|
| 305 |
+
|---|---|---|
|
| 306 |
+
| **ItColBERT** | **~135M** | **0.4008** (0.4610 chunked) |
|
| 307 |
+
| SauerkrautLM-Multi-ModernColBERT | 149M | 0.3122 |
|
| 308 |
+
| ColBERT-XM | 277M | 0.2734 |
|
| 309 |
+
| mLateOn | 307M | 0.4623 |
|
| 310 |
+
| multilingual-e5-large (dense) | 560M | 0.4310 † |
|
| 311 |
+
| bge-m3 (dense) | 568M | 0.4531 |
|
| 312 |
+
| jina-colbert-v2 | ~0.6B | 0.3858 † |
|
| 313 |
+
|
| 314 |
+
At roughly a quarter to a sixth the size of the ~560M-parameter multilingual
|
| 315 |
+
giants, ItColBERT beats `SauerkrautLM-Multi-ModernColBERT` (the same size
|
| 316 |
+
class) and `ColBERT-XM` (2× the parameters) outright, and statistically ties
|
| 317 |
+
`jina-colbert-v2` (~4.4× the parameters) on the primary out-of-domain
|
| 318 |
+
benchmark. `mLateOn` is the one model that beats it outright while also being
|
| 319 |
+
smaller than the dense giants — included here rather than left out, since
|
| 320 |
+
citing only the flattering comparisons would defeat the point of this
|
| 321 |
+
section. Fewer parameters also means a smaller index and cheaper inference,
|
| 322 |
+
which is part of why training and evaluating this entirely on one consumer
|
| 323 |
+
GPU was practical in the first place.
|
| 324 |
+
|
| 325 |
**Protocol notes that matter for these numbers:**
|
| 326 |
|
| 327 |
- All late-interaction models above are indexed at the same document length
|