Feature Extraction
sentence-transformers
Safetensors
modernbert
multi-vector
colbert
late-interaction
Generated from Trainer
dataset_size:1000000
loss:CachedMultiVectorMultipleNegativesRankingLoss
Eval Results (legacy)
text-embeddings-inference
Instructions to use multi-vector-encoder/mLateOn-medical with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use multi-vector-encoder/mLateOn-medical with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("multi-vector-encoder/mLateOn-medical") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Expand introduction with images
Browse files
README.md
CHANGED
|
@@ -183,6 +183,17 @@ model-index:
|
|
| 183 |
# Multi-Vector Encoder
|
| 184 |
|
| 185 |
This is a [Multi-Vector Encoder](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html) model finetuned from [lightonai/mLateOn-unsupervised](https://huggingface.co/lightonai/mLateOn-unsupervised) on the [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 186 |
## Model Details
|
| 187 |
|
| 188 |
### Model Description
|
|
|
|
| 183 |
# Multi-Vector Encoder
|
| 184 |
|
| 185 |
This is a [Multi-Vector Encoder](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html) model finetuned from [lightonai/mLateOn-unsupervised](https://huggingface.co/lightonai/mLateOn-unsupervised) on the [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.
|
| 186 |
+
|
| 187 |
+
**mLateOn-medical is a domain specialist.** It was trained on 1 million medical question and passage pairs from [MIRIAD](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) in 14 hours on a single RTX 3090. On a held-out benchmark of 1,000 medical questions searching 200,000 passages, it beats every general-purpose retrieval model I evaluated, dense, sparse, lexical, and multi-vector alike, including dense models with roughly 30 times as many active parameters.
|
| 188 |
+
|
| 189 |
+

|
| 190 |
+
|
| 191 |
+
Late interaction stores one vector per token, so its index is larger than a dense one, and these medical passages are long (about 878 stored vectors each). [`HierarchicalTokenPooling`](https://sbert.net/docs/package_reference/multi_vector_encoder/modules.html#hierarchicaltokenpooling) compresses that cheaply: halving the index costs 0.003 NDCG@10, and even a tenth of the index still scores higher than every other model in the benchmark.
|
| 192 |
+
|
| 193 |
+

|
| 194 |
+
|
| 195 |
+
The full recipe, the evaluation protocol, and the experiments behind each choice are described in [Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-multi-vector-encoder).
|
| 196 |
+
|
| 197 |
## Model Details
|
| 198 |
|
| 199 |
### Model Description
|