tomaarsen HF Staff commited on
Commit
3eeba7f
·
verified ·
1 Parent(s): 3b7c221

Expand introduction with images

Browse files
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -183,6 +183,17 @@ model-index:
183
  # Multi-Vector Encoder
184
 
185
  This is a [Multi-Vector Encoder](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html) model finetuned from [lightonai/mLateOn-unsupervised](https://huggingface.co/lightonai/mLateOn-unsupervised) on the [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.
 
 
 
 
 
 
 
 
 
 
 
186
  ## Model Details
187
 
188
  ### Model Description
 
183
  # Multi-Vector Encoder
184
 
185
  This is a [Multi-Vector Encoder](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html) model finetuned from [lightonai/mLateOn-unsupervised](https://huggingface.co/lightonai/mLateOn-unsupervised) on the [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.
186
+
187
+ **mLateOn-medical is a domain specialist.** It was trained on 1 million medical question and passage pairs from [MIRIAD](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) in 14 hours on a single RTX 3090. On a held-out benchmark of 1,000 medical questions searching 200,000 passages, it beats every general-purpose retrieval model I evaluated, dense, sparse, lexical, and multi-vector alike, including dense models with roughly 30 times as many active parameters.
188
+
189
+ ![NDCG@10 on the MIRIAD medical benchmark versus active parameters, with mLateOn-medical at the top](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/train-multi-vector-encoder/mve_medical_model_size_ndcg.png)
190
+
191
+ Late interaction stores one vector per token, so its index is larger than a dense one, and these medical passages are long (about 878 stored vectors each). [`HierarchicalTokenPooling`](https://sbert.net/docs/package_reference/multi_vector_encoder/modules.html#hierarchicaltokenpooling) compresses that cheaply: halving the index costs 0.003 NDCG@10, and even a tenth of the index still scores higher than every other model in the benchmark.
192
+
193
+ ![Index size for a 200,000-passage corpus versus NDCG@10, including the token pooling trajectory](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/train-multi-vector-encoder/mve_medical_index_size_ndcg.png)
194
+
195
+ The full recipe, the evaluation protocol, and the experiments behind each choice are described in [Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-multi-vector-encoder).
196
+
197
  ## Model Details
198
 
199
  ### Model Description