rasyosef commited on
Commit
ea1c4e6
·
verified ·
1 Parent(s): 57d9bd8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -1
README.md CHANGED
@@ -137,7 +137,24 @@ datasets:
137
 
138
  # SPLADE-BERT-Mini-Distil
139
 
140
- This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [prajjwal1/bert-mini](https://huggingface.co/prajjwal1/bert-mini) using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
 
142
  ## Usage
143
 
 
137
 
138
  # SPLADE-BERT-Mini-Distil
139
 
140
+ This is a SPLADE sparse retrieval model based on BERT-Mini (11M) that was trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2).
141
+
142
+ This tiny SPLADE model is `6x` smaller than Naver's official `splade-v3-distilbert` while having `85%` of it's performance on the MSMARCO benchmark. This model is small enough to be used without a GPU on a dataset of a few thousand documents.
143
+
144
+ - `Collection:` https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
145
+ - `Distillation Dataset:` https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
146
+ - `Code:` https://github.com/rasyosef/splade-tiny-msmarco
147
+
148
+ ## Performance
149
+
150
+ The splade models were evaluated on 55 thousand queries and 8 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
151
+
152
+ ||Size (# Params)|MRR@10 (MS MARCO dev)|
153
+ |:---|:----|:-------------------|
154
+ |`BM25`|-|18.6|-|-|
155
+ |`rasyosef/splade-tiny`|4.4M|30.8|
156
+ |`rasyosef/splade-mini`|11.2M|32.8|
157
+ |`naver/splade-v3-distilbert`|67.0M|38.7|
158
 
159
  ## Usage
160