Update README.md
Browse files
README.md
CHANGED
|
@@ -150,13 +150,11 @@ datasets:
|
|
| 150 |
|
| 151 |
# SPLADE-BERT-Tiny-Distil
|
| 152 |
|
| 153 |
-
This is a SPLADE sparse retrieval
|
| 154 |
|
| 155 |
-
|
| 156 |
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
- Models: https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
|
| 160 |
- Distillation Dataset: https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
|
| 161 |
- Code: https://github.com/rasyosef/splade-tiny-msmarco
|
| 162 |
|
|
|
|
| 150 |
|
| 151 |
# SPLADE-BERT-Tiny-Distil
|
| 152 |
|
| 153 |
+
This is a SPLADE sparse retrieval model based on BERT-Tiny (4M) that was trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2).
|
| 154 |
|
| 155 |
+
This tiny SPLADE model is `15x` smaller than Naver's official `splade-v3-distilbert` while having `80%` of it's performance on the MSMARCO benchmark. This model is small enough to be used without a GPU on a dataset of a few thousand documents.
|
| 156 |
|
| 157 |
+
- Collection: https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
|
|
|
|
|
|
|
| 158 |
- Distillation Dataset: https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
|
| 159 |
- Code: https://github.com/rasyosef/splade-tiny-msmarco
|
| 160 |
|