Feature Extraction
sentence-transformers
Safetensors
English
bert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:1200000
loss:SpladeLoss
loss:SparseMarginMSELoss
loss:FlopsLoss
Eval Results (legacy)
text-embeddings-inference
Instructions to use rasyosef/splade-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use rasyosef/splade-tiny with sentence-transformers:
from sentence_transformers import SparseEncoder model = SparseEncoder("rasyosef/splade-tiny") queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -150,7 +150,26 @@ datasets:
|
|
| 150 |
|
| 151 |
# SPLADE-BERT-Tiny-Distil
|
| 152 |
|
| 153 |
-
This is a
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 154 |
|
| 155 |
## Usage
|
| 156 |
|
|
|
|
| 150 |
|
| 151 |
# SPLADE-BERT-Tiny-Distil
|
| 152 |
|
| 153 |
+
This is a SPLADE sparse retrieval models based on BERT-Tiny (4M) trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2).
|
| 154 |
+
|
| 155 |
+
The tiny SPLADE models are `6-15x` smaller than Naver's official `splade-v3-distilbert` while having `80-85%` of it's performance on the MSMARCO benchmark. The models are small enough to be used without a GPU on a dataset of a few thousand documents.
|
| 156 |
+
|
| 157 |
+
You can download the models from the following huggingface collection.
|
| 158 |
+
|
| 159 |
+
- Models: https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
|
| 160 |
+
- Distillation Dataset: https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
|
| 161 |
+
- Code: https://github.com/rasyosef/splade-tiny-msmarco
|
| 162 |
+
|
| 163 |
+
## Performance
|
| 164 |
+
|
| 165 |
+
The splade models were evaluated on 55 thousand queries and 8 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
|
| 166 |
+
|
| 167 |
+
||Size (# Params)|MRR@10 (MS MARCO dev)|
|
| 168 |
+
|:---|:----|:-------------------|
|
| 169 |
+
|`BM25`|-|18.6|-|-|
|
| 170 |
+
|`rasyosef/splade-tiny`|4.4M|30.8|
|
| 171 |
+
|`rasyosef/splade-mini`|11.2M|32.8|
|
| 172 |
+
|`naver/splade-v3-distilbert`|67.0M|38.7|
|
| 173 |
|
| 174 |
## Usage
|
| 175 |
|