Update README.md

README.md

@@ -9,21 +9,25 @@ tags:
 - loss:SpladeLoss
 - loss:SparseMarginMSELoss
 - loss:FlopsLoss
-base_model: yosefw/SPLADE-BERT-Small-BS256
+base_model:
+- prajjwal1/bert-small
 widget:
 - text: leagues, define
-- text: WATCH HOW YOU WANT. STARZ lets you stream hit original series and movies on
-    your favorite devices. Plus you can get the STARZ app on your smartphone or tablet
-    and download full movies and shows to watch off-line, anytime, anywhere. START
-    YOUR FREE TRIAL NOW.
-- text: Furthermore, priority must be given to national jurisdiction. Pointing out
-    that States applied universal jurisdiction differently, he expressed concern at
-    the abuse of its application by some national courts, which rendered it a source
-    of international conflict.
-- text: My sil tells me that my mil cooked the eggplant at high heat for a very long
-    time until it was almost burned. Is it possible that cooking it in such a way
-    gets rid of the bitterness? My mil bought her eggplants at the chain grocery store-
-    so this is not a freshness issue. Thanks for any ideas.
+- text: >-
+    WATCH HOW YOU WANT. STARZ lets you stream hit original series and movies on
+    your favorite devices. Plus you can get the STARZ app on your smartphone or
+    tablet and download full movies and shows to watch off-line, anytime,
+    anywhere. START YOUR FREE TRIAL NOW.
+- text: >-
+    Furthermore, priority must be given to national jurisdiction. Pointing out
+    that States applied universal jurisdiction differently, he expressed concern
+    at the abuse of its application by some national courts, which rendered it a
+    source of international conflict.
+- text: >-
+    My sil tells me that my mil cooked the eggplant at high heat for a very long
+    time until it was almost burned. Is it possible that cooking it in such a
+    way gets rid of the bitterness? My mil bought her eggplants at the chain
+    grocery store- so this is not a freshness issue. Thanks for any ideas.
 - text: how many tablespoons of garlic powder are in an ounce
 pipeline_tag: feature-extraction
 library_name: sentence-transformers
@@ -114,38 +118,35 @@ model-index:
     - type: corpus_sparsity_ratio
       value: 0.9943693333766356
       name: Corpus Sparsity Ratio
+license: mit
+datasets:
+- microsoft/ms_marco
+language:
+- en
 ---
 
-# SPLADE Sparse Encoder
+# SPLADE-BERT-Small-Distil
 
-This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [yosefw/SPLADE-BERT-Small-BS256](https://huggingface.co/yosefw/SPLADE-BERT-Small-BS256) using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space   and can be used for semantic search and sparse retrieval.
-## Model Details
+This is a SPLADE sparse retrieval model based on BERT-Small (29M) that was trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2). 
 
-### Model Description
-- **Model Type:** SPLADE Sparse Encoder
-- **Base model:** [yosefw/SPLADE-BERT-Small-BS256](https://huggingface.co/yosefw/SPLADE-BERT-Small-BS256) <!-- at revision 43b8c4a930896cdbab236b2a46fe1b762216df1a -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 30522 dimensions
-- **Similarity Function:** Dot Product
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+This SPLADE model is `2x` smaller than Naver's official `splade-v3-distilbert` while having `91%` of it's performance on the MSMARCO benchmark. This model is small enough to be used without a GPU on a dataset of a few thousand documents. 
 
-### Model Sources
+- `Collection:` https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
+- `Distillation Dataset:` https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
+- `Code:` https://github.com/rasyosef/splade-tiny-msmarco
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
+## Performance
 
-### Full Model Architecture
+The splade models were evaluated on 55 thousand queries and 8.84 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
+
+||Size (# Params)|MRR@10 (MS MARCO dev)|
+|:---|:----|:-------------------|
+|`BM25`|-|18.0|-|-|
+|`rasyosef/splade-tiny`|4.4M|30.9|
+|`rasyosef/splade-mini`|11.2M|34.1|
+|`rasyosef/splade-small`|28.8M|35.4|
+|`naver/splade-v3-distilbert`|67.0M|38.7|
 
-```
-SparseEncoder(
-  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
-  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
-)
-```
 
 ## Usage
 
@@ -162,7 +163,7 @@ Then you can load this model and run inference.
 from sentence_transformers import SparseEncoder
 
 # Download from the 🤗 Hub
-model = SparseEncoder("yosefw/SPLADE-BERT-Small-BS256-distil")
+model = SparseEncoder("rasyosef/splade-small")
 # Run inference
 queries = [
     "how many tablespoons of garlic powder are in an ounce",
@@ -183,6 +184,34 @@ print(similarities)
 # tensor([[26.3104, 20.4381, 15.5539]])
 ```
 
+## Model Details
+
+### Model Description
+- **Model Type:** SPLADE Sparse Encoder
+- **Base model:** [prajjwal1/bert-small](https://huggingface.co/prajjwal1/bert-small)
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 30522 dimensions
+- **Similarity Function:** Dot Product
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
+
+### Full Model Architecture
+
+```
+SparseEncoder(
+  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
+  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
+)
+```
+
 <!--
 ### Direct Usage (Transformers)
 
@@ -207,6 +236,9 @@ You can finetune this model on your own dataset.
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 
+## More
+<details><summary>Click to expand</summary>
+
 ## Evaluation
 
 ### Metrics
@@ -499,4 +531,5 @@ You can finetune this model on your own dataset.
 ## Model Card Contact
 
 *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
---> 
+--> 
+</details>