Update README.md

README.md

@@ -9,25 +9,31 @@ tags:
 - loss:SpladeLoss
 - loss:SparseMarginMSELoss
 - loss:FlopsLoss
-base_model: yosefw/SPLADE-BERT-Small-BS128
+base_model:
+- prajjwal1/bert-small
 widget:
-- text: Donate to the Breast Cancer Research Foundation Now BCRF is the largest nonprofit
-    funder of breast cancer research worldwide. Over the years, it has raised more
-    than half a billion dollars in support of research that has made a major impact
-    on how we view and treat breast cancer.
-- text: Macular degeneration—Loss of central vision, blurred vision (especially while
-    reading), distorted vision (like seeing wavy lines), and colors appearing faded.
-    The most common cause of blindness in people over age 60. Eye infection, inflammation,
-    or injury.
+- text: >-
+    Donate to the Breast Cancer Research Foundation Now BCRF is the largest
+    nonprofit funder of breast cancer research worldwide. Over the years, it has
+    raised more than half a billion dollars in support of research that has made
+    a major impact on how we view and treat breast cancer.
+- text: >-
+    Macular degeneration—Loss of central vision, blurred vision (especially
+    while reading), distorted vision (like seeing wavy lines), and colors
+    appearing faded. The most common cause of blindness in people over age 60.
+    Eye infection, inflammation, or injury.
 - text: how do i find the tongue weight of a trailer?
-- text: Feathers (1-3) Pidgey are docile Pokémon, and generally prefer to flee from
-    their enemies rather than fight them. Pidgey's small size permits it to hide easily
-    in long grass, where it is typically found foraging for small insects. It is known
-    to flush out potential prey from long grass by flapping its wings rapidly.
-- text: 10 hilariously insightful foreign words. One of the most obvious differences
-    between cognac and whiskey is that cognac makers use grapes, and whiskey makers
-    use grains. Although both processes use fermentation to create the liquors, cognac
-    makers use a double distillation process.
+- text: >-
+    Feathers (1-3) Pidgey are docile Pokémon, and generally prefer to flee from
+    their enemies rather than fight them. Pidgey's small size permits it to hide
+    easily in long grass, where it is typically found foraging for small
+    insects. It is known to flush out potential prey from long grass by flapping
+    its wings rapidly.
+- text: >-
+    10 hilariously insightful foreign words. One of the most obvious differences
+    between cognac and whiskey is that cognac makers use grapes, and whiskey
+    makers use grains. Although both processes use fermentation to create the
+    liquors, cognac makers use a double distillation process.
 pipeline_tag: feature-extraction
 library_name: sentence-transformers
 metrics:
@@ -117,38 +123,34 @@ model-index:
     - type: corpus_sparsity_ratio
       value: 0.9944841805248121
       name: Corpus Sparsity Ratio
+license: mit
+datasets:
+- microsoft/ms_marco
+language:
+- en
 ---
 
-# SPLADE Sparse Encoder
+# SPLADE-BERT-Small-Distil
 
-This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [yosefw/SPLADE-BERT-Small-BS128](https://huggingface.co/yosefw/SPLADE-BERT-Small-BS128) using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space   and can be used for semantic search and sparse retrieval.
-## Model Details
+This is a SPLADE sparse retrieval model based on BERT-Small (29M) that was trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2). 
 
-### Model Description
-- **Model Type:** SPLADE Sparse Encoder
-- **Base model:** [yosefw/SPLADE-BERT-Small-BS128](https://huggingface.co/yosefw/SPLADE-BERT-Small-BS128) <!-- at revision 27575d2504e7400b5ed11f94d0e162e3e7c01af6 -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 30522 dimensions
-- **Similarity Function:** Dot Product
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+This tiny SPLADE model is `2x` smaller than Naver's official `splade-v3-distilbert` while having `91%` of it's performance on the MSMARCO benchmark. This model is small enough to be used without a GPU on a dataset of a few thousand documents. 
 
-### Model Sources
+- `Collection:` https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
+- `Distillation Dataset:` https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
+- `Code:` https://github.com/rasyosef/splade-tiny-msmarco
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
+## Performance
 
-### Full Model Architecture
+The splade models were evaluated on 55 thousand queries and 8.84 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
 
-```
-SparseEncoder(
-  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
-  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
-)
-```
+||Size (# Params)|MRR@10 (MS MARCO dev)|
+|:---|:----|:-------------------|
+|`BM25`|-|18.0|-|-|
+|`rasyosef/splade-tiny`|4.4M|30.9|
+|`rasyosef/splade-mini`|11.2M|33.2|
+|`rasyosef/splade-small`|28.8M|35.2|
+|`naver/splade-v3-distilbert`|67.0M|38.7|
 
 ## Usage
 
@@ -165,7 +167,7 @@ Then you can load this model and run inference.
 from sentence_transformers import SparseEncoder
 
 # Download from the 🤗 Hub
-model = SparseEncoder("yosefw/SPLADE-BERT-Small-BS128-distil")
+model = SparseEncoder("rasyosef/splade-small")
 # Run inference
 queries = [
     "is cognac whisky",
@@ -210,6 +212,37 @@ You can finetune this model on your own dataset.
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 
+## Model Details
+
+### Model Description
+- **Model Type:** SPLADE Sparse Encoder
+- **Base model:** [prajjwal1/bert-small](https://huggingface.co/prajjwal1/bert-small) <!-- at revision 27575d2504e7400b5ed11f94d0e162e3e7c01af6 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 30522 dimensions
+- **Similarity Function:** Dot Product
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
+
+### Full Model Architecture
+
+```
+SparseEncoder(
+  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
+  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
+)
+```
+
+## More
+<details><summary>Click to expand</summary>
+
 ## Evaluation
 
 ### Metrics
@@ -506,4 +539,5 @@ You can finetune this model on your own dataset.
 ## Model Card Contact
 
 *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
---> 
+--> 
+<details>