Update Splade Index

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+corpus.jsonl filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,60 @@
+---
+language: en
+library_name: splade-index
+tags:
+- splade
+- splade-index
+- retrieval
+- search
+- sparse
+---
+
+# Splade-Index
+
+This is an index created with the [splade-index](https://github.com/rasyosef/splade-index) library (version `0.1.2`)
+
+## Installation
+
+You can install the `splade-index` library with `pip`:
+
+```bash
+pip install "splade-index==0.1.2"
+
+# Include extra dependencies like stemmer
+pip install "splade-index[full]==0.1.2"
+
+# For huggingface hub usage
+pip install huggingface_hub
+```
+
+## Load this Index
+
+You can use the following code to load this SPLADE index from Hugging Face hub:
+
+```python
+import os
+from sentence_transformers import SparseEncoder
+from splade_index import SPLADE
+
+# Download the SPLADE model that was used to create the index from the HuggingFace Hub
+model_id = "the-splade-model-id" # Enter the splade model id
+model = SparseEncoder(model_id)
+
+# Set your huggingface token if repo is private
+token = os.environ["HF_TOKEN"]
+repo_id = "rasyosef/msmarco_dev_1M_splade_index"
+
+# Load a SPLADE index from the Hugging Face model hub
+retriever = SPLADE.load_from_hub(repo_id, model=model, token=token)
+```
+
+## Stats
+
+This dataset was created using the following data:
+
+| Statistic | Value |
+| --- | --- |
+| Number of documents | 980853 |
+| Number of tokens | 180385384 |
+| Average tokens per document | 183.91 |
+

corpus.jsonl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30fcb411b8bb38c6476de85a8e45e8832b80ac4685097806a9fa77397d62ea9e
+size 362884563

csc.index.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31a1e31b0ef1186f0ac9badce35cc972d679b776af3a88c48868970ea2fdb9d1
+size 984921987

params.index.json

@@ -0,0 +1,7 @@
+{
+    "dtype": "float32",
+    "int_dtype": "int32",
+    "num_docs": 980853,
+    "version": "0.1.2",
+    "backend": "numpy"
+}