MongoDB
/

mdbr-leaf-ir

@@ -29,21 +29,21 @@ Enabling even greater efficiency, `mdbr-leaf-ir` supports [flexible asymmetric a
 If you are looking to perform other tasks such as classification, clustering, semantic sentence similarity, summarization, please check out our [`mdbr-leaf-mt`](https://huggingface.co/MongoDB/mdbr-leaf-mt) model.
 > [!Note]
-> **Note**: this model is the result of MongoDB Research's ML team. At the time of writing it is not used in any of MongoDB's commercial product or service offerings.
 # Technical Report
 A technical report detailing our proposed `LEAF` training procedure is [available here (TBD)](http://FILL_HERE_ARXIV_LINK).
-## Highlights
 * **State-of-the-Art Performance**: `mdbr-leaf-ir` achieves new state-of-the-art results for compact embedding models, ranking <span style="color:red">#TBD</span> on the public BEIR benchmark leaderboard for models <30M parameters with an average nDCG@10 score of <span style="color:red">[TBD HERE]</span>.
 * **Flexible Architecture Support**: `mdbr-leaf-ir` supports asymmetric retrieval architectures enabling even greater retrieval results. [See below](#asymmetric-retrieval-setup) for more information.
 * **MRL and quantization support**: embedding vectors generated by `mdbr-leaf-ir` compress well when truncated (MRL) and/or are stored using more efficient types like `int8` and `binary`.  [See below](#mrl) for more information.
-## Quickstart
-### Sentence Transformers
 ```python
 from sentence_transformers import SentenceTransformer
@@ -84,11 +84,11 @@ for i, query in enumerate(queries):
 #  Similarity: 0.5723 | Document 1: Neural networks are trained ...
 ```
-### Transformers Usage
 See [here](https://huggingface.co/MongoDB/mdbr-leaf-ir/blob/main/transformers_example.ipynb).
-### Asymmetric Retrieval Setup
 `mdbr-leaf-ir` is *aligned* to [`snowflake-arctic-embed-m-v1.5`](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5), the model it has been distilled from. This enables flexible architectures in which, for example, documents are encoded using the larger model, while queries can be encoded faster and more efficiently with the compact `leaf` model:
 ```python
@@ -105,7 +105,7 @@ scores = query_model.similarity(query_embeddings, document_embeddings)
 ```
 Retrieval results from asymmetric mode are usually superior to the [standard mode above](#sentence-transformers).
-### MRL
 Embeddings have been trained via [MRL](https://arxiv.org/abs/2205.13147) and can be truncated for more efficient storage:
 ```python
@@ -131,7 +131,7 @@ print(f"* Similarities:\n\t{similarities}")
 #           [0.4567, 0.6022]])
 ```
-### Vector Quantization
 Vector quantization, for example to `int8` or `binary`, can be performed as follows:
 **Note**: For vector quantization to types other than binary, we suggest performing a calibration to determine the optimal ranges, [see here](https://sbert.net/examples/sentence_transformer/applications/embedding-quantization/README.html#scalar-int8-quantization).
@@ -164,12 +164,12 @@ print(f"* Similarities:\n{similarities}")
 #   [ 72961  98333]]
 ```
-## Evaluation
 Please refer to this <span style="color:red">TBD</span> script to replicate results.
 The checkpoint used to produce the scores presented in the paper [is here](https://huggingface.co/MongoDB/mdbr-leaf-ir/commit/ea98995e96beac21b820aa8ad9afaa6fd29b243d). The current model has been trained further to achieve higher scores.
-## Citation
 If you use this model in your work, please cite:
@@ -185,10 +185,10 @@ If you use this model in your work, please cite:
 }
 ```
-## License
 This model is released under Apache 2.0 License.
-## Contact
 For questions or issues, please open an issue or pull request. You can also contact the MongoDB ML research team at robin.vujanic@mongodb.com.

 If you are looking to perform other tasks such as classification, clustering, semantic sentence similarity, summarization, please check out our [`mdbr-leaf-mt`](https://huggingface.co/MongoDB/mdbr-leaf-mt) model.
 > [!Note]
+> **Note**: this model has been developed by the ML team of MongoDB Research. At the time of writing it is not used in any of MongoDB's commercial product or service offerings.
 # Technical Report
 A technical report detailing our proposed `LEAF` training procedure is [available here (TBD)](http://FILL_HERE_ARXIV_LINK).
+# Highlights
 * **State-of-the-Art Performance**: `mdbr-leaf-ir` achieves new state-of-the-art results for compact embedding models, ranking <span style="color:red">#TBD</span> on the public BEIR benchmark leaderboard for models <30M parameters with an average nDCG@10 score of <span style="color:red">[TBD HERE]</span>.
 * **Flexible Architecture Support**: `mdbr-leaf-ir` supports asymmetric retrieval architectures enabling even greater retrieval results. [See below](#asymmetric-retrieval-setup) for more information.
 * **MRL and quantization support**: embedding vectors generated by `mdbr-leaf-ir` compress well when truncated (MRL) and/or are stored using more efficient types like `int8` and `binary`.  [See below](#mrl) for more information.
+# Quickstart
+## Sentence Transformers
 ```python
 from sentence_transformers import SentenceTransformer
 #  Similarity: 0.5723 | Document 1: Neural networks are trained ...
 ```
+## Transformers Usage
 See [here](https://huggingface.co/MongoDB/mdbr-leaf-ir/blob/main/transformers_example.ipynb).
+## Asymmetric Retrieval Setup
 `mdbr-leaf-ir` is *aligned* to [`snowflake-arctic-embed-m-v1.5`](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5), the model it has been distilled from. This enables flexible architectures in which, for example, documents are encoded using the larger model, while queries can be encoded faster and more efficiently with the compact `leaf` model:
 ```python
 ```
 Retrieval results from asymmetric mode are usually superior to the [standard mode above](#sentence-transformers).
+## MRL
 Embeddings have been trained via [MRL](https://arxiv.org/abs/2205.13147) and can be truncated for more efficient storage:
 ```python
 #           [0.4567, 0.6022]])
 ```
+## Vector Quantization
 Vector quantization, for example to `int8` or `binary`, can be performed as follows:
 **Note**: For vector quantization to types other than binary, we suggest performing a calibration to determine the optimal ranges, [see here](https://sbert.net/examples/sentence_transformer/applications/embedding-quantization/README.html#scalar-int8-quantization).
 #   [ 72961  98333]]
 ```
+# Evaluation
 Please refer to this <span style="color:red">TBD</span> script to replicate results.
 The checkpoint used to produce the scores presented in the paper [is here](https://huggingface.co/MongoDB/mdbr-leaf-ir/commit/ea98995e96beac21b820aa8ad9afaa6fd29b243d). The current model has been trained further to achieve higher scores.
+# Citation
 If you use this model in your work, please cite:
 }
 ```
+# License
 This model is released under Apache 2.0 License.
+# Contact
 For questions or issues, please open an issue or pull request. You can also contact the MongoDB ML research team at robin.vujanic@mongodb.com.