| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte |
| pipeline_tag: feature-extraction |
| tags: |
| - onnx |
| - fastembed |
| - sparse-encoder |
| - sparse-retrieval |
| - qdrant |
| - passage-retrieval |
| - asymmetric |
| --- |
| |
| # OpenSearch Neural Sparse Encoding Doc v3 GTE ONNX |
|
|
| This repository contains an ONNX export of |
| [`opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte`](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte), |
| converted by Qdrant for inference with |
| [FastEmbed](https://github.com/qdrant/fastembed). |
|
|
|
|
| ## Model Description |
|
|
| This is a learned sparse retrieval model. |
|
|
| It encodes documents into 30,522-dimensional sparse vectors. Queries are |
| encoded using a tokenizer and a weight lookup table. A non-zero dimension |
| represents the corresponding token in the vocabulary, and its value represents |
| the importance of that token. |
|
|
| The similarity score is the inner product of the query and document sparse |
| vectors. |
|
|
| ## Usage |
|
|
| This ONNX model is designed for use with FastEmbed. |
|
|
| ```python |
| from fastembed import SparseTextEmbedding |
| |
| model = SparseTextEmbedding( |
| model_name="opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte" |
| ) |
| |
| queries = ["What's the weather in New York now?"] |
| documents = ["Currently New York is rainy."] |
| |
| query_embeddings = list(model.query_embed(queries)) |
| document_embeddings = list(model.embed(documents)) |
| ``` |
|
|
| Use `query_embed()` for queries and `embed()` for documents. |
|
|
| For the original PyTorch model and additional usage examples, refer to the |
| [original model repository](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte). |
|
|
| ## Performance |
|
|
| The original model card reports an average NDCG@10 of `0.546` and average FLOPS |
| of `1.7` on the evaluated subset of BEIR. |
|
|
| For detailed benchmark results, refer to the |
| [original model card](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte). |
|
|
| ## License |
|
|
| This project is licensed under the |
| [Apache v2.0 License](https://github.com/opensearch-project/neural-search/blob/main/LICENSE). |
|
|
| ## Copyright |
|
|
| Copyright OpenSearch Contributors. See |
| [NOTICE](https://github.com/opensearch-project/neural-search/blob/main/NOTICE) |
| for details. |
|
|
| ## Acknowledgments |
|
|
| The original model was developed by the OpenSearch project. The ONNX conversion |
| was performed by Qdrant for use with FastEmbed. |
|
|
|
|
