Add model card

README.md

@@ -0,0 +1,366 @@
+---
+tags:
+- mteb
+- bitnet
+- transformers
+language:
+- multilingual
+- af
+- am
+- ar
+- as
+- az
+- be
+- bg
+- bn
+- br
+- bs
+- ca
+- cs
+- cy
+- da
+- de
+- el
+- en
+- eo
+- es
+- et
+- eu
+- fa
+- fi
+- fr
+- fy
+- ga
+- gd
+- gl
+- gu
+- ha
+- he
+- hi
+- hr
+- hu
+- hy
+- id
+- is
+- it
+- ja
+- jv
+- ka
+- kk
+- km
+- kn
+- ko
+- ku
+- ky
+- la
+- lo
+- lt
+- lv
+- mg
+- mk
+- ml
+- mn
+- mr
+- ms
+- my
+- ne
+- nl
+- 'no'
+- om
+- or
+- pa
+- pl
+- ps
+- pt
+- ro
+- ru
+- sa
+- sd
+- si
+- sk
+- sl
+- so
+- sq
+- sr
+- su
+- sv
+- sw
+- ta
+- te
+- th
+- tl
+- tr
+- ug
+- uk
+- ur
+- uz
+- vi
+- xh
+- yi
+- zh
+license: mit
+---
+
+# BitNet Embeddings
+
+## Model Description
+
+BitNet-Embeddings is a family of multilingual text embedding models developed by Microsoft BitNet team.
+The models use decoder-only architecture with last-token pooling and L2 normalization to produce dense text embeddings.
+They can be applied to a wide range of tasks, including text retrieval, clustering, semantic similarity, classification, bitext mining, and reranking.
+They achieve competitive performance on public benchmarks while maintaining excellent inference and storage efficiency.
+
+- **Developed by:** BitNet Team, Microsoft Research
+- **Model type:** BitNet b1.58 based Text Embeddings
+- **Language(s):** Multilingual (see [Supported Languages](#supported-languages))
+- **License:** MIT License
+
+## Model Sources
+
+- **Repository:** [https://github.com/microsoft/BitNet](https://github.com/microsoft/BitNet)
+- **Paper:** [The Era of 1-bit LLMs: BitNet b1.58 and its Inference Optimization](https://arxiv.org/abs/2402.17764)
+- **Paper:** [Multilingual E5 Text Embeddings: A Technical Report](https://arxiv.org/abs/2402.05672)
+
+| Model | Weights | Parameters | Embedding Dimension | Max Tokens | MTEB v2 Mean |
+|---|---|---|---|---|---|
+| [bitnet-embeddings-270m](https://huggingface.co/microsoft/bitnet-embedding-270m) | 1.58-bit | 270M | 640 | 32,768 | 66.26 |
+| [harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m) | bf16 | 270M | 640 | 32,768 | 66.5 |
+| [bitnet-embeddings-0.6b](https://huggingface.co/microsoft/bitnet-embedding-0.6b) | 1.58-bit | 0.6B | 1,024 | 32,768 | 67.49 |
+| [harrier-oss-v1-0.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b) | bf16 | 0.6B | 1,024 | 32,768 | 69.0 |
+
+## Model Details
+
+- **Architecture**: Transformer-based, modified with BitLinear layers (BitNet framework).
+  - Uses Rotary Position Embeddings (RoPE).
+  - Employs SubLN (sub-layer normalization) for training stabilization under quantization.
+  - No bias terms in linear or normalization layers.
+- **Quantization**: Native 1.58-bit weights and 8-bit activations (W1.58A8).
+  - Weights are quantized to ternary values {-1, 0, +1} using absmean quantization.
+  - Activations are quantized to 8-bit integers using absmax quantization (per-token).
+  - Trained from scratch with this quantization scheme, not post-training quantized.
+- **Context Length**: 32,768 tokens.
+- **Pooling Strategy**: Last-token (EOS) pooling followed by L2 normalization.
+- **Training Pipeline**:
+  1. **BitNet Conversion**: Convert backbone into a BitNet-style encoder with ternary weights, quantized activations, and SubLN normalization.
+  2. **Continual Contrastive Pre-training**: Trained on 1B text pairs with InfoNCE loss.
+  3. **Distillation-based Supervised Fine-tuning**: Contrastive loss + similarity-distribution distillation + attention-relation distillation from FP16 teacher.
+
+| Model | [bitnet-embedding-0.6B](https://huggingface.co/microsoft/bitnet-embedding-0.6b) | [bitnet-embedding-270M](https://huggingface.co/microsoft/bitnet-embedding-270m) |
+|---|---|---|
+| Backbone | Qwen3-0.6B | Gemma3 |
+| Parameters | ~0.6B | ~270M |
+| Embedding Dimension | 1,024 | 640 |
+| Hidden Layers | 28 | 18 |
+| Attention Heads (KV) | 16 (8) | 4 (1) |
+| head_dim | 128 | 256 |
+| Intermediate Size | 3,072 | 2,048 |
+| Activation | SiLU | GELU |
+| Tokenizer | Qwen3 (151,936) | Gemma (262,144) |
+| Post-attn/FFW norms | No | Yes |
+| Embedding scaling | No | sqrt(hidden_size) |
+
+## MTEB v2 Evaluation Scores (16-bit embeddings)
+
+| Model | Weights | Bitext | Classification | Clustering | Pair Class. | Reranking | Retrieval | STS | **Mean** |
+|---|---|---|---|---|---|---|---|---|---|
+| bitnet-embeddings-270m | 1.58-bit | 80.47 | 71.09 | 52.37 | 79.72 | 60.50 | 66.71 | 74.35 | **66.26** |
+| bitnet-embeddings-0.6b | 1.58-bit | 81.47 | 72.65 | 53.06 | 80.47 | 62.12 | 68.33 | 74.97 | **67.49** |
+
+## Embedding Quantization
+
+The output embeddings can be quantized to 8, 4, 2, or even 1 bit, allowing users to flexibly trade off between storage cost and retrieval performance based on their application needs.
+
+![Embedding Quantization — Mean MTEB v2 Score](fig1_quant_per_task.png)
+
+## Training
+
+The models are trained with contrastive learning objectives on a large-scale mixture of multilingual datasets covering diverse tasks.
+Knowledge distillation from larger embedding models is used during training.
+The BitNet quantization is applied to all linear layers, resulting in 1.58-bit ternary weights while keeping activations in higher precision.
+
+## How to Use (with bitnet.cpp)
+
+For achieving the efficiency benefits, use the dedicated C++ implementation: [bitnet.cpp](https://github.com/microsoft/BitNet).
+
+### Build
+
+```bash
+git clone --recursive https://github.com/microsoft/BitNet.git
+cd BitNet
+cmake -S . -B build \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCMAKE_C_COMPILER=clang \
+  -DCMAKE_CXX_COMPILER=clang++ \
+  -DGGML_NATIVE=ON \
+  -DGGML_OPENMP=OFF \
+  -DLLAMA_BUILD_COMMON=ON \
+  -DLLAMA_BUILD_TOOLS=ON \
+  -DLLAMA_BUILD_EXAMPLES=ON
+cmake --build build --target llama-embedding llama-bench -j$(nproc)
+```
+
+### Download
+
+```bash
+# 270M
+hf download microsoft/bitnet-embedding-270m \
+  bitnet-embeddings-270m-bf16-i2_s.gguf \
+  --local-dir models/bitnet-embedding-270m
+
+# 0.6B
+hf download microsoft/bitnet-embedding-0.6b \
+  bitnet-embeddings-0.6b-bf16-i2_s.gguf \
+  --local-dir models/bitnet-embedding-0.6b
+```
+
+### Inference
+
+```bash
+# Choose one model:
+MODEL=models/bitnet-embedding-270m/bitnet-embeddings-270m-bf16-i2_s.gguf
+# MODEL=models/bitnet-embedding-0.6b/bitnet-embeddings-0.6b-bf16-i2_s.gguf
+
+./build/bin/llama-embedding \
+  -m "$MODEL" \
+  -p "query: What is BitNet?" \
+  --embd-normalize 2 \
+  --embd-output-format array
+```
+
+**Example output** (L2-normalized embedding; 640 dimensions for 270M or 1,024 for 0.6B; truncated):
+
+```json
+[[0.0239517, 0.6826404, -0.0000000, -0.0644535, 0.0613754, 0.0473094, 0.0114330, ...]]
+```
+
+Please refer to the [bitnet.cpp GitHub repository](https://github.com/microsoft/BitNet) for detailed compilation steps, usage examples, and command-line options.
+
+## Evaluation
+
+### Embedding Quality (MMTEB eng, v2)
+
+BitNet Embedding 0.6B was evaluated against its full-precision FP16 teacher model on the MMTEB (eng, v2) benchmark:
+
+| Model | Cls. | Clust. | PairCls. | Rerank. | Retr. | STS | Summ. | Avg. | Speed (t/s) |
+|-------|------|--------|----------|--------|-------|-----|-------|------|-------------|
+| FP16 Teacher | 86.37 | 55.48 | 82.56 | 43.89 | 55.34 | 81.15 | 31.87 | 67.95 | 382.15 |
+| **BitNet Embedding 0.6B** | **86.49** | **55.42** | **82.30** | **43.41** | **54.03** | **81.15** | **32.06** | **67.60** | **870.90** |
+
+The model achieves **67.60** average score on MMTEB (eng, v2), only **0.35 points** below the FP16 teacher, while delivering **2.28x** higher CPU throughput.
+
+
+### Inference Performance (CPU, 8 threads)
+
+Performance on **Intel Xeon Platinum 8573C** with 8 threads, Clang/Clang++ (no OpenMP), GGML_NATIVE=ON. All results in tokens/second (mean +/- std over 3 runs).
+
+> **Note on build flags:** The results below use `-DGGML_NATIVE=ON`, which auto-detects and enables the best instruction set supported by the host CPU (e.g., AVX, AVX2, AVX-VNNI, FMA, F16C). This yields optimal performance. To target only AVX2 (e.g., for portable binaries), set `-DGGML_NATIVE=OFF` and manually specify:
+> ```
+> -DGGML_AVX=ON -DGGML_AVX2=ON -DGGML_FMA=ON -DGGML_F16C=ON
+> -DGGML_AVX512=OFF -DGGML_AVX512_VBMI=OFF -DGGML_AVX512_VNNI=OFF -DGGML_AVX512_BF16=OFF
+> ```
+
+#### bitnet-embedding-0.6B
+
+| Test | F16.gguf (t/s) | **I2_S.gguf (t/s)** | Speedup |
+|------|---------------|-----------------|---------|
+| pp128 | 382.15 | **870.90** | **2.28x** |
+| pp256 | 373.95 | **827.75** | **2.21x** |
+| pp512 | 371.86 | **716.27** | **1.93x** |
+| pp1024 | 341.55 | **620.58** | **1.82x** |
+| pp2048 | 298.21 | **481.14** | **1.61x** |
+| pp4096 | 236.76 | **336.32** | **1.42x** |
+
+#### bitnet-embedding-270m
+
+| Test | F16.gguf (t/s) | **I2_S.gguf (t/s)** | Speedup |
+|------|---------------|-----------------|---------|
+| pp128 | 1212.68 | **2019.59** | **1.67x** |
+| pp256 | 1221.28 | **2119.50** | **1.74x** |
+| pp512 | 1394.99 | **2181.23** | **1.56x** |
+| pp1024 | 1265.22 | **2086.46** | **1.65x** |
+| pp2048 | 1024.47 | **1471.60** | **1.44x** |
+| pp4096 | 785.54 | **1033.46** | **1.32x** |
+
+## Uses
+
+### Direct Use
+
+BitNet-Embeddings is designed for direct use in tasks requiring text embeddings. Its primary applications include:
+
+- Efficient information retrieval for RAG, web search, enterprise search, and question answering applications.
+- Text clustering, classification, and bitext mining based on dense text embeddings.
+
+### Out-of-Scope Use
+
+- BitNet-Embeddings does not generate any human-readable texts or other multi-modal content. Instead, it maps input texts into dense embedding vectors that are used by downstream Vector DB or classifiers.
+- **Limited Training Data Representation:** While the training data includes a mix of multilingual datasets, it may not represent diverse linguistic, cultural, or geographical contexts for low-resource languages. As a result, performance in low-resource languages may be significantly limited.
+- **Domain-Specific Limitations:** The training data predominantly covers general-purpose knowledge tasks. Specific or niche domains such as legal, medical, or scientific literature may not be adequately represented, and results in these areas should be interpreted with caution.
+- **Use in High-Risk Applications:** We do not recommend using BitNet-Embeddings in commercial or real-world applications without further testing and development. As the model is being released for research purposes, additional evaluation and fine-tuning are required to ensure reliability and fairness in high-stakes scenarios.
+
+## Supported Languages
+
+The models are trained on multilingual data and support a wide range of languages,
+including but not limited to: Arabic, Bulgarian, Catalan, Czech, Danish, German, Greek, English, Spanish,
+Estonian, Persian, Finnish, French, Hebrew, Hindi, Croatian, Hungarian, Indonesian, Italian, Japanese,
+Korean, Lithuanian, Latvian, Macedonian, Malay, Dutch, Norwegian, Polish, Portuguese, Romanian, Russian,
+Slovak, Slovenian, Albanian, Serbian, Swedish, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Chinese.
+
+## Evaluation
+
+Please follow the [mteb](https://github.com/embeddings-benchmark/mteb) repository on how to reproduce our scores.
+The evaluation prompts used for each task are also available at [mteb_v2_eval_prompts.json](mteb_v2_eval_prompts.json).
+
+## Bias, Risks, and Limitations
+
+- Predictions may perpetuate biases present in the training data.
+- There is limited support for extremely low-resource languages and underrepresented domains.
+- The model is designed for text embedding tasks (retrieval, similarity, classification) and is not suitable for text generation.
+
+## Disclaimer
+
+We do not recommend using BitNet Embedding 0.6B in commercial or real-world applications without further testing and development. This model is intended for research and development purposes. Please use responsibly.
+
+## FAQ
+
+**1. Do I need to add instructions to the query?**
+
+Yes, this is how the model is trained, otherwise you will see a performance degradation.
+The task definition should be a one-sentence instruction that describes the task.
+This is a way to customize text embeddings for different scenarios through natural language instructions.
+
+On the other hand, there is no need to add instructions to the document side.
+
+**2. Why are my reproduced results slightly different from reported in the model card?**
+
+Different versions of `transformers` and `pytorch` could cause negligible but non-zero performance differences.
+
+**3. What pooling strategy does this model use?**
+
+The model uses **last-token pooling** — the embedding of the last non-padding token is used as the sentence representation.
+The embedding is then L2-normalized.
+
+## Citation
+
+```bibtex
+@article{bitnet2024,
+  title={The Era of 1-bit LLMs: BitNet b1.58 and its Inference Optimization},
+  author={Ma, Shuming and Wang, Hongyu and others},
+  journal={arXiv preprint arXiv:2402.17764},
+  year={2024}
+}
+
+@inproceedings{wang2025bitnet,
+  title={BitNet.cpp: Efficient Edge Inference for Ternary LLMs},
+  author={Wang, Jinheng and Zhou, Hansong and Song, Ting and Cao, Shijie and Xia, Yan and Cao, Ting and Wei, Jianyu and Ma, Shuming and Wang, Hongyu and Wei, Furu},
+  booktitle={Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
+  pages={9305--9322},
+  year={2025}
+}
+
+@article{wang2024multilingual,
+  title={Multilingual E5 Text Embeddings: A Technical Report},
+  author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
+  journal={arXiv preprint arXiv:2402.05672},
+  year={2024}
+}
+```

mteb_v2_eval_prompts.json

@@ -0,0 +1,133 @@
+{
+  "AILAStatutes": "Identifying the most relevant statutes for a given situation",
+  "AfriSentiClassification": "Given a text, categorized by sentiment into positive, negative, or neutral",
+  "AlloProfClusteringS2S.v2": "Identify the topic of document titles from Allo Prof dataset",
+  "AlloprofReranking": "Given a question, retrieve passages that answer the question",
+  "AmazonCounterfactualClassification": "Given an Amazon review, judge whether it is counterfactual.",
+  "ArXivHierarchicalClusteringP2P": "Identify the main and secondary category of Arxiv papers based on the titles and abstracts",
+  "ArXivHierarchicalClusteringS2S": "Identify the main and secondary category of Arxiv papers based on the titles",
+  "ArguAna": "Given a claim, find documents that refute the claim",
+  "ArmenianParaphrasePC": "Retrieve semantically similar text",
+  "BUCC.v2": "Retrieve parallel sentences",
+  "BelebeleRetrieval": "Retrieval the relevant passage for the given query",
+  "BibleNLPBitextMining": "Retrieve parallel sentences",
+  "BigPatentClustering.v2": "Identify the category of documents from the Big Patent dataset",
+  "BiorxivClusteringP2P.v2": "Identify the main category of Biorxiv papers based on the titles and abstracts",
+  "BornholmBitextMining": "Retrieve parallel sentences",
+  "BrazilianToxicTweetsClassification": "Classify the toxic tweets in Brazilian Portuguese into one of the six categories: LGBTQ+phobia, Xenophobia, Obscene, Insult, Misogyny and Racism.",
+  "BulgarianStoreReviewSentimentClassfication": "Classify user reviews into positive, negative or mixed sentiment",
+  "CEDRClassification": "Given a comment as query, classify expressed emotions into joy, sadness, surprise, fear, and anger",
+  "CLSClusteringP2P.v2": "Identify the main category of scholar papers based on the titles and abstracts",
+  "CSFDSKMovieReviewSentimentClassification": "Given a movie review, classify its rating on a scale from 0 to 5",
+  "CTKFactsNLI": "Retrieve semantically similar text",
+  "CataloniaTweetClassification": "Given a tweet, classify its sentiment into AGAINST, FAVOR or NEUTRAL towards Catalonia's independence.",
+  "Core17InstructionRetrieval": "Retrieve relevant passages for the given query with conditions",
+  "CovidRetrieval": "Given a question on COVID-19, retrieve news articles that answer the question",
+  "CyrillicTurkicLangClassification": "Given a text, classify its language",
+  "CzechProductReviewSentimentClassification": "Classify product reviews into positive, neutral, or negative sentiment",
+  "DBpediaClassification": "Given the following text, retrieve the appropriate DBpedia category including Company, EducationalInstitution, Artist, Athlete, OfficeHolder, MeanOfTransportation, Building, NaturalPlace, Village, Animal, Plant, Album, Film, WrittenWork.",
+  "DalajClassification": "Classify texts based on linguistic acceptability in Swedish",
+  "DiaBlaBitextMining": "Retrieve parallel sentences",
+  "EstonianValenceClassification": "Given a news article, categorized by sentiment into negatiivne, positiivne, neutraalne or vastuolulin",
+  "FaroeseSTS": "Retrieve semantically similar text",
+  "FilipinoShopeeReviewsClassification": "Given a shop review, classify its rating on a scale from 1 to 5",
+  "FinParaSTS": "Retrieve semantically similar text",
+  "FinancialPhrasebankClassification": "Given financial news, categorized by sentiment into positive, negative, or neutral",
+  "FloresBitextMining": "Retrieve parallel sentences",
+  "GermanSTSBenchmark": "Retrieve semantically similar text",
+  "GreekLegalCodeClassification": "Given a greek legal text, classify its topic",
+  "GujaratiNewsClassification": "Given a Gujarati news articles, classify ist topic",
+  "HALClusteringS2S.v2": "Identify the topic of titles from HAL",
+  "HagridRetrieval": "Given a question, retrieve relevant responses",
+  "IN22GenBitextMining": "Retrieve parallel sentences",
+  "IndicCrosslingualSTS": "Retrieve semantically similar text",
+  "IndicGenBenchFloresBitextMining": "Retrieve parallel sentences",
+  "IndicLangClassification": "Given a text, classify its language",
+  "IndonesianIdClickbaitClassification": "Given an Indonesian news headlines, classify its into clickbait or non-clickbait",
+  "IsiZuluNewsClassification": "Given a news article, classify its topic",
+  "ItaCaseholdClassification": "Given a judgments, classify its topic",
+  "JSICK": "Retrieve semantically similar text",
+  "KorHateSpeechMLClassification": "Given a Korean online news comments, classify its fine-grained hate speech classes",
+  "KorSarcasmClassification": "Given a twitter, categorized it into sarcasm or not_sarcasm",
+  "KurdishSentimentClassification": "Given a text, categorized by sentiment into positive or negative",
+  "LEMBPasskeyRetrieval": "Retrieval the relevant passage for the given query",
+  "LegalBenchCorporateLobbying": "Given a query, retrieve relevant legal bill summaries",
+  "MIRACLRetrievalHardNegatives": "Retrieve Wikipedia passages that answer the question",
+  "MLQARetrieval": "Retrieval the relevant passage for the given query",
+  "MacedonianTweetSentimentClassification": "Given a Macedonian tweet, categorized by sentiment into positive, negative, or neutral",
+  "MalteseNewsClassification": "Given a maltese new, classify its topic",
+  "MasakhaNEWSClassification": "Classify the News in the given texts into one of the seven category: politics,sports,health,business,entertainment,technology,religion ",
+  "MasakhaNEWSClusteringS2S": "Identify the topic or theme of the given news articles based on the titles",
+  "MassiveIntentClassification": "Given a user utterance as query, find the user intents",
+  "MedrxivClusteringP2P.v2": "Identify the main category of Medrxiv papers based on the titles and abstracts",
+  "MultiEURLEXMultilabelClassification": "Given a text, classify its topic",
+  "MultiHateClassification": "Given a text, categorized by sentiment into hate or non-hate",
+  "NTREXBitextMining": "Retrieve parallel sentences",
+  "NepaliNewsClassification": "Given a news article, categorized it into business, entertainment or sports",
+  "News21InstructionRetrieval": "Retrieve relevant passages for the given query with conditions",
+  "NollySentiBitextMining": "Retrieve parallel sentences",
+  "NordicLangClassification": "Given a text in a Nordic language, classify the language into one of the following categories: Danish, Swedish, Norwegian (Nynorsk), Norwegian (Bokmål), Faroese, Icelandic.",
+  "NorwegianCourtsBitextMining": "Retrieve parallel sentences",
+  "NusaParagraphEmotionClassification": "Classify the emotion into one of the following categories: fear, sadness, anger, happy, love, surprise, shame.",
+  "NusaTranslationBitextMining": "Retrieve parallel sentences",
+  "NusaX-senti": "Given a text, categorized by sentiment into positive or negative",
+  "NusaXBitextMining": "Retrieve parallel sentences",
+  "OdiaNewsClassification": "Given a news article, categorized it into business, entertainment or sports",
+  "OpusparcusPC": "Retrieve semantically similar text",
+  "PAC": "Classify Polish contract clauses into one of the following two types: \"Safe Contract Clauses\" and \"Unfair Contract Clauses\".",
+  "PawsXPairClassification": "Retrieve semantically similar text",
+  "PlscClusteringP2P.v2": "Identify the category of titles+abstracts from Library of Science",
+  "PoemSentimentClassification": "Given the following verse from a poem, classify its sentiment as negative, neutral, positive, or mixed.",
+  "PolEmo2.0-OUT": "Classify the sentiment of products and school online reviews",
+  "PpcPC": "Retrieve semantically similar text",
+  "PunjabiNewsClassification": "Given a news article, categorized it into two-classes",
+  "RTE3": "Retrieve semantically similar text",
+  "Robust04InstructionRetrieval": "Retrieve relevant passages for the given query with conditions",
+  "RomaniBibleClustering": "Identify verses from the Bible in Kalderash Romani by book.",
+  "RuBQReranking": "Given a question, retrieve Wikipedia passages that answer the question",
+  "SCIDOCS": "Given a scientific paper title, retrieve paper abstracts that are cited by the given paper",
+  "SIB200ClusteringS2S": "Identify the category of documents",
+  "SICK-R": "Retrieve semantically similar text",
+  "STS12": "Retrieve semantically related sentences",
+  "STS13": "Retrieve semantically similar text",
+  "STS14": "Retrieve semantically similar text",
+  "STS15": "Retrieve semantically similar text",
+  "STS17": "Retrieve semantically similar text",
+  "STS22.v2": "Given a document, retrieve semantically related documents",
+  "STSB": "Retrieve semantically similar text",
+  "STSBenchmark": "Retrieve semantically similar text",
+  "STSES": "Given a Spanish sentence, retrieve semantically related Spanish sentences",
+  "ScalaClassification": "Classify passages into correct or correct in Scandinavian Languages based on linguistic acceptability",
+  "SemRel24STS": "Retrieve semantically similar text",
+  "SentimentAnalysisHindi": "Given a hindi text, categorized by sentiment into positive, negative or neutral",
+  "SinhalaNewsClassification": "Given a news article, categorized it into political, business, technology, sports and Entertainment",
+  "SiswatiNewsClassification": "Identify fine-grained news categories in Siswati language.",
+  "SlovakMovieReviewSentimentClassification": "Given a movie review, categorized it into positive or negative",
+  "SpartQA": "Given the following spatial reasoning question, retrieve the right answer.",
+  "SprintDuplicateQuestions": "Find questions that have the same meaning as the input question",
+  "StackExchangeClustering.v2": "Identify the topic or theme of StackExchange posts based on the titles",
+  "StackOverflowQA": "Given a question about coding, retrieval code or passage that can solve user's question",
+  "StatcanDialogueDatasetRetrieval": "Retrieval the relevant passage for the given query",
+  "SwahiliNewsClassification": "Given a news article, classify its domain",
+  "SwednClusteringP2P": "Identify news categories in Swedish passages",
+  "SwissJudgementClassification": "Given a news article, categorized it into approval or dismissal",
+  "T2Reranking": "Given a Chinese search query, retrieve web passages that answer the question",
+  "TERRa": "Given a premise, retrieve a hypothesis that is entailed by the premise",
+  "TRECCOVID": "Given a medical query, retrieve documents that answer the query",
+  "Tatoeba": "Retrieve parallel sentences",
+  "TempReasonL1": "Given the following question about time, retrieve the correct answer.",
+  "ToxicConversationsClassification": "Classify the given comments as either toxic or not toxic",
+  "TswanaNewsClassification": "Given a news article, classify its topic",
+  "TweetTopicSingleClassification": "Gvien a twitter, classify its topic",
+  "TwitterHjerneRetrieval": "Retrieve answers to questions asked in Danish tweets",
+  "TwitterURLCorpus": "Find tweets that have the same meaning as the input tweet",
+  "VoyageMMarcoReranking": "Given a Japanese search query, retrieve web passages that answer the question",
+  "WebLINXCandidatesReranking": "Retrieval the relevant passage for the given query",
+  "WikiCitiesClustering": "Identify of Wikipedia articles of cities by country",
+  "WikiClusteringP2P.v2": "Identify the category of wiki passages",
+  "WikipediaRerankingMultilingual": "Retrieval the relevant passage for the given query",
+  "WikipediaRetrievalMultilingual": "Retrieval the relevant passage for the given query",
+  "WinoGrande": "Given the following sentence, retrieve an appropriate answer to fill in the missing underscored part.",
+  "XNLI": "Retrieve semantically similar text",
+  "indonli": "Retrieve semantically similar text"
+}