MenteEAI
/

mentee-embed-v3

@@ -1,5 +1,7 @@
 ---
 license: apache-2.0
 language:
 - en
 - ar
@@ -66,14 +68,16 @@ All baselines evaluated under identical conditions on the same hardware.
 ### Protocol A — In-batch Retrieval (pool ≈ 97 candidates)
 *Format: acc@1 / R@5 / MRR@10*
-| Model | val | MIRACL-EN | MIRACL-AR | MIRACL-UR | xling EN↔UR | MS-MARCO | **Avg MRR@10** |
-|-------|-----|-----------|-----------|-----------|-------------|----------|----------------|
-| **mentee-embed-v3 (ours)** | 0.766/0.988/0.870 | 0.636/0.920/0.766 | 0.326/0.604/0.475 | 0.290/0.568/0.443 | 0.781/0.925/0.848 | 0.517/0.981/0.742 | **0.691** |
-| paraphrase-multilingual-mpnet-base-v2 | 0.823/0.972/0.895 | 0.864/1.000/0.931 | 0.722/0.975/0.839 | 0.686/0.950/0.806 | 0.831/0.937/0.880 | 0.665/0.998/0.830 | **0.864** |
-| paraphrase-multilingual-MiniLM-L12-v2 | 0.802/0.973/0.883 | 0.854/0.997/0.924 | 0.696/0.964/0.819 | 0.621/0.908/0.753 | 0.782/0.907/0.841 | 0.600/0.998/0.796 | **0.836** |
-| all-MiniLM-L6-v2 | 0.524/0.688/0.627 | 0.856/0.999/0.927 | 0.025/0.109/0.144 | 0.028/0.088/0.140 | 0.065/0.172/0.186 | 0.696/1.000/0.848 | **0.479** |
-> ✅ **mentee-embed-v3 beats all-MiniLM-L6-v2** (0.691 vs 0.479) on Protocol A avg MRR@10 — despite all-MiniLM being a pretrained model.
 ---
@@ -146,9 +150,11 @@ Stage 4 — Distillation Round 2 (with hard negatives)
 ## Usage
 ```python
-from sentence_transformers import SentenceTransformer
-model = SentenceTransformer("MenteEAI/mentee-embed-v3")
 sentences = [
     "Hello, how are you?",
@@ -156,27 +162,28 @@ sentences = [
     "ہیلو، آپ کیسے ہیں؟"
 ]
-embeddings = model.encode(sentences)
-print(embeddings.shape)  # (3, 384)
 ```
 ### Similarity search
 ```python
-from sentence_transformers import SentenceTransformer, util
-model = SentenceTransformer("MenteEAI/mentee-embed-v3")
-query = "What is machine learning?"
-passages = [
     "Machine learning is a subset of artificial intelligence.",
     "The weather today is sunny.",
     "تعلم الآلة هو فرع من فروع الذكاء الاصطناعي.",
-]
-q_emb = model.encode(query)
-p_emb = model.encode(passages)
-scores = util.cos_sim(q_emb, p_emb)
-print(scores)
 ```
 ---
@@ -194,11 +201,13 @@ print(scores)
 ```bibtex
 @misc{mentee-embed-v3-2026,
-  title   = {mentee-embed-v3: Trilingual Text Embeddings Trained from Scratch},
-  author  = {Syed Syab Ahmad Shah and Team MenteE AI},
   year    = {2026},
-  url     = {https://huggingface.co/MenteEAI/mentee-embed-v3},
-  note    = {Apache-2.0 License}
 }
 ```
@@ -206,6 +215,6 @@ print(scores)
 ## About MenteE AI
-Built by **Syed Syab Ahmad Shah** and Team MenteE AI.
-🌐 [menteeai.org](https://menteeai.org) · 📧 syab@menteeai.org
-Research paper: [10.5281/zenodo.22087139](https://doi.org/10.5281/zenodo.22087139)

 ---
 license: apache-2.0
+pipeline_tag: sentence-similarity
+library_name: transformers
 language:
 - en
 - ar
 ### Protocol A — In-batch Retrieval (pool ≈ 97 candidates)
 *Format: acc@1 / R@5 / MRR@10*
+*Avg MRR@10 computed over 5 datasets: MIRACL-EN, MIRACL-AR, MIRACL-UR, xling EN-UR, MS-MARCO (val excluded from avg).*
+| Model | MIRACL-EN | MIRACL-AR | MIRACL-UR | xling EN↔UR | MS-MARCO | **Avg MRR@10** |
+|-------|-----------|-----------|-----------|-------------|----------|----------------|
+| **mentee-embed-v3 (ours)** | 0.636/0.920/0.766 | 0.326/0.604/0.475 | 0.290/0.568/0.443 | 0.781/0.925/0.848 | 0.517/0.981/0.742 | **0.655** |
+| paraphrase-multilingual-mpnet-base-v2 | 0.864/1.000/0.931 | 0.722/0.975/0.839 | 0.686/0.950/0.806 | 0.831/0.937/0.880 | 0.665/0.998/0.830 | **0.857** |
+| paraphrase-multilingual-MiniLM-L12-v2 | 0.854/0.997/0.924 | 0.696/0.964/0.819 | 0.621/0.908/0.753 | 0.782/0.907/0.841 | 0.600/0.998/0.796 | **0.827** |
+| all-MiniLM-L6-v2 | 0.856/0.999/0.927 | 0.025/0.109/0.144 | 0.028/0.088/0.140 | 0.065/0.172/0.186 | 0.696/1.000/0.848 | **0.449** |
+> ✅ **mentee-embed-v3 beats all-MiniLM-L6-v2** (0.655 vs 0.449) on Protocol A avg MRR@10 — despite all-MiniLM being a pretrained model.
 ---
 ## Usage
 ```python
+# pip install torch transformers tokenizers huggingface_hub
+from transformers import AutoModel, AutoTokenizer
+tok   = AutoTokenizer.from_pretrained("MenteEAI/mentee-embed-v3", trust_remote_code=True)
+model = AutoModel.from_pretrained("MenteEAI/mentee-embed-v3",   trust_remote_code=True)
 sentences = [
     "Hello, how are you?",
     "ہیلو، آپ کیسے ہیں؟"
 ]
+embeddings = model.encode(sentences, tokenizer=tok)
+print(embeddings.shape)  # torch.Size([3, 384])
 ```
+> `trust_remote_code=True` is required — standard for custom-architecture models on HuggingFace. The code runs entirely on your machine.
 ### Similarity search
 ```python
+from transformers import AutoModel, AutoTokenizer
+tok   = AutoTokenizer.from_pretrained("MenteEAI/mentee-embed-v3", trust_remote_code=True)
+model = AutoModel.from_pretrained("MenteEAI/mentee-embed-v3",   trust_remote_code=True)
+query    = model.encode(["What is machine learning?"], tokenizer=tok)
+passages = model.encode([
     "Machine learning is a subset of artificial intelligence.",
     "The weather today is sunny.",
     "تعلم الآلة هو فرع من فروع الذكاء الاصطناعي.",
+], tokenizer=tok)
+scores = query @ passages.T
+print(scores)  # tensor([[0.81, 0.60, 0.79]])
 ```
 ---
 ```bibtex
 @misc{mentee-embed-v3-2026,
+  title   = {How Far Can Multilingual Text Embeddings Be Trained From Scratch?
+             A Compute-Efficient Study of Arabic, English, and Urdu},
+  author  = {Shah, Syed Syab Ahmad and Sania, Shakeel and Hamza, Rustam and Mahboob, Iqbal},
   year    = {2026},
+  doi     = {10.5281/zenodo.22117673},
+  url     = {https://doi.org/10.5281/zenodo.22117673},
+  note    = {MenteE AI. Apache-2.0 License}
 }
 ```
 ## About MenteE AI
+Built by **Syed Syab Ahmad Shah** and Team MenteE AI.
+🌐 [menteeai.org](https://menteeai.org) · 📧 syab@menteeai.org
+Research paper: [10.5281/zenodo.22117673](https://doi.org/10.5281/zenodo.22117673)