| --- |
| license: apache-2.0 |
| language: |
| - en |
| - ar |
| - ur |
| tags: |
| - embeddings |
| - sentence-similarity |
| - retrieval |
| - contrastive-learning |
| - multilingual |
| --- |
| |
| # mentee-embed-v2 (125M) — trilingual embeddings trained from scratch |
|
|
| A compact **125M-parameter, 768-dim** text embedding model for **Arabic, English and Urdu**, trained entirely from scratch (no pretrained base) in two stages on a single consumer GPU. |
|
|
| ## Recipe |
| 1. **Stage A — Masked language modeling** on ~31M tokens across the three languages |
| 2. **Stage B — Contrastive fine-tuning** (symmetric InfoNCE, in-batch negatives + hard-negative column, temperature 0.02) on 1797858 triplets incl. **300K English↔Urdu parallel pairs (OPUS-100)** |
|
|
| ## Results (Protocol B — 15K-doc corpus-pool retrieval, higher is better) |
| | 1 | paraphrase-multilingual-mpnet-base-v2 | MRR 0.904 · R@5 0.960 · R@100 0.997 | MRR 0.677 · R@5 0.763 · R@100 0.917 | MRR 0.556 · R@5 0.623 · R@100 0.847 | **0.712** | |
| | 2 | paraphrase-multilingual-MiniLM-L12-v2 | MRR 0.896 · R@5 0.943 · R@100 0.993 | MRR 0.659 · R@5 0.730 · R@100 0.930 | MRR 0.499 · R@5 0.577 · R@100 0.800 | **0.685** | |
| | 3 | all-MiniLM-L6-v2 | MRR 0.916 · R@5 0.967 · R@100 1.000 | MRR 0.100 · R@5 0.000 · R@100 0.000 | MRR 0.100 · R@5 0.000 · R@100 0.007 | **0.372** | |
| | 4 | mentee-embed-v2 (ours) ⭐ **ours** | MRR 0.236 · R@5 0.280 · R@100 0.747 | MRR 0.116 · R@5 0.040 · R@100 0.330 | MRR 0.124 · R@5 0.050 · R@100 0.400 | **0.159** | |
|
|
| Our model reaches **0.159** vs leader paraphrase-multilingual-mpnet-base-v2 at **0.712** on this suite. |
| *Protocol details: queries ranked against the full pool; baselines run with their recommended usage (E5 query/passage prefixes applied).* |
|
|
| ## Usage |
| ```python |
| from sentence_transformers import SentenceTransformer |
| model = SentenceTransformer("<your-hf-user>/mentee-embed-v1") |
| emb = model.encode(["مرحبا بالعالم", "hello world", "دنیا میں خوش آمدید"]) |
| ``` |
|
|
| ## Limitations |
| NLI+parallel derived training data; small-pool protocol is easier than full MIRACL; expect open-domain gaps versus web-scale models. |
|
|
