---
license: apache-2.0
language:
- en
- ar
- ur
tags:
- embeddings
- sentence-similarity
- retrieval
- contrastive-learning
- multilingual
---

# mentee-embed-v2 (125M) — trilingual embeddings trained from scratch

A compact **125M-parameter, 768-dim** text embedding model for **Arabic, English and Urdu**, trained entirely from scratch (no pretrained base) in two stages on a single consumer GPU.

## Recipe
1. **Stage A — Masked language modeling** on ~31M tokens across the three languages
2. **Stage B — Contrastive fine-tuning** (symmetric InfoNCE, in-batch negatives + hard-negative column, temperature 0.02) on 1797858 triplets incl. **300K English↔Urdu parallel pairs (OPUS-100)**

## Results (Protocol B — 15K-doc corpus-pool retrieval, higher is better)
| 1 | paraphrase-multilingual-mpnet-base-v2 | MRR 0.904 · R@5 0.960 · R@100 0.997 | MRR 0.677 · R@5 0.763 · R@100 0.917 | MRR 0.556 · R@5 0.623 · R@100 0.847 | **0.712** |
| 2 | paraphrase-multilingual-MiniLM-L12-v2 | MRR 0.896 · R@5 0.943 · R@100 0.993 | MRR 0.659 · R@5 0.730 · R@100 0.930 | MRR 0.499 · R@5 0.577 · R@100 0.800 | **0.685** |
| 3 | all-MiniLM-L6-v2 | MRR 0.916 · R@5 0.967 · R@100 1.000 | MRR 0.100 · R@5 0.000 · R@100 0.000 | MRR 0.100 · R@5 0.000 · R@100 0.007 | **0.372** |
| 4 | mentee-embed-v2 (ours) ⭐ **ours** | MRR 0.236 · R@5 0.280 · R@100 0.747 | MRR 0.116 · R@5 0.040 · R@100 0.330 | MRR 0.124 · R@5 0.050 · R@100 0.400 | **0.159** |

Our model reaches **0.159** vs leader paraphrase-multilingual-mpnet-base-v2 at **0.712** on this suite.
*Protocol details: queries ranked against the full pool; baselines run with their recommended usage (E5 query/passage prefixes applied).*

## Usage
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("<your-hf-user>/mentee-embed-v1")
emb = model.encode(["مرحبا بالعالم", "hello world", "دنیا میں خوش آمدید"])
```

## Limitations
NLI+parallel derived training data; small-pool protocol is easier than full MIRACL; expect open-domain gaps versus web-scale models.