---
language:
  - en
  - ms
tags:
  - sentence-transformers
  - embedding
  - retrieval
  - rag
  - malaysian
  - manglish
  - multilingual
license: mit
model_type: sentence-transformers
base_model: paraphrase-multilingual-mpnet-base-v2
---

# Aranda-v1

Aranda-v1 is a sentence embedding model specialized for **Malaysian text retrieval**, including Bahasa Malaysia, Manglish (Malaysian English code-switching), and cross-lingual BM↔English matching. It outperforms all tested baselines on overall retrieval Recall@1, Recall@5, Recall@10, and MRR, and is the best model on every individual language category (BM, Manglish, English, Cross-lingual).

## Training

Two-phase contrastive curriculum:

**Phase 1 — Breadth:** MultipleNegativesRankingLoss on 1M Malaysian positive pairs (paraphrases, social media, news, QA). LR=1e-5, 1000 steps.

**Phase 2 — Discrimination:** Fine-tuned on 58K diverse hard-negative triplets (Lowyat, Twitter, Facebook, formal BM QA, English anchors, cross-lingual pairs) with explicit mined hard negatives. LR=2e-6, 2000 steps.

Base model: `paraphrase-multilingual-mpnet-base-v2`

## Evaluation

Tested on 4,149 retrieval queries (BM, Manglish, English, Cross-lingual) with ~25 candidates per query, plus 4 additional eval sets (heuristic similarity, mined holdout, Mesolitica reranker, English STS).

### Overall Retrieval (4,149 queries)

| Model | Recall@1 | Recall@5 | Recall@10 | MRR | Precision@10 |
|---|---:|---:|---:|---:|---:|
| mpnet-base | 0.8631 | 0.9937 | 0.9973 | 0.9200 | 0.1785 |
| multilingual-e5-base | 0.8470 | 0.9940 | **0.9988** | 0.9091 | 0.1792 |
| labse | 0.8009 | 0.9882 | 0.9986 | 0.8821 | 0.1780 |
| distilbert-multilingual-quora | 0.7973 | 0.9636 | 0.9858 | 0.8719 | 0.1728 |
| **Aranda-v1** | **0.8891** | **0.9961** | **0.9998** | **0.9364** | **0.1788** |

### Per-Language Recall@1

| Model | BM | Manglish | English | Cross-lingual |
|---|---:|---:|---:|---:|
| mpnet-base | 0.8200 | 0.8988 | 0.8577 | 0.8267 |
| multilingual-e5-base | 0.7938 | 0.9104 | 0.8658 | 0.6167 |
| labse | 0.7354 | 0.8993 | 0.6872 | 0.7300 |
| distilbert-multilingual-quora | 0.7584 | 0.8320 | 0.7866 | 0.7633 |
| **Aranda-v1** | **0.8431** | **0.9290** | **0.8792** | **0.8500** |

### Heuristic Similarity 2K (per-language Spearman)

| Model | Overall | BM | Manglish |
|---|---:|---:|---:|
| mpnet-base | **0.4799** | **0.5535** | 0.3937 |
| multilingual-e5-base | 0.3582 | 0.2714 | 0.4680 |
| labse | 0.3249 | 0.2866 | 0.4397 |
| distilbert-multilingual-quora | 0.4585 | 0.4857 | 0.4682 |
| Aranda-v1 | 0.4532 | 0.4915 | **0.4536** |

### Mined Holdout 2K (positive vs negative margins)

| Model | pos>neg % | mean margin | mean pos sim | mean neg sim |
|---|---:|---:|---:|---:|
| mpnet-base | 98.1% | **0.4880** | 0.6576 | **0.1696** |
| multilingual-e5-base | 98.15% | 0.1129 | **0.8805** | 0.7676 |
| labse | **98.3%** | 0.4193 | 0.5960 | 0.1768 |
| distilbert-multilingual-quora | 94.6% | 0.1041 | 0.9336 | 0.8295 |
| Aranda-v1 | 98.2% | 0.4318 | 0.7334 | 0.3016 |

### Mesolitica Reranker Test (NDCG@10)

| Model | Overall NDCG@10 |
|---|---:|
| mpnet-base | 0.4699 |
| multilingual-e5-base | 0.4705 |
| labse | 0.4695 |
| distilbert-multilingual-quora | 0.4690 |
| **Aranda-v1** | **0.4706** |

### English STS (mteb/stsbenchmark-sts)

| Model | Spearman |
|---|---:|
| mpnet-base | **0.8682** |
| multilingual-e5-base | 0.8420 |
| labse | 0.7225 |
| distilbert-multilingual-quora | 0.7866 |
| Aranda-v1 | 0.8369 |

## Usage

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("rekabytes/Aranda-v1")

# Encode texts
embeddings = model.encode([
    "macam mana nak renew lesen memandu",
    "how to renew driving license",
    "saya nak makan nasi lemak"
], normalize_embeddings=True)

# Cosine similarity
similarities = embeddings @ embeddings.T
print(similarities)
```

### With RAG / vector search

```python
# Encode your document corpus (do this once)
doc_embeddings = model.encode(documents, normalize_embeddings=True)

# At query time
query_embedding = model.encode([query], normalize_embeddings=True)
scores = query_embedding @ doc_embeddings.T
top_k = scores.argsort()[0][-5:][::-1]
```

## Intended Use

- **RAG context retrieval** for Malaysian applications
- **Semantic search** over BM/Manglish/English document corpora
- **Cross-lingual matching** (BM ↔ English)
- **Dense retrieval** in hybrid search pipelines (paired with BM25)

## Limitations

- English STS performance is below the base mpnet model (0.8369 vs 0.8682) — the model specialized for Malaysian text
- Not a reranker — use a cross-encoder for second-stage reranking
- Tested on Malaysian web data; performance may vary on other Southeast Asian languages

## Model Details

- **Architecture:** XLM-RoBERTa (base)
- **Embedding dimension:** 768
- **Max sequence length:** 128
- **Pooling:** Mean
- **Normalization:** L2