MargEmbed-Akan

MargEmbed-Akan is a 560M-parameter embedding model for Twi (Akan) → English retrieval. A question written in Twi finds the right English passage directly, without translating the question first, which makes it suitable for retrieval-augmented generation (RAG), search and voice assistants for Twi speakers.

It is a single model: use it to encode both your questions and your passages. If you already have a passage index built with McGill-NLP/AfriE5-Large-instruct, you can keep it and use MargEmbed-Akan for the questions only, with equivalent results.

📄 Paper: MargEmbed-Akan: Bridging Twi Questions and English Knowledge through Cross-Lingual Score Distillation · 💻 Code: github.com/princeannan4332/MargEmbed-Akan

Results

All systems were evaluated in a single run with identical queries and corpora (nDCG@10 and Recall@1, %).

AfriQA: 1,221 Twi questions written by native speakers; each must retrieve the English Wikipedia passage that answers it among 5,357 passages.

Model	nDCG@10	Recall@1	nDCG@10, Twi typed without ɛ/ɔ	nDCG@10, English questions
MargEmbed-Akan	71.5	60.9	69.5	84.7
gemini-embedding-001	66.6	54.1	62.0	86.9
AfriE5-Large-instruct (base)	66.6	53.8	61.5	85.2
multilingual-e5-large-instruct	59.1	44.9	53.8	84.1
BGE-M3	58.9	45.8	55.2	84.2
EmbeddingGemma 2	56.6	41.3	55.8	85.1
OpenAI text-embedding-3-large	53.4	40.7	53.0	86.2
Qwen3-VL-Embedding-2B	51.3	37.6	49.4	82.4

Bible translation retrieval: 1,000 passages must retrieve their translation among 6,195 passages in the other language (Recall@1).

Model	Twi → English	Twi without ɛ/ɔ → English	English → Twi
MargEmbed-Akan	94.6	87.6	93.6
gemini-embedding-001	93.7	78.2	89.7
AfriE5-Large-instruct (base)	91.4	62.5	91.7
OpenAI text-embedding-3-large	24.8	24.0	22.6

Latency (one NVIDIA T4 GPU, half precision): 17.8 ms per question end to end (median; 23.1 ms at p95), up to 480 questions per second with batching.

Highlights:

• Best system tested on Twi → English retrieval: +4.9 nDCG@10 over gemini-embedding-001 and +18.1 over OpenAI text-embedding-3-large on AfriQA (both significant).
• Most robust to Twi typed without ɛ and ɔ, as on standard keyboards: +7.5 over gemini-embedding-001.
• English retrieval does not differ significantly from the base model; the proprietary models remain 1.5–2.2 points better on English-only search.
• No AfriQA data was used for training or model selection. Results hold when AfriQA questions whose gold passage resembles a training passage are removed (71.6 nDCG@10 on the remaining 1,015 questions).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("DexelLabs/MargEmbed-Akan")
prompt = "Instruct: Given a question, retrieve Wikipedia passages that answer the question\nQuery: "

questions = ["Kurow bɛn ne Old Tafo ahenkurow?"]
passages = [
    "Tafo. Tafo is a town in Kumasi Metropolitan District in the Ashanti Region of Ghana ...",
    "Dormaa Ahenkro. Dormaa Ahenkro is a city and the capital of Dormaa Traditional Area ...",
]

q = model.encode(questions, prompt=prompt, normalize_embeddings=True)   # questions: with the instruction
p = model.encode(passages, normalize_embeddings=True)                   # passages: no instruction
print(q @ p.T)   # higher = more relevant

Always use the instruction for questions and no instruction for passages. Encode your passages once, store the vectors (e.g. in FAISS, Qdrant or Chroma), and encode each incoming question at search time.

Training

MargEmbed-Akan is trained by cross-lingual listwise score distillation from AfriE5-Large-instruct (the frozen teacher):

1. Data: English question → passage pairs from Natural Questions, TriviaQA, SQuAD and GooAQ, cleaned, deduplicated, filtered against AfriQA questions, and sampled to favour questions that share few words with their answer (55% low, 34% medium, 11% high keyword overlap).
2. Translation: questions translated into Asante Twi with NLLB-200 3.3B and kept only if their back-translation into English still matched the original (round-trip check); 35,495 questions kept.
3. Hard negatives: for each English question, the teacher scored its answer and 7 similar but wrong passages.
4. Objective: during training the teacher encodes the passages (targets) and the student encodes the questions (source). The student learns to reproduce the teacher's score distribution over the 8 passages from the Twi question (Twi → English) and from the English question (English → English, which prevents English drift). A light alignment loss on 5,000 GhanaNLP English–Twi sentence pairs, plus copies of questions without ɛ/ɔ and with look-alike characters (ε, ᴐ).
5. Setup: 2 epochs (412 steps), batch size 256, learning rate 1e-5, 8-bit AdamW, bfloat16, one NVIDIA L4 GPU (about 34 minutes).

Passage encoding was not trained directly; it was verified by evaluation (71.5 nDCG@10 with MargEmbed-Akan passages vs. 71.3 with AfriE5 passages on AfriQA).

Limitations

• Trained on machine-translated Twi, so translation errors may remain in the training data.
• Evaluated on Asante Twi; other Akan varieties (Akuapem, Fante) are untested.
• Optimized for Twi questions over English passages; for English-only search, the base model or proprietary models are equal or slightly better.
• The AfriQA evaluation corpus has 5,357 passages; scores on a full Wikipedia-scale index will be lower.

License

The training data includes NLLB-200 translations (CC BY-NC 4.0), so this model is released for research and non-commercial use.

Citation

@misc{annan2026margembed,
  title  = {MargEmbed-Akan: Bridging Twi Questions and English Knowledge through Cross-Lingual Score Distillation},
  author = {Annan, Prince N. Y. and Aziz, Ali A. A.},
  year   = {2026},
  note   = {University of Cape Coast. https://github.com/princeannan4332/MargEmbed-Akan}
}