bahith-arabic-retrieval-4b

Arabic passage-retrieval embedding model (query → relevant passage), fine-tuned from Qwen/Qwen3-Embedding-4B with Matryoshka Representation Learning.

On MIRACLRetrievalHardNegatives (Arabic) it reaches 0.7971 nDCG@10, outperforming intfloat/multilingual-e5-large (0.7658) — while its embeddings stay truncatable via Matryoshka Representation Learning.

Highlights

• 🇸🇦 Arabic-first retrieval — trained on human-annotated MSA data (MIRACL, Mr. TyDi).
• 🪆 Matryoshka embeddings — truncate to any of [2560, 1024, 512, 256, 128] dimensions (and re-normalize) with graceful quality loss.
• 🎯 Instruction-aware queries — the correct query/document prompts are baked in and applied automatically.
• 📐 Output dimension: 2560 · Max sequence length: 512 · Similarity: cosine.

Results

Evaluated with MTEB on Arabic retrieval tasks (nDCG@10 / MRR@10). Baselines are run under identical conditions.

MIRACLRetrievalHardNegatives — subset ar, split dev

Model	nDCG@10	MRR@10	dim
qwen3-embedding-4b-arabic-matryoshka (ours)	0.7971	0.8138	2560
qwen3-embedding-4b-arabic-matryoshka (ours) @dim=1024	0.7930	0.8091	1024
qwen3-embedding-4b-arabic-matryoshka (ours) @dim=256	0.7697	0.7877	256
intfloat/multilingual-e5-large	0.7658	0.7989	1024
qwen3-embedding-4b-arabic-matryoshka (ours) @dim=128	0.7469	0.7729	128

SadeemQuestionRetrieval — subset default, split test

Model	nDCG@10	MRR@10	dim
intfloat/multilingual-e5-large	0.6973	0.5955	1024
qwen3-embedding-4b-arabic-matryoshka (ours)	0.6933	0.5842	2560
qwen3-embedding-4b-arabic-matryoshka (ours) @dim=1024	0.6911	0.5879	1024
qwen3-embedding-4b-arabic-matryoshka (ours) @dim=256	0.6745	0.5652	256
qwen3-embedding-4b-arabic-matryoshka (ours) @dim=128	0.6616	0.5612	128

Usage

Install the latest Sentence Transformers:

pip install -U sentence-transformers

Encode queries and documents (prompts are applied automatically):

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("mabdulaziz499/bahith-arabic-retrieval-4b")

# Queries and documents — the correct prompts are stored in the model and applied
# automatically by encode_query / encode_document (no manual prefixing needed).
queries = [
    "ما هي عاصمة مصر؟",
    "من كتب رواية موسم الهجرة إلى الشمال؟",
]
documents = [
    "القاهرة هي عاصمة جمهورية مصر العربية وأكبر مدنها وأكثرها سكانًا.",
    "موسم الهجرة إلى الشمال رواية للكاتب السوداني الطيب صالح صدرت عام 1966.",
    "برج إيفل معلم شهير يقع في مدينة باريس عاصمة فرنسا.",
]

query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)

# Cosine similarity: rows = queries, columns = documents
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)

Matryoshka: smaller embeddings

# Matryoshka: truncate embeddings to a smaller dimension for cheaper storage / faster
# search. The output is re-normalized automatically, so cosine similarity still works.
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("mabdulaziz499/bahith-arabic-retrieval-4b", truncate_dim=256)  # any supported dim
embeddings = model.encode_document(["نص عربي للفهرسة والاسترجاع."])
print(embeddings.shape)  # (1, 256)

The prompts stored in the model (and applied by encode_query / encode_document):

• Query prompt: Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:
• Document prompt: (none)

Choosing an embedding dimension

Matryoshka lets you trade a little accuracy for much cheaper storage and faster search. Pick the smallest dimension that meets your quality bar (see the Results table above).

dim	relative size
2560	100% of full size
1024	40% of full size
512	20% of full size
256	10% of full size
128	5% of full size

Model details

Base model	Qwen/Qwen3-Embedding-4B
Output dimension	2560 (Matryoshka: [2560, 1024, 512, 256, 128])
Max sequence length	512
Similarity function	cosine
Language	Arabic (ar)
Library	sentence-transformers

Training details

• Data: human-annotated Arabic training pairs from MIRACL and Mr. TyDi (query → positive passage).
• Hard negatives: mined with a strong teacher encoder using rank-range and relative-margin filtering to reduce false negatives.
• Loss: MatryoshkaLoss wrapping CachedMultipleNegativesRankingLoss (gradient caching for large effective batches with bounded memory), dims [2560, 1024, 512, 256, 128].
• Schedule: 3 epochs, learning rate 1e-05, cosine schedule with 10% warmup, bf16/fp16 selected by GPU.
• Model selection: best checkpoint by MIRACL-ar dev nDCG@10 (in-loop InformationRetrievalEvaluator).

Full training, evaluation, and publishing code: github.com/m0d9/arabic-retrieval-embeddings

Intended use & limitations

• Intended for: Arabic passage / document retrieval, semantic search, and retrieval-augmented generation over Arabic corpora.
• Limitations: trained primarily on Modern Standard Arabic; dialectal coverage is limited. As with any retriever, validate on your own domain before production use.

License

Released under apache-2.0, matching the base model Qwen/Qwen3-Embedding-4B.

Acknowledgements

Built on Qwen/Qwen3-Embedding-4B; trained and evaluated with sentence-transformers and MTEB.