GoktugD/DUSUNEN-Mercek-118M-v1

DUSUNEN Mercek 118M v1

A Turkish learning-to-rank cross-encoder that improves its untouched multilingual base on a frozen held-out candidate list. Mercek reads a query and document together and should be applied after a fast first-stage retriever.

Held-out top-100 reranking

DUSUNEN Rota 270M v2 produced one frozen top-100 list for 1,024 untouched TurHistQuad queries. Both cross-encoders scored the same 102,400 pairs.

Stage	MRR@10	nDCG@10	Recall@10	Recall@100
Retriever only	0.573709	0.422559	0.475098	0.684082
Untouched multilingual base	0.770884	0.529506	0.515137	0.684082
DUSUNEN Mercek v1	0.772386	0.530312	0.515137	0.684082

Held-out gains over the untouched reranker are +0.001502 MRR@10 and +0.000806 nDCG@10. Raw metrics and timing are included in held-out-top100.json.

Use

from sentence_transformers import CrossEncoder

model = CrossEncoder("GoktugD/DUSUNEN-Mercek-118M-v1")
pairs = [
    ["Hard negative neden önemlidir?", "Zor negatifler karar sınırını güçlendirir."],
    ["Hard negative neden önemlidir?", "Ankara Türkiye'nin başkentidir."],
]
print(model.predict(pairs))

Scores are ranking signals, not calibrated probabilities. Use Mercek on a small candidate list rather than an entire corpus.

Training and validation

• Pinned base revision: 1427fd652930e4ba29e8149678df786c240d8825
• 50,000 lists, each containing one positive and one mined hard negative
• LambdaLoss at k=2; maximum length 256; effective batch size 32
• Learning rate 2e-6; one epoch; BF16; seed 3407
• Validation MRR@2: 0.985750 → 0.986250
• Validation nDCG@2: 0.989574 → 0.989851
• Hardware: NVIDIA GeForce RTX 3090 24 GB
• Training time: 1443.7 seconds
• Peak training allocation: 1.274 GiB

Limitations

• The held-out result covers one Turkish retrieval benchmark and one fixed first-stage retriever; other domains require evaluation.
• Hard negatives can include false negatives or retriever-specific bias.
• Reranking adds latency and cannot recover a relevant document missing from the first-stage candidate list.
• Scores need application-specific threshold calibration.

Integrity

model.safetensors SHA-256:

62bde1977113e9870133b733ce681c4767ae4425c18317ec8b0ba7d270d7d43d

Developed and released by Göktuğ Düşünen.