mist-reranker-150m

A compact English cross-encoder reranker for the second stage of a RAG / search pipeline. Given a query and a candidate passage, it outputs a relevance score to re-sort the top-k from a first-stage retriever (BM25 or a bi-encoder).

Built on answerdotai/ModernBERT-base (~150M) and fine-tuned on olaverse/reranker-general-en-llm-judged. Competitive with purpose-built 150M ModernBERT rerankers on QA/fact retrieval, at equal or smaller size.

📄 Model details

Property	Value
Type	Cross-encoder reranker (single relevance score)
Backbone	answerdotai/ModernBERT-base
Parameters	~150M
Max sequence length	512 tokens
Training data	olaverse/reranker-general-en-llm-judged
Language	English
License	Apache-2.0

🏃 How to run

pip install -U sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("olaverse/mist-reranker-150m")

query = "What causes ocean tides?"
passages = [
    "Tides are caused by the gravitational pull of the Moon and Sun on Earth's oceans.",
    "The Pacific Ocean is the largest of Earth's oceanic divisions.",
    "Tidal energy is a renewable power source generated from the movement of tides.",
]
scores = model.predict([(query, p) for p in passages])
for s, p in sorted(zip(scores, passages), reverse=True):
    print(f"{s:.3f}  {p}")

📈 Performance

NanoBEIR nDCG@10, all models run through the same evaluation harness (CrossEncoderNanoBEIREvaluator). Every task is zero-shot — held out from training for every model. Tasks are split into in-domain QA/fact and out-of-domain argument retrieval.

Model	params	nfcorpus	scifact	fiqa	dbpedia	QA mean	arguana	touche2020	arg mean
granite-embedding-reranker-r2	150M	0.437	0.811	0.561	0.723	0.633	0.555	0.597	0.576
mist-reranker-150m	150M	0.440	0.785	0.578	0.711	0.628	0.421	0.581	0.501
ms-marco-MiniLM-L12-v2	33M	0.399	0.738	0.514	0.713	0.591	0.365	0.602	0.483
bge-reranker-base	278M	0.385	0.734	0.447	0.697	0.566	0.360	0.422	0.391

On QA/fact retrieval, mist-reranker-150m (0.628) is within ~0.005 of granite (0.633) and ahead of bge-reranker-base (0.566) — 1.8× larger — and MiniLM-L12. It was trained on a fully-disclosed, MS-MARCO-free data mix; the comparison models use larger undisclosed training data. The parity is achieved with cleaner, verifiable data provenance.

🧪 Training

• Base: answerdotai/ModernBERT-base, trained from scratch into a reranker.
• Data: LLM-judged graded 0–3 relevance labels (Qwen2.5-72B, UMBRELA-style rubric), validated at Cohen's κ (quadratic) = 0.491 vs TREC-DL human qrels.
• Loss: BinaryCrossEntropyLoss on labels normalized to [0,1]; 2 epochs, best checkpoint by NanoBEIR mean nDCG@10.
• Raw ModernBERT-base scored 0.108 mean nDCG@10 before training; the dataset does effectively all the reranking learning.

License

Released under Apache-2.0.

Citation

@misc{mist-reranker-150m,
  title  = {mist-reranker-150m},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/mist-reranker-150m}
}