mist-reranker-22.7M

A compact, fast English cross-encoder reranker. It scores a (query, passage) pair directly and reorders the top-k candidates from a first-stage retriever — the second stage of a search / RAG pipeline. At ~22.7M parameters it runs comfortably on a single modest GPU or CPU, and recovers most of the ranking quality of rerankers an order of magnitude larger.

📄 Model details

Property	mist-reranker-22.7M
Type	Cross-encoder reranker (single relevance score)
Total parameters	~22.7M
Backbone	cross-encoder/ms-marco-MiniLM-L6-v2 (MiniLM-L6-H384)
Layers	6
Hidden size	384
Vocabulary size	30,522
Output	2-class logits → relevance = softmax(logits)[:, 1]
Max sequence length	512
Training precision	BF16
Language	English
License	Apache-2.0

Training: fine-tuned on olaverse/reranker-general-en-llm-judged (pairs-graded, 844k pairs) with a hybrid objective — binary cross-entropy on the LLM-judge relevance label, plus an auxiliary term distilling the continuous teacher score from BAAI/bge-reranker-v2-m3.

🏃 How to run

Install sentence-transformers:

pip install -U sentence-transformers

This is a 2-class head, so relevance is the positive-class probability, softmax(logits)[:, 1]:

import torch
from sentence_transformers import CrossEncoder

model = CrossEncoder("olaverse/mist-reranker-22.7M")

query = "who wrote hamlet"
passages = [
    "Hamlet is a tragedy written by William Shakespeare around 1600.",
    "The capital of France is Paris.",
    "Macbeth is one of Shakespeare's shortest tragedies.",
]

logits = model.predict([[query, p] for p in passages], convert_to_tensor=True)
scores = torch.softmax(logits, dim=-1)[:, 1]          # relevance = P(relevant)

for p, s in sorted(zip(passages, scores.tolist()), key=lambda x: -x[1]):
    print(f"{s:.4f}  {p}")

To rerank a retrieved candidate list, score every candidate against the query and sort by the relevance score descending. Keep query first and passage second in each pair — the model is trained on that order.

📈 Performance

NanoBEIR (NanoNQ, NanoHotpotQA, NanoFEVER), NDCG@10. Every reranker reorders the same candidate sets; the candidate order before reranking (first-stage floor) scores 0.7126.

Model	Params	NDCG@10
BAAI/bge-reranker-v2-m3	~568M	0.9058
cross-encoder/ms-marco-MiniLM-L12-v2	~33M	0.8670
mist-reranker-22.7M	~22.7M	0.8543
cross-encoder/ms-marco-MiniLM-L6-v2	~22.7M	0.8495
BAAI/bge-reranker-base	~278M	0.8238
mixedbread-ai/mxbai-rerank-xsmall-v1	~70M	0.8184
first-stage floor (no reranker)	—	0.7126

At 22.7M it lifts NDCG@10 by +0.14 over the first-stage candidate order, beats bge-reranker-base (12× larger) and mxbai-rerank-xsmall-v1, and closes most of the gap to its bge-reranker-v2-m3 teacher.

License

Released under Apache-2.0.

Citation

@misc{mist-reranker-22.7M,
  title  = {mist-reranker-22.7M},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/mist-reranker-22.7M}
}