ce-bm25cat-distilbert

MiniLM-L12 cross-encoder reranker trained under the configuration stated in §4 of Askari et al., "Injecting the BM25 Score as Text Improves BERT-Based Re-rankers", ECIR 2023: cross-entropy loss with early stopping on validation nDCG@10.

Paired with Amdestya/ce-cat-distilbert — identical data, schedule and seed; the arms differ only in the input construction.

Input format

text_a = f"{query} [SEP] {bm25_score}"
text_b = passage
# tokenises to: [CLS] query [SEP] score [SEP] passage [SEP]

Paper Eq. 3 ordering: the score sits BETWEEN query and passage. This differs from Amdestya/bm25cat-minilm-l12, which follows the authors' released code and puts the score BEFORE the query. The two are not interchangeable.

Normalise the score the same way

bm25_score = int((raw_bm25 / 50) * 100)   # global Min-Max, fixed constants, then x100 as an integer

Min-Max in the global setting (fixed min=0, max=50, not per-query), scaled by 100 and truncated to an integer — paper §3.3 row (f). Training scores came from Anserini/Lucene BM25 with k1=0.82, b=0.68. Other BM25 implementations produce a different score distribution.

Usage

from sentence_transformers import CrossEncoder
model = CrossEncoder("Amdestya/ce-bm25cat-distilbert", max_length=256)

Outputs are raw logits (identity activation), unbounded, higher = more relevant.

Training

objective	cross-entropy on binary relevance
optimizer	Adam, lr 7e-06
batch size	32
length caps	query 30, passage 200 tokens
stopping	early stopping on msmarco-passage/trec-dl-2020/judged nDCG@10, patience 3
steps	120000
best validation nDCG@10	0.7104
seed	42

Built as a reference artifact for a dissertation study on reproducing IR papers with LLMs.