ce-cat-distilbert

MiniLM-L12 cross-encoder reranker trained under the configuration stated in §4 of Askari et al., "Injecting the BM25 Score as Text Improves BERT-Based Re-rankers", ECIR 2023: cross-entropy loss with early stopping on validation nDCG@10.

Paired with Amdestya/ce-bm25cat-distilbert — identical data, schedule and seed; the arms differ only in the input construction.

Input format

text_a = f"{query}"
text_b = passage
# tokenises to: [CLS] query [SEP] passage [SEP]

Matched no-injection baseline. Do NOT inject anything into text_a -- this model has never seen a numeric token there.

Usage

from sentence_transformers import CrossEncoder
model = CrossEncoder("Amdestya/ce-cat-distilbert", max_length=256)

Outputs are raw logits (identity activation), unbounded, higher = more relevant.

Training

objective	cross-entropy on binary relevance
optimizer	Adam, lr 7e-06
batch size	32
length caps	query 30, passage 200 tokens
stopping	early stopping on msmarco-passage/trec-dl-2020/judged nDCG@10, patience 3
steps	100000
best validation nDCG@10	0.7016
seed	42

Built as a reference artifact for a dissertation study on reproducing IR papers with LLMs.