tevatron3-reranker-8b-contrastive-hf

A pointwise passage reranker built on Qwen3-8B (dense), trained with the Tevatron 3.0 toolkit.

Backbone: Qwen3-8B (dense)
Training backend: HF FSDP1
Objective: contrastive
Parameter efficiency: full fine-tuning
Training data: RLHN-680K

Scoring contract (important)

This checkpoint is a sequence-classification scorer, saved as Qwen3ForSequenceClassification with num_labels=1. The relevance score of a (query, passage) pair is the single regression logit read at the last (EOS) token. Training builds the pair as "query: {{query}} passage: {{title}} {{text}}" (no yes/no suffix) and appends EOS; score = logits[:, 0].

This differs from the Megatron causal-LM rerankers in this release, which score logit(" yes") − logit(" no") at a prompt suffix. Load this one with AutoModelForSequenceClassification, not AutoModelForCausalLM.

Usage

Score with Tevatron's reranker eval backend (see the Tevatron repo), or directly:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

name = "brutusxu/tevatron3-reranker-8b-contrastive-hf"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(
    name, num_labels=1, dtype=torch.bfloat16).cuda().eval()
pair = "query: what is the capital of france passage: Paris is the capital of France."
ids = tok(pair + tok.eos_token, return_tensors="pt").to("cuda")
with torch.no_grad():
    print(model(**ids).logits[0, 0].item())