KIEFERSA/Sophea-Nemo-Reranker

Sophea-Nemo-Reranker

Greek + English cross-encoder reranker — fine-tuned NVIDIA Llama-Nemotron-Rerank-1B-v2

Sophea-Nemo-Reranker is a Greek + English cross-encoder reranker, fine-tuned from NVIDIA llama-nemotron-rerank-1b-v2 for Greek and English reranking. It is the second (precision) stage of a two-stage Greek RAG stack; the matching first stage is Sophea-Nemo-Embedding.

• Creator: Kiefer SA
• Base model: nvidia/llama-nemotron-rerank-1b-v2 (LlamaBidirectionalForSequenceClassification, bidirectional, single relevance logit; trust_remote_code)
• Languages: Greek + English
• Input template: a single sequence — question:{q} \n \n passage:{p}
• Training: BCE on labeled (query, doc) pairs, 1 epoch, max-len 4096, bf16

Greek is out-of-distribution for the base. The base reranker lists 26 languages and Greek is not one of them — this fine-tune adapts its trained relevance head to Greek.

Why it matters — base vs fine-tuned

Reranking the fixed top-50 candidates over 5 domains (nDCG@10), the fine-tune improves on the base Nemotron reranker on every domain:

metric	Val	Energy	Legal	Finance	Medical	Mean
nDCG@10 — base	0.793	0.741	0.966	0.702	0.651	0.770
nDCG@10 — fine-tuned	0.874	0.780	0.969	0.754	0.709	0.817
Δ nDCG@10	+0.081	+0.039	+0.004	+0.052	+0.058	+0.047
Recall@10 — base	0.953	0.847	0.993	0.853	0.773	0.884
Recall@10 — fine-tuned	0.993	0.873	0.987	0.907	0.873	0.927

Read. Mean nDCG@10 0.770 → 0.817 (+0.047), better on all five domains, with the largest gains on the hardest ones (medical +0.06, finance +0.05, val +0.08). In the two-stage stack it is also the strongest reranker we measured — edging a fine-tuned Qwen3-0.6B cross-encoder and beating the base by a clear margin.

Size comparison — vs base Qwen3-Reranker (off-the-shelf)

Reranking the same fixed top-50 candidates, same protocol (150 queries/domain), nDCG@10 — your fine-tuned 1B reranker edges the base Qwen3-Reranker-8B and clearly beats 4B / 0.6B, at a fraction of the size:

model	Val	Energy	Legal	Finance	Medical	Mean
Sophea-Nemo-Reranker (ours · 1B · fine-tuned)	0.874	0.780	0.969	0.754	0.709	0.817
Qwen3-Reranker-8B (base, off-the-shelf)	0.858	0.796	0.975	0.763	0.672	0.813
Qwen3-Reranker-4B (base, off-the-shelf)	0.865	0.782	0.966	0.735	0.647	0.799
Qwen3-Reranker-0.6B (base, off-the-shelf)	0.812	0.746	0.960	0.692	0.634	0.769

Read. Mean nDCG@10 0.817 (ours, 1B) vs 0.813 (Qwen3-Reranker-8B), 0.799 (4B), 0.769 (0.6B) — and ours is strongest on the hardest domain (medical 0.709 vs 8B's 0.672). A small, Greek-tuned model beats a general reranker 8× its size.

End-to-end two-stage pipeline

The two models are designed to run together: the embedder retrieves the top-50 candidates, the reranker re-orders them. Measured end-to-end on the same 5-domain Greek+English eval (150 queries/set, correct query:/passage: prompts):

metric	Val	Energy	Legal	Finance	Medical	Mean
nDCG@10 — embedder alone	0.863	0.802	0.939	0.702	0.689	0.799
nDCG@10 — + reranker (stack)	0.867	0.780	0.976	0.763	0.703	0.818
Δ from reranking	+0.004	-0.022	+0.037	+0.061	+0.013	+0.019
Recall@10 — full stack	0.987	0.873	0.993	0.920	0.860	0.927

Read. The full stack reaches mean nDCG@10 0.818 (Recall@10 0.927) — the strongest configuration measured. The reranker adds the most where the embedder is weakest (finance +0.061, legal +0.037), i.e. it sharpens precision on the hard cases. On energy it slightly regresses (-0.022) and on val the gain is marginal, because the embedder alone is already very strong there and leaves little headroom — for the easiest, already-high-recall domains you can skip the rerank stage if latency matters.

Usage (transformers)

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

name = "KIEFERSA/Sophea-Nemo-Reranker"
tok = AutoTokenizer.from_pretrained(name, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(
    name, num_labels=1, dtype=torch.bfloat16, trust_remote_code=True).eval()

def score(query, passage):
    text = f"question:{query} \n \n passage:{passage}"
    enc = tok(text, return_tensors="pt", truncation=True, max_length=4096)
    with torch.no_grad():
        return model(**enc).logits.squeeze(-1).item()   # higher = more relevant

# rank a candidate list by descending score

Feed the base's own question:/passage: template — it has a trained relevance head, so matching its template adapts that head rather than fighting it. Needs trust_remote_code=True.

Serving with vLLM

vllm serve KIEFERSA/Sophea-Nemo-Reranker --runner pooling --trust-remote-code

Exposes /rerank, /v1/rerank (Jina), /v2/rerank (Cohere) and /score:

curl http://localhost:8000/rerank -H 'Content-Type: application/json' -d '{
  "model": "KIEFERSA/Sophea-Nemo-Reranker",
  "query": "Ποια είναι η πρωτεύουσα της Ελλάδας;",
  "documents": ["Η Αθήνα είναι η πρωτεύουσα της Ελλάδας.", "Το Παρίσι είναι στη Γαλλία."]
}'

License

Fine-tune of nvidia/llama-nemotron-rerank-1b-v2 — governed by the NVIDIA Open Model License and the Llama 3.2 Community License; commercial use permitted under those terms (note: not Apache-2.0). Review the base model's license before use.