--- language: - el - en license: other license_name: nvidia-open-model-license base_model: nvidia/llama-nemotron-rerank-1b-v2 pipeline_tag: text-ranking library_name: transformers tags: - greek - english - reranker - cross-encoder - rag - nemotron model-index: - name: Sophea-Nemo-Reranker results: - task: type: text-ranking name: Greek+English reranking (5 domains, rerank top-50) dataset: type: retrieval name: Internal Greek+English retrieval benchmark metrics: - type: nDCG@10 name: nDCG@10 (mean of 5 domains) value: 0.8172 - type: recall@10 name: Recall@10 (mean of 5 domains) value: 0.9267 ---

Sophea-Nemo-Reranker

Greek + English cross-encoder reranker — fine-tuned NVIDIA Llama-Nemotron-Rerank-1B-v2

**Sophea-Nemo-Reranker** is a Greek + English **cross-encoder reranker**, fine-tuned from **[NVIDIA llama-nemotron-rerank-1b-v2](https://huggingface.co/nvidia/llama-nemotron-rerank-1b-v2)** for Greek and English reranking. It is the second (precision) stage of a two-stage Greek RAG stack; the matching first stage is **[Sophea-Nemo-Embedding](https://huggingface.co/KIEFERSA/Sophea-Nemo-Embedding)**. - **Creator:** Kiefer SA - **Base model:** `nvidia/llama-nemotron-rerank-1b-v2` (`LlamaBidirectionalForSequenceClassification`, **bidirectional**, single relevance logit; `trust_remote_code`) - **Languages:** Greek + English - **Input template:** a single sequence — `question:{q} \n \n passage:{p}` - **Training:** BCE on labeled (query, doc) pairs, 1 epoch, max-len 4096, bf16 > **Greek is out-of-distribution for the base.** The base reranker lists 26 languages and **Greek is > not one of them** — this fine-tune adapts its trained relevance head to Greek. ## Why it matters — base vs fine-tuned Reranking the fixed top-50 candidates over 5 domains (nDCG@10), the fine-tune improves on the base Nemotron reranker **on every domain**: ![Greek+English reranking — base vs fine-tuned](base_vs_ft.png)

metric	Val	Energy	Legal	Finance	Medical	Mean
nDCG@10 — base	0.793	0.741	0.966	0.702	0.651	0.770
nDCG@10 — fine-tuned	0.874	0.780	0.969	0.754	0.709	0.817
Δ nDCG@10	+0.081	+0.039	+0.004	+0.052	+0.058	+0.047
Recall@10 — base	0.953	0.847	0.993	0.853	0.773	0.884
Recall@10 — fine-tuned	0.993	0.873	0.987	0.907	0.873	0.927

**Read.** Mean **nDCG@10 0.770 → 0.817 (+0.047)**, better on all five domains, with the largest gains on the hardest ones (medical +0.06, finance +0.05, val +0.08). In the two-stage stack it is also the strongest reranker we measured — edging a fine-tuned Qwen3-0.6B cross-encoder and beating the base by a clear margin. ## Size comparison — vs base Qwen3-Reranker (off-the-shelf) Reranking the same fixed top-50 candidates, same protocol (150 queries/domain), nDCG@10 — your **fine-tuned 1B** reranker **edges the base Qwen3-Reranker-8B** and clearly beats 4B / 0.6B, at a fraction of the size:

model	Val	Energy	Legal	Finance	Medical	Mean
Sophea-Nemo-Reranker (ours · 1B · fine-tuned)	0.874	0.780	0.969	0.754	0.709	0.817
Qwen3-Reranker-8B (base, off-the-shelf)	0.858	0.796	0.975	0.763	0.672	0.813
Qwen3-Reranker-4B (base, off-the-shelf)	0.865	0.782	0.966	0.735	0.647	0.799
Qwen3-Reranker-0.6B (base, off-the-shelf)	0.812	0.746	0.960	0.692	0.634	0.769

**Read.** Mean nDCG@10 **0.817** (ours, 1B) vs **0.813** (Qwen3-Reranker-8B), **0.799** (4B), **0.769** (0.6B) — and ours is strongest on the hardest domain (medical 0.709 vs 8B's 0.672). A small, Greek-tuned model beats a general reranker 8× its size. ## End-to-end two-stage pipeline The two models are designed to run **together**: the embedder retrieves the top-50 candidates, the reranker re-orders them. Measured end-to-end on the same 5-domain Greek+English eval (150 queries/set, correct `query:`/`passage:` prompts): ![End-to-end two-stage stack — embedder alone vs + reranker](stack.png)

metric	Val	Energy	Legal	Finance	Medical	Mean
nDCG@10 — embedder alone	0.863	0.802	0.939	0.702	0.689	0.799
nDCG@10 — + reranker (stack)	0.867	0.780	0.976	0.763	0.703	0.818
Δ from reranking	+0.004	-0.022	+0.037	+0.061	+0.013	+0.019
Recall@10 — full stack	0.987	0.873	0.993	0.920	0.860	0.927

**Read.** The full stack reaches **mean nDCG@10 0.818** (Recall@10 0.927) — the strongest configuration measured. The reranker adds the most where the embedder is weakest (**finance +0.061, legal +0.037**), i.e. it sharpens precision on the hard cases. On **energy** it slightly regresses (-0.022) and on `val` the gain is marginal, because the embedder alone is already very strong there and leaves little headroom — for the easiest, already-high-recall domains you can skip the rerank stage if latency matters. ## Usage (transformers) ```python import torch from transformers import AutoTokenizer, AutoModelForSequenceClassification name = "KIEFERSA/Sophea-Nemo-Reranker" tok = AutoTokenizer.from_pretrained(name, trust_remote_code=True) model = AutoModelForSequenceClassification.from_pretrained( name, num_labels=1, dtype=torch.bfloat16, trust_remote_code=True).eval() def score(query, passage): text = f"question:{query} \n \n passage:{passage}" enc = tok(text, return_tensors="pt", truncation=True, max_length=4096) with torch.no_grad(): return model(**enc).logits.squeeze(-1).item() # higher = more relevant # rank a candidate list by descending score ``` Feed the base's own `question:/passage:` template — it has a **trained** relevance head, so matching its template adapts that head rather than fighting it. Needs `trust_remote_code=True`. ## Serving with vLLM ```bash vllm serve KIEFERSA/Sophea-Nemo-Reranker --runner pooling --trust-remote-code ``` Exposes `/rerank`, `/v1/rerank` (Jina), `/v2/rerank` (Cohere) and `/score`: ```bash curl http://localhost:8000/rerank -H 'Content-Type: application/json' -d '{ "model": "KIEFERSA/Sophea-Nemo-Reranker", "query": "Ποια είναι η πρωτεύουσα της Ελλάδας;", "documents": ["Η Αθήνα είναι η πρωτεύουσα της Ελλάδας.", "Το Παρίσι είναι στη Γαλλία."] }' ``` ## License Fine-tune of `nvidia/llama-nemotron-rerank-1b-v2` — governed by the **NVIDIA Open Model License** and the **Llama 3.2 Community License**; commercial use permitted under those terms (note: **not** Apache-2.0). Review the base model's license before use.