Text Ranking
sentence-transformers
Safetensors
Korean
English
qwen3
cross-encoder
reranker
korean
english
Instructions to use nlpai-lab/KURE-Reranker-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use nlpai-lab/KURE-Reranker-base with sentence-transformers:
from sentence_transformers import CrossEncoder model = CrossEncoder("nlpai-lab/KURE-Reranker-base") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) - Notebooks
- Google Colab
- Kaggle
Show usage unfolded, fold the per-dataset tables
Browse files
README.md
CHANGED
|
@@ -51,8 +51,7 @@ pip install "sentence-transformers>=5.6.1" torch
|
|
| 51 |
|
| 52 |
The prompt template ships with the model. The default instruction is *"Given a web search query, retrieve relevant passages that answer the query"*, the same instruction used in training.
|
| 53 |
|
| 54 |
-
|
| 55 |
-
<summary>Sentence Transformers</summary>
|
| 56 |
|
| 57 |
```python
|
| 58 |
import torch
|
|
@@ -79,10 +78,7 @@ pairs = [(query, document) for document in documents]
|
|
| 79 |
scores = model.predict(pairs, batch_size=8)
|
| 80 |
```
|
| 81 |
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
<details>
|
| 85 |
-
<summary>Transformers</summary>
|
| 86 |
|
| 87 |
```python
|
| 88 |
import torch
|
|
@@ -130,8 +126,6 @@ for index in torch.argsort(scores, descending=True).tolist():
|
|
| 130 |
print(f"{scores[index]:.4f}\t{documents[index]}")
|
| 131 |
```
|
| 132 |
|
| 133 |
-
</details>
|
| 134 |
-
|
| 135 |
## Korean Reranking Evaluation
|
| 136 |
|
| 137 |
Scores are **nDCG@10**, and throughput is pairs per second (**PPS**).
|
|
@@ -167,7 +161,8 @@ PPS is measured on a single NVIDIA RTX A6000 48GB per model and averaged over th
|
|
| 167 |
|
| 168 |
`jinaai/jina-reranker-v3` and `jinaai/jina-reranker-v3.5` are listwise rerankers that cannot be compared under the same 8,192-token condition on MLDR, so they are excluded from the 9-subset table and their MLDR cells are left blank below.
|
| 169 |
|
| 170 |
-
|
|
|
|
| 171 |
|
| 172 |
| Model | Params | Ko-StrategyQA | AutoRAGRetrieval | PublicHealthQA | BelebeleRetrieval | MIRACLRetrieval | MrTidyRetrieval | MultiLongDocRetrieval | SQuADKorV1Retrieval | LawIRKo |
|
| 173 |
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -191,7 +186,10 @@ PPS is measured on a single NVIDIA RTX A6000 48GB per model and averaged over th
|
|
| 191 |
| Dongjin-kr/ko-reranker | 560M | 0.8468 | 0.9014 | 0.6976 | 0.9655 | 0.8017 | 0.7772 | 0.3721 | 0.9785 | 0.7343 |
|
| 192 |
| telepix/PIXIE-Spell-Reranker-Preview-0.6B | 596M | 0.8329 | 0.9794 | 0.8211 | 0.9689 | 0.8449 | 0.7650 | 0.1829 | 0.9850 | 0.6042 |
|
| 193 |
|
| 194 |
-
|
|
|
|
|
|
|
|
|
|
| 195 |
|
| 196 |
| Model | Params | Ko-StrategyQA | AutoRAGRetrieval | PublicHealthQA | BelebeleRetrieval | MIRACLRetrieval | MrTidyRetrieval | MultiLongDocRetrieval | SQuADKorV1Retrieval | LawIRKo |
|
| 197 |
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -217,6 +215,8 @@ PPS is measured on a single NVIDIA RTX A6000 48GB per model and averaged over th
|
|
| 217 |
|
| 218 |
Batch size starts at 8 and doubles until an out-of-memory error. Samples are repeated to fill complete batches. After warmup, three full passes are timed with CUDA events. Throughput is the total number of processed pairs divided by the accumulated GPU forward time. The highest-throughput successful batch is reported. Tokenization, data loading, and CPU preprocessing are excluded. The batch-search approach is informed by the [Ettin reranker speed benchmark](https://huggingface.co/blog/ettin-reranker#speed).
|
| 219 |
|
|
|
|
|
|
|
| 220 |
## Citation
|
| 221 |
|
| 222 |
```bibtex
|
|
|
|
| 51 |
|
| 52 |
The prompt template ships with the model. The default instruction is *"Given a web search query, retrieve relevant passages that answer the query"*, the same instruction used in training.
|
| 53 |
|
| 54 |
+
### Sentence Transformers
|
|
|
|
| 55 |
|
| 56 |
```python
|
| 57 |
import torch
|
|
|
|
| 78 |
scores = model.predict(pairs, batch_size=8)
|
| 79 |
```
|
| 80 |
|
| 81 |
+
### Transformers
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
```python
|
| 84 |
import torch
|
|
|
|
| 126 |
print(f"{scores[index]:.4f}\t{documents[index]}")
|
| 127 |
```
|
| 128 |
|
|
|
|
|
|
|
| 129 |
## Korean Reranking Evaluation
|
| 130 |
|
| 131 |
Scores are **nDCG@10**, and throughput is pairs per second (**PPS**).
|
|
|
|
| 161 |
|
| 162 |
`jinaai/jina-reranker-v3` and `jinaai/jina-reranker-v3.5` are listwise rerankers that cannot be compared under the same 8,192-token condition on MLDR, so they are excluded from the 9-subset table and their MLDR cells are left blank below.
|
| 163 |
|
| 164 |
+
<details>
|
| 165 |
+
<summary>Per-dataset NDCG@10</summary>
|
| 166 |
|
| 167 |
| Model | Params | Ko-StrategyQA | AutoRAGRetrieval | PublicHealthQA | BelebeleRetrieval | MIRACLRetrieval | MrTidyRetrieval | MultiLongDocRetrieval | SQuADKorV1Retrieval | LawIRKo |
|
| 168 |
|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
| 186 |
| Dongjin-kr/ko-reranker | 560M | 0.8468 | 0.9014 | 0.6976 | 0.9655 | 0.8017 | 0.7772 | 0.3721 | 0.9785 | 0.7343 |
|
| 187 |
| telepix/PIXIE-Spell-Reranker-Preview-0.6B | 596M | 0.8329 | 0.9794 | 0.8211 | 0.9689 | 0.8449 | 0.7650 | 0.1829 | 0.9850 | 0.6042 |
|
| 188 |
|
| 189 |
+
</details>
|
| 190 |
+
|
| 191 |
+
<details>
|
| 192 |
+
<summary>Per-dataset PPS</summary>
|
| 193 |
|
| 194 |
| Model | Params | Ko-StrategyQA | AutoRAGRetrieval | PublicHealthQA | BelebeleRetrieval | MIRACLRetrieval | MrTidyRetrieval | MultiLongDocRetrieval | SQuADKorV1Retrieval | LawIRKo |
|
| 195 |
|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
| 215 |
|
| 216 |
Batch size starts at 8 and doubles until an out-of-memory error. Samples are repeated to fill complete batches. After warmup, three full passes are timed with CUDA events. Throughput is the total number of processed pairs divided by the accumulated GPU forward time. The highest-throughput successful batch is reported. Tokenization, data loading, and CPU preprocessing are excluded. The batch-search approach is informed by the [Ettin reranker speed benchmark](https://huggingface.co/blog/ettin-reranker#speed).
|
| 217 |
|
| 218 |
+
</details>
|
| 219 |
+
|
| 220 |
## Citation
|
| 221 |
|
| 222 |
```bibtex
|