Add comprehensive model card for TFRank
Browse filesThis PR significantly improves the model card for TFRank, including:
- Relevant metadata: `license` (Apache 2.0), `library_name` (transformers), and `pipeline_tag` (text-ranking).
- A clear introduction and description of the model based on the paper abstract and GitHub README.
- Links to the official paper ([TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking](https://huggingface.co/papers/2508.09539)) and the GitHub repository (https://github.com/JOHNNY-fans/TFRank).
- Details on installation, available models, datasets, and performance figures.
- A comprehensive "Inference Quick Start" guide with code snippets, directly from the GitHub README.
- The official BibTeX citation information.
This update will make the TFRank model more discoverable and easier for users to understand and utilize on the Hugging Face Hub.
|
@@ -0,0 +1,168 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: transformers
|
| 4 |
+
pipeline_tag: text-ranking
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking
|
| 8 |
+
|
| 9 |
+
**TFRank** is an efficient pointwise reasoning ranker based on small-scale Large Language Models (LLMs). It addresses the high computational cost and latency of existing reasoning-intensive ranking models by effectively integrating Chain-of-Thought (CoT) data, fine-grained score supervision, and multi-task training. TFRank achieves an efficient "Think-Free" reasoning capability by leveraging explicit reasoning during training while delivering precise relevance scores for complex queries at inference without generating any reasoning chains.
|
| 10 |
+
|
| 11 |
+
This model was presented in the paper [TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking](https://huggingface.co/papers/2508.09539).
|
| 12 |
+
|
| 13 |
+
The source code and additional resources are available on the [GitHub repository](https://github.com/JOHNNY-fans/TFRank).
|
| 14 |
+
|
| 15 |
+
## Introduction
|
| 16 |
+
|
| 17 |
+
Welcome to **TFRank** 🎉
|
| 18 |
+
We propose a training framework **TFRank** for small-scale LLMs that enables **efficient and effective pointwise reasoning ranking** *without explicit reasoning output at inference*.
|
| 19 |
+
|
| 20 |
+
<div align="center">
|
| 21 |
+
<img src="https://github.com/JOHNNY-fans/TFRank/raw/main/figure/framework.png">
|
| 22 |
+
</div>
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
## 📦 Installation
|
| 27 |
+
|
| 28 |
+
Clone repository:
|
| 29 |
+
|
| 30 |
+
```bash
|
| 31 |
+
git clone https://github.com/JOHNNY-fans/TFRank.git
|
| 32 |
+
cd TFRank
|
| 33 |
+
pip install -r requirements.txt
|
| 34 |
+
pip install vllm # Install vLLM for the API server
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
---
|
| 38 |
+
|
| 39 |
+
## Resources
|
| 40 |
+
|
| 41 |
+
### 📦 Models
|
| 42 |
+
| Model | Description |
|
| 43 |
+
|:---------|:------------|
|
| 44 |
+
| [TFRank-SFT-Qwen2.5-7B-Instruct](https://huggingface.co/Johnnyfans/TFRank-SFT-Qwen2.5-7B-Instruct) | Built on Qwen2.5-7B-Instruct with full SFT data. |
|
| 45 |
+
| [TFRank-SFT-Qwen3-8B](https://huggingface.co/Johnnyfans/TFRank-SFT-Qwen3-8B) | Built on Qwen3 8B with full SFT data.|
|
| 46 |
+
| [TFRank-SFT-GRPO-Qwen3-8B](https://huggingface.co/Johnnyfans/TFRank-SFT-GRPO-Qwen3-8B) | Built on Qwen3 8B with full SFT plus a small set of GRPO data.|
|
| 47 |
+
| [TFRank-GRPO-Qwen3-0.6B](https://huggingface.co/Johnnyfans/TFRank-GRPO-Qwen3-0.6B) | Built on Qwen3 0.6B and trained on full GRPO data. Efficiency oriented (high throughput, low latency). Suited for large scale online reranking and high concurrency. |
|
| 48 |
+
| [TFRank-GRPO-Qwen3-1.7B](https://huggingface.co/Johnnyfans/TFRank-GRPO-Qwen3-1.7B) | Built on Qwen3 1.7B and trained on full GRPO data. |
|
| 49 |
+
| [TFRank-GRPO-Qwen3-4B](https://huggingface.co/Johnnyfans/TFRank-GRPO-Qwen3-4B) | Built on Qwen3 4B and trained on full GRPO data. |
|
| 50 |
+
| [TFRank-GRPO-Qwen3-8B](https://huggingface.co/Johnnyfans/TFRank-GRPO-Qwen3-8B) | Built on Qwen3 8B and trained on full GRPO data. |
|
| 51 |
+
|
| 52 |
+
Additional, stronger models will be released progressively. *To be released soon...*
|
| 53 |
+
|
| 54 |
+
### 📂 Datasets
|
| 55 |
+
|
| 56 |
+
We provide high-quality datasets constructed from multiple sources, integrating **multi-task supervision**, **reasoning chains (CoT)**, and **think-mode-swift** training samples.
|
| 57 |
+
|
| 58 |
+
| Dataset | Description |
|
| 59 |
+
| :------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| 60 |
+
| [TFRank-sft-training-data](https://huggingface.co/datasets/Johnnyfans/TFRank-sft-training-data) | Supervised fine-tuning (SFT) samples aggregated from [Rank1](https://github.com/orionw/rank1), MS MARCO, and DeepSeek-R1. Includes multi-task, CoT, and think-mode-swift samples. |
|
| 61 |
+
| [TFRank-grpo-training-data](https://huggingface.co/datasets/Johnnyfans/TFRank-grpo-training-data) | GRPO-based training samples from the same sources. |
|
| 62 |
+
|
| 63 |
+
---
|
| 64 |
+
|
| 65 |
+
## Performance
|
| 66 |
+
<div align="center">
|
| 67 |
+
<img src="https://github.com/JOHNNY-fans/TFRank/raw/main/figure/size_efficiency_performance.png" width="720px">
|
| 68 |
+
<p><em>
|
| 69 |
+
Size and efficiency trade-offs for ranking performance on the BRIGHT benchmark.
|
| 70 |
+
(a) NDCG@10 versus model size for different ranker families;
|
| 71 |
+
(b) NDCG@10 versus processed queries per hour (efficiency).
|
| 72 |
+
All TFRank models are trained on the Qwen3 series.
|
| 73 |
+
</em></p>
|
| 74 |
+
</div>
|
| 75 |
+
|
| 76 |
+
---
|
| 77 |
+
|
| 78 |
+
## 🚀 Inference Quick Start
|
| 79 |
+
|
| 80 |
+
Below are two minimal examples demonstrating how to run TFRank for query–document relevance scoring.
|
| 81 |
+
|
| 82 |
+
### 1️⃣ Start a vLLM Server
|
| 83 |
+
|
| 84 |
+
```bash
|
| 85 |
+
python -m vllm.entrypoints.openai.api_server \
|
| 86 |
+
--model /path/to/your/tfrank_checkpoint # such as Johnnyfans/TFRank-GRPO-Qwen3-0.6B \
|
| 87 |
+
--served-model-name rele_pointwise \
|
| 88 |
+
--port 8113
|
| 89 |
+
```
|
| 90 |
+
|
| 91 |
+
---
|
| 92 |
+
|
| 93 |
+
### 2️⃣ Initialize the Ranker
|
| 94 |
+
|
| 95 |
+
```python
|
| 96 |
+
from evaluation.minimal_ranker import TFRankDemoRanker
|
| 97 |
+
|
| 98 |
+
ranker = TFRankDemoRanker(
|
| 99 |
+
model_name="/path/to/your/tfrank_checkpoint",
|
| 100 |
+
api_base="http://localhost:8113/v1",
|
| 101 |
+
api_key="any-string", # vLLM usually ignores this
|
| 102 |
+
think_mode=False, # set True to enable /think reasoning
|
| 103 |
+
reasoning_model=False, # set True if using a reasoning-head model
|
| 104 |
+
)
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
---
|
| 108 |
+
|
| 109 |
+
### 📝 Example 1 — Highly Relevant Document
|
| 110 |
+
|
| 111 |
+
```python
|
| 112 |
+
query = "what nano means"
|
| 113 |
+
|
| 114 |
+
document = "What does nano mean? Nano means very, very small. When it comes to making your body work, nano-materials are very important. A nanometre is one millionth of a millimetre. Your fingernail is about one millimetre thick. There are a lot of nano-materials making up your finger nail! Nanotechnology scientists move atoms and molecules around to make amazing new technologies. Nanotechnology is already in products like sunscreen."
|
| 115 |
+
|
| 116 |
+
final_score, fg_score, yes_score, response = ranker.score(query, document)
|
| 117 |
+
|
| 118 |
+
print("Final relevance score (0–1):", final_score) # 0.9997
|
| 119 |
+
print("Fine-grained score (normalized):", fg_score)
|
| 120 |
+
print("Yes-probability:", yes_score)
|
| 121 |
+
print("
|
| 122 |
+
Model response:
|
| 123 |
+
", response) # yes(4)
|
| 124 |
+
```
|
| 125 |
+
|
| 126 |
+
---
|
| 127 |
+
|
| 128 |
+
### 📝 Example 2 — Irrelevant Document
|
| 129 |
+
|
| 130 |
+
```python
|
| 131 |
+
query = "what is a musket?"
|
| 132 |
+
|
| 133 |
+
document = "8 Unusual Civil War Weapons You might think the Civil War was only fought with muskets, bayonets and cannons, but those weren’t the only deadly weapons to haunt the battlefields of the 1860s."
|
| 134 |
+
|
| 135 |
+
final_score, fg_score, yes_score, response = ranker.score(query, document)
|
| 136 |
+
|
| 137 |
+
print("Final relevance score (0–1):", final_score) # 0.1228
|
| 138 |
+
print("Fine-grained score (normalized):", fg_score)
|
| 139 |
+
print("Yes-probability:", yes_score)
|
| 140 |
+
print("
|
| 141 |
+
Model response:
|
| 142 |
+
", response) # no(1)
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
---
|
| 146 |
+
|
| 147 |
+
## 📓 Full Notebook Demo
|
| 148 |
+
|
| 149 |
+
A full inference notebook is available at:
|
| 150 |
+
|
| 151 |
+
```
|
| 152 |
+
evaluation/inference_demo.ipynb
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
---
|
| 156 |
+
|
| 157 |
+
# 📚 Citation
|
| 158 |
+
|
| 159 |
+
If you use TFRank in your research, please cite:
|
| 160 |
+
|
| 161 |
+
```bibtex
|
| 162 |
+
@article{fan2025tfrank,
|
| 163 |
+
title={TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking},
|
| 164 |
+
author={Fan, Yongqi and Chen, Xiaoyang and Ye, Dezhi and Liu, Jie and Liang, Haijin and Ma, Jin and He, Ben and Sun, Yingfei and Ruan, Tong},
|
| 165 |
+
journal={arXiv preprint arXiv:2508.09539},
|
| 166 |
+
year={2025}
|
| 167 |
+
}
|
| 168 |
+
```
|