File size: 5,250 Bytes
3b28c67 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 | ---
license: apache-2.0
base_model: Qwen/Qwen3-8B
library_name: transformers
pipeline_tag: text-generation
tags:
- table-ranking
- listwise-reranking
- table-retrieval
- table-question-answering
- qwen3
language:
- en
---
# TabRankMultiTableNaive
A single call generative listwise reranker for table retrieval built on Qwen3 8B.
Given a question and a set of candidate tables the model returns the tables ordered
from most to least useful for answering the question. It reads all candidates in one
prompt and emits the full ranking in a single generation.
This checkpoint is the **Single plus Multi Table Answer Only** variant.
## What this variant does
The model is trained to output the ranking directly. Reasoning traces were shown during training but masked from the loss so the model learns to rank without writing a chain of thought. This makes it the fastest variant at inference.
## Training data
This model is trained on a mix of **NQ Tables** and **MultiTabQA** which adds multi table retrieval questions. The sibling checkpoint [TabRankSingleTableNaive](https://huggingface.co/AdarshSingh7647/TabRankSingleTableNaive)
uses the other training mix with the same objective.
## Input and output format
The user message lists candidate tables as blocks headed by `### Table 1` `### Table 2`
and so on. The model returns a JSON object whose value is the ranked list of one based
candidate positions best first:
```json
{"ranked_tables": [3, 1, 5, 2, 4]}
```
Map those positions back to your table ids to obtain the reranked list.
## Evaluation
Scored as a listwise reranker that reorders a first stage top 25 candidate list on four
table question answering benchmarks. SQA and TAT QA use the full test split. HybridQA and
TabFact use a fixed shared 500 query sample. `acc@10` counts a query correct only when every
gold table falls inside the top 10.
| Dataset | n | recall@10 | ndcg@10 | acc@10 | MRR |
|---|---|---|---|---|---|
| SQA | 148 | 0.878 | 0.727 | 0.818 | 0.689 |
| TAT QA | 362 | 0.678 | 0.544 | 0.467 | 0.610 |
| HybridQA | 500 | 0.901 | 0.788 | 0.810 | 0.818 |
| TabFact | 500 | 0.801 | 0.719 | 0.624 | 0.807 |
| **Mean** | | **0.815** | **0.695** | **0.680** | **0.731** |
## Usage with vLLM
```python
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer
repo = "AdarshSingh7647/TabRankMultiTableNaive"
tok = AutoTokenizer.from_pretrained(repo)
llm = LLM(model=repo, dtype="bfloat16", max_model_len=32768)
system = ("You are a table relevance expert. Given a question and a set of candidate tables "
"rank them from most to least useful for answering the question. Reason in a "
"<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"
msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
# the model outputs the ranking json directly
out = llm.generate([text], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=8192))
print(out[0].outputs[0].text)
```
## Usage with Transformers
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "AdarshSingh7647/TabRankMultiTableNaive"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")
system = ("You are a table relevance expert. Given a question and a set of candidate tables "
"rank them from most to least useful for answering the question. Reason in a "
"<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"
msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```
## Model details
* Base model Qwen3 8B
* Method LoRA rank 16 fine tuning merged into the base weights so it loads directly
* Precision bfloat16 single file safetensors near 16 GB
* Family the six TabRank checkpoints span three objectives (Answer Only, Reasoning Generation,
Reasoning Conditioned) across two training mixes (Single Table, Single plus Multi Table)
## Citation
The MultiTabQA data comes from RAG over Tables. Please cite it when using the Single plus
Multi Table checkpoints:
```bibtex
@misc{zou2025ragtableshierarchicalmemory,
title={RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking},
author={Jiaru Zou and Dongqi Fu and Sirui Chen and Xinrui He and Zihao Li and Yada Zhu and Jiawei Han and Jingrui He},
year={2025},
eprint={2504.01346},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2504.01346}
}
```
The accompanying TabRank paper is currently under review. A citation will be added here once it
is available on arXiv.
|