README.md

---
title: RPC-Bench Leaderboard
emoji: 📄
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: 5.0.0
python_version: 3.11
app_file: app.py
pinned: false
license: apache-2.0
---

# RPC-Bench Leaderboard Space

<p align="center">
  🌐 <a href="https://rpc-bench.github.io/" target="_blank">Project Page</a> ·
  📖 <a href="https://arxiv.org/abs/2601.14289" target="_blank">Paper</a> ·
  🤗 <a href="https://huggingface.co/datasets/zai-org/RPC-Bench" target="_blank">Hugging Face</a> ·
  🧭 <a href="https://modelscope.cn/datasets/ZhipuAI/RPC-Bench" target="_blank">ModelScope</a>
</p>


## How to Submit

Fork the RPC-Bench dataset repository, create a branch, add one submission folder, and open a Pull Request:

```text
submissions/
  <organization>__<model>__<input_config>/
    metadata.yaml
    predictions.jsonl
    generation_config.json      # optional
    artifacts/                  # optional logs, prompts, run notes
```

Use URL-safe directory names. Replace spaces, slashes, and special characters with hyphens; keep `input_config` as `TEXT` or `VISUAL`.

## `metadata.yaml`

```yaml
model_name: "My Model"
organization: "My Org"
model_url: https://...        # optional work link: paper, GitHub, model card, etc.
date: "2026-06-17"           # model release date, not submission date
split: test
input_config: TEXT            # TEXT or VISUAL
```

## `predictions.jsonl`

Each line must be one JSON object:

```json
{
  "id": "paper-id",
  "part_idx": 1,
  "question": "question text",
  "category": "category",
  "gen_answer": "model answer"
}
```

`part_idx` is the question index in the current paper's `qa_pairs` list (`1` for the first item). `category` must match the corresponding item in `test.json`.