README.md

---
title: LEADBOARD - ADMET, Kinase and Toxicity Prediction Benchmark
emoji: 🎯
colorFrom: blue
colorTo: gray
sdk: docker
app_port: 7860
# hf_oauth 를 켜야 플랫폼이 OAUTH_CLIENT_ID/SECRET 를 컨테이너에 넣어준다.
hf_oauth: true
pinned: false
short_description: Benchmark for drug prediction tools - ADMET, kinase, tox
tags:
  - drug-discovery
  - admet
  - leaderboard
  - benchmark
  - cheminformatics
  - molecular-property-prediction
  - qsar
  - toxicity
  - herg
  - solubility
  - kinase
  - chembl
  - cell-painting
  - preclinical
---

# LEADBOARD

**A benchmark for drug property prediction tools.** One yardstick, one discipline at a time.

Submit predictions for a held-out set of molecules. We score them against labels
we never release, and place your tool on the board for that discipline.

👉 **[Open the leaderboard](https://huggingface.co/spaces/FINAL-Bench/leadboard)**

---

## What is measured

Most published benchmarks split their molecules at random. That is the wrong
question. A tool is used to rank compounds nobody has measured yet, so the honest
test is: *train on what was known by a cut-off year, predict what came after.*

We ran both splits on identical data with an identical model. On the hERG
cardiotoxicity board, AUROC was **0.818** under a random split and **0.606** under
a time split. Same molecules, same code — only the question changed. Every board
here uses the harder one.

## What we publish before you submit

For each board, before any entry arrives:

| Published | Why it matters |
|---|---|
| **Untrained baselines** | Constant prediction, nearest neighbour, Morgan+LightGBM. If a tool cannot beat these, the board says so. |
| **Experimental noise floor** | How far apart two labs land when they measure the same compound. Nothing can be measured below it. |
| **Split grade and answer grade** | Exactly how the test set was cut, and whether the labels can be looked up anywhere. |
| **Data and scorer fingerprints** | The SHA of the exact test file and scoring code behind every score. |

A score without its noise floor is a number without a unit. Both are shown.

## Grading notation

Each board carries two letters, for example `[T/P2]`.

**Split grade** — what the board asks of a tool
- `T` time split by first-report year
- `S` scaffold split by Murcko core
- `R` random split (we do not open these)

**Answer grade** — whether an entrant can look the answer up
- `P1` public source, our curation
- `P2` public source, our unit conversion and selection define this revision
- `P3` labels held privately
- `P4` prospective — the answer does not exist yet

## Boards

21 boards across 7 disciplines, 18,382 held-out compounds.

- 🫗 **Absorption** — Solubility
- 🔥 **Metabolism** — CYP3A4, CYP2D6, CYP2C9
- ☠️ **Toxicity** — hERG
- 🎯 **Potency** — AChE, MAOB, COX2
- 🔑 **Kinase** — EGFR, JAK2, PI3Kα, FLT3, VEGFR2, CDK2, HER2, ABL1, BRAF, KIT, ALK
- 🔬 **Cell / Phenotype** — JUMP Cell Painting morphology, 115,689 compounds, 16 profile axes
- 🏥 **Clinical** — Post-marketing withdrawal, year-matched controls

The withdrawal board is worth a note. Withdrawal rate tracks approval decade
(7.8% in the 1990s, 1.2% in the 2010s), so a predictor that reads nothing but the
approval year scores AUROC 0.636. After year-matching the controls it scores 0.504
— chance. That is the version we opened.

## How to enter

1. Pick a board and download its test set — structures only, no labels.
2. Predict with your own tool. Anything goes: a trained model, a physics engine,
   a language model, a rule of thumb.
3. Upload a CSV of `compound_id,prediction`. Sign in with your Hugging Face account.
4. Scoring runs off-platform on hardware that holds the labels. The Space never
   sees them.

The Space carries a filled-in prompt and a runnable skeleton for each board, so a
first entry does not require building anything from scratch.

### Repeated submissions

New scores are revealed under the ladder rule (Blum & Hardt, ICML 2015): a fresh
score replaces your best only when it beats it by more than the noise floor.
Otherwise your previous best stands. This is what stops a leaderboard from being
won by whoever submits the most times.

## Data and licences

- **ChEMBL 37** (EMBL-EBI) — CC BY-SA 3.0. Re-curated; attribution travels with every board card.
- **JUMP Cell Painting Consortium** — CC0.
- Withdrawal board assembled from public regulatory records.

Non-commercial research benchmark. Test sets carry structures only; labels are
never distributed.

---

## 한국어 요약

**신약 예측 도구를, 분야별로, 같은 잣대로 잽니다.**

정답은 배포하지 않습니다. 대신 채점자가 먼저 공개합니다 — 아무것도 학습하지 않은
기준선, 그 분야의 실험 잡음 바닥, 점수마다의 데이터·채점기 지문.

무작위 분할은 틀린 질문입니다. 도구는 아직 아무도 재지 않은 화합물의 순위를 매기는
데 쓰이니, 정직한 시험은 **어느 해까지 알려진 것으로 배우고 그 뒤에 나온 것을
맞히는 것**입니다. 같은 자료·같은 모델로 재 보면 hERG 심장독성 AUROC 가 무작위
분할 0.818, 시간 분할 0.606 이었습니다. 여기 부문은 전부 어려운 쪽을 씁니다.

7개 분야 21부문 · 테스트 화합물 18,382. 화면은 한국어·영어를 자동으로 가릅니다.

---

규격 v1.1 (8개 조항) · **FINAL-Bench / VIDRAFT**