README.md

metadata

title: LEADBOARD - ADMET, Kinase and Toxicity Prediction Benchmark
emoji: 🎯
colorFrom: blue
colorTo: gray
sdk: docker
app_port: 7860
hf_oauth: true
pinned: false
short_description: Benchmark for drug prediction tools - ADMET, kinase, tox
tags:
  - drug-discovery
  - admet
  - leaderboard
  - benchmark
  - cheminformatics
  - molecular-property-prediction
  - qsar
  - toxicity
  - herg
  - solubility
  - kinase
  - chembl
  - cell-painting
  - preclinical

LEADBOARD

A benchmark for drug property prediction tools. One yardstick, one discipline at a time.

Submit predictions for a held-out set of molecules. We score them against labels we never release, and place your tool on the board for that discipline.

👉 Open the leaderboard

---

What is measured

Most published benchmarks split their molecules at random. That is the wrong question. A tool is used to rank compounds nobody has measured yet, so the honest test is: train on what was known by a cut-off year, predict what came after.

We ran both splits on identical data with an identical model. On the hERG cardiotoxicity board, AUROC was 0.818 under a random split and 0.606 under a time split. Same molecules, same code — only the question changed. Every board here uses the harder one.

What we publish before you submit

For each board, before any entry arrives:

Published	Why it matters
Untrained baselines	Constant prediction, nearest neighbour, Morgan+LightGBM. If a tool cannot beat these, the board says so.
Experimental noise floor	How far apart two labs land when they measure the same compound. Nothing can be measured below it.
Split grade and answer grade	Exactly how the test set was cut, and whether the labels can be looked up anywhere.
Data and scorer fingerprints	The SHA of the exact test file and scoring code behind every score.

A score without its noise floor is a number without a unit. Both are shown.

Grading notation

Each board carries two letters, for example [T/P2].

Split grade — what the board asks of a tool

• T time split by first-report year
• S scaffold split by Murcko core
• R random split (we do not open these)

Answer grade — whether an entrant can look the answer up

• P1 public source, our curation
• P2 public source, our unit conversion and selection define this revision
• P3 labels held privately
• P4 prospective — the answer does not exist yet

Boards

21 boards across 7 disciplines, 18,382 held-out compounds.

• 🫗 Absorption — Solubility
• 🔥 Metabolism — CYP3A4, CYP2D6, CYP2C9
• ☠️ Toxicity — hERG
• 🎯 Potency — AChE, MAOB, COX2
• 🔑 Kinase — EGFR, JAK2, PI3Kα, FLT3, VEGFR2, CDK2, HER2, ABL1, BRAF, KIT, ALK
• 🔬 Cell / Phenotype — JUMP Cell Painting morphology, 115,689 compounds, 16 profile axes
• 🏥 Clinical — Post-marketing withdrawal, year-matched controls

The withdrawal board is worth a note. Withdrawal rate tracks approval decade (7.8% in the 1990s, 1.2% in the 2010s), so a predictor that reads nothing but the approval year scores AUROC 0.636. After year-matching the controls it scores 0.504 — chance. That is the version we opened.

How to enter

1. Pick a board and download its test set — structures only, no labels.
2. Predict with your own tool. Anything goes: a trained model, a physics engine, a language model, a rule of thumb.
3. Upload a CSV of compound_id,prediction. Sign in with your Hugging Face account.
4. Scoring runs off-platform on hardware that holds the labels. The Space never sees them.

The Space carries a filled-in prompt and a runnable skeleton for each board, so a first entry does not require building anything from scratch.

Repeated submissions

New scores are revealed under the ladder rule (Blum & Hardt, ICML 2015): a fresh score replaces your best only when it beats it by more than the noise floor. Otherwise your previous best stands. This is what stops a leaderboard from being won by whoever submits the most times.

Data and licences

• ChEMBL 37 (EMBL-EBI) — CC BY-SA 3.0. Re-curated; attribution travels with every board card.
• JUMP Cell Painting Consortium — CC0.
• Withdrawal board assembled from public regulatory records.

Non-commercial research benchmark. Test sets carry structures only; labels are never distributed.

---

한국어 요약

신약 예측 도구를, 분야별로, 같은 잣대로 잽니다.

정답은 배포하지 않습니다. 대신 채점자가 먼저 공개합니다 — 아무것도 학습하지 않은 기준선, 그 분야의 실험 잡음 바닥, 점수마다의 데이터·채점기 지문.

무작위 분할은 틀린 질문입니다. 도구는 아직 아무도 재지 않은 화합물의 순위를 매기는 데 쓰이니, 정직한 시험은 어느 해까지 알려진 것으로 배우고 그 뒤에 나온 것을 맞히는 것입니다. 같은 자료·같은 모델로 재 보면 hERG 심장독성 AUROC 가 무작위 분할 0.818, 시간 분할 0.606 이었습니다. 여기 부문은 전부 어려운 쪽을 씁니다.

7개 분야 21부문 · 테스트 화합물 18,382. 화면은 한국어·영어를 자동으로 가릅니다.

---

규격 v1.1 (8개 조항) · FINAL-Bench / VIDRAFT