README: document RelBench leaderboard checkpoints

#3
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -64,4 +64,23 @@ The architecture and training loop build on the [Relational Transformer](https:/
64
  huggingface-cli download kvignesh1420/relational-transformer-plurel \
65
  --repo-type model \
66
  --local-dir ~/scratch/rt_hf_ckpts
67
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
64
  huggingface-cli download kvignesh1420/relational-transformer-plurel \
65
  --repo-type model \
66
  --local-dir ~/scratch/rt_hf_ckpts
67
+ ```
68
+
69
+
70
+ ## RelBench leaderboard checkpoints (added 2026-06)
71
+
72
+ These files back the PluRel numbers on the [RelBench leaderboard](https://relbench.stanford.edu/leaderboard/).
73
+ Protocols follow the repo's continued-pretraining script (50k steps, batch 128,
74
+ lr 5e-4 cosine, from `synthetic-pretrain_rdb_1024_size_4b.pt`) and the RT
75
+ `example_finetune` protocol (lr 1e-4, batch 32, 2^15+1 steps), with regression
76
+ best-checkpoint selection by val NMAE (MAE / train-split std, ddof=1) — the
77
+ leaderboard metric — instead of R². Evaluation = full official test split.
78
+
79
+ - `cntd-pretrain_<db>_<task>.pt` — synthetic+real continued pretraining, one
80
+ leave-one-DB-out run per database (incl. rel-event), per-task best checkpoint.
81
+ These produce the "PluRel | synthetic + real" zero-shot regression and
82
+ rel-event cells.
83
+ - `leaderboard-finetuned/finetune_<db>_<task>.pt` — fine-tuned from the matching
84
+ `cntd-pretrain` checkpoint (chosen over synthetic-only by val zero-shot, which
85
+ the synthetic+real checkpoint won on every task). These produce the
86
+ "PluRel | pretrained + fine-tuned" leaderboard row.