RelBench leaderboard checkpoints: synth+real continued-pretraining (7 leave-one-DB runs) + 21 fine-tuned ckpts (NMAE selection); see LEADERBOARD.md

LEADERBOARD.md

@@ -0,0 +1,21 @@
+# Checkpoints for the RelBench leaderboard numbers
+
+Produced by the RelBench team (2026-06) to make every PluRel number on
+https://relbench.stanford.edu/leaderboard/ reproducible. Protocols follow the repo's
+continued-pretraining script (50k steps, batch 128, lr 5e-4 cosine, from
+`synthetic-pretrain_rdb_1024_size_4b.pt`) and the RT `example_finetune` protocol
+(lr 1e-4, batch 32, 2^15+1 steps), with regression best-checkpoint selection by val
+NMAE (MAE / train-split std, ddof=1) — the leaderboard metric — instead of R².
+
+## New files
+
+- `cntd-pretrain_<db>_<task>.pt` — synthetic+real continued pretraining, one
+  leave-one-DB-out run per database (incl. rel-event), per-task best checkpoint.
+  These produce the "PluRel | synthetic + real" zero-shot regression and rel-event
+  leaderboard cells.
+- `leaderboard-finetuned/finetune_<db>_<task>.pt` — fine-tuned from the matching
+  `cntd-pretrain` checkpoint (chosen over synthetic-only by val zero-shot, which the
+  synthetic+real checkpoint won on every task). These produce the
+  "PluRel | pretrained + fine-tuned" leaderboard row.
+
+Evaluation = full official test split; AUROC for classification, NMAE for regression.

cntd-pretrain_rel-amazon_item-churn.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68d5a4819c62cc2cdce4e371383cf8f0501dc881af874ee6b7dd9cc95078b747
+size 38508537

cntd-pretrain_rel-amazon_item-ltv.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5983f64a2ecc98ae59d5a72d5ab837c48f3f3db8ea62e5478d8b7c8ea0ea067f
+size 38507869

cntd-pretrain_rel-amazon_user-churn.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:426ff3a62dae99c6069746f0bde1e1aeecc1155833df00a38fdba5c13c067464
+size 38508537

cntd-pretrain_rel-amazon_user-ltv.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5842ec317697fac9e35aec54f84a62aa8e734866d1837c7decd25c8fdd141ca7
+size 38507869

cntd-pretrain_rel-avito_ad-ctr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c74a12a38a4322feabd30a08ffa31821848733b8f798263cd64a68370e93bb77
+size 38506803

cntd-pretrain_rel-avito_user-clicks.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39c4b465dc788a59ae8a60e19e48779123bd68e42098e01f87007c12bc68ee43
+size 38508537

cntd-pretrain_rel-avito_user-visits.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c5323de187f61490974587d9d27aa6dd6185bd61b368f0f044d7708f87efef9
+size 38508537

cntd-pretrain_rel-event_user-attendance.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e879234e9f1c55ebd9ecf485f9ca3eeddf1c24eb6e449d410ad30593a2b06d0
+size 38509873

cntd-pretrain_rel-event_user-ignore.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87c8a754cc901ff83fe50109ac02772035b8e220591d137198c03edb752af2de
+size 38508537

cntd-pretrain_rel-event_user-repeat.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97590e07b0d86cf02d4e6ce9b929d80f9846a112416fd53a575caaa8f8547c87
+size 38508537

cntd-pretrain_rel-f1_driver-dnf.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b2bbfaa036748990b59518408c02ade12d84b54b30f355fdc0c37f5917df8e7
+size 38507201

cntd-pretrain_rel-f1_driver-position.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00efb5143fcaf9f6bc8e1ac6278a7941edf9393b8d459aac3b7feed4b883e29d
+size 38508871

cntd-pretrain_rel-f1_driver-top3.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6aa09238e2bf24ea0a16bf2f72bbe7d85462ca50959775ba1cfeb00bbb3b3d31
+size 38507535

cntd-pretrain_rel-hm_item-sales.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66600ac8f302309956bd0f0b5291ebf1963d77744e785978773696cd74dd6551
+size 38507201

cntd-pretrain_rel-hm_user-churn.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:461933a516fb411ed0dc0f0e10df1716ba94ae186761c9b954a0adb297df2aa5
+size 38507201

cntd-pretrain_rel-stack_post-votes.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e36dc889bde38e116bbf7fef774a7a02a5df463db6dda2fadec8d63108eb5a7
+size 38508203

cntd-pretrain_rel-stack_user-badge.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d29c5f58220f430066183debe95067cd7a075dea9af249a236b0dc87f4328082
+size 38508203

cntd-pretrain_rel-stack_user-engagement.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b017e5657d1b2729733b7802b74f20dbe01e27418ed5dbc4982c7cb77d1af50
+size 38509873

cntd-pretrain_rel-trial_site-success.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bb7163dfcc1dddd43d24adbe315abd75961b25b523395752d980abd83143a7a
+size 38508871

cntd-pretrain_rel-trial_study-adverse.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:565fdbce2cce6da6fc10f8adc8f54e3f326991298200eed1b575b26ee98d6ae0
+size 38509205

cntd-pretrain_rel-trial_study-outcome.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6309238cf510012dfeda9aa36fb2affdec5d66588f894bb0cc97ffb27c259e98
+size 38509205

leaderboard-finetuned/finetune_rel-amazon_item-churn.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2018ff196429a6f68c1a5fbb4d81c88cb9f09c729c26e5145b5654e0d835fd4
+size 38508537

leaderboard-finetuned/finetune_rel-amazon_item-ltv.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20612762172c140f2d5f632874c88346d99f92cf28266dc977fb877d7796dd27
+size 38507869

leaderboard-finetuned/finetune_rel-amazon_user-churn.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45f7c3c4c4b2301d6d43d54d37d9be44b6ef6486946543627aa4eacafb32877c
+size 38508537

leaderboard-finetuned/finetune_rel-amazon_user-ltv.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa9ea454860f614c4178b07489e35509ba9635338f013a6c64c75b7c7b26aa50
+size 38507869

leaderboard-finetuned/finetune_rel-avito_ad-ctr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6e5fd929db2c0f6425446d7383398830c3c46611e3bd88a19ec79f73e94e910
+size 38506803

leaderboard-finetuned/finetune_rel-avito_user-clicks.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1c20c2f197f4761a475e501415a545653fad0e2433346bb2aeb8f7f45555eac
+size 38508537

leaderboard-finetuned/finetune_rel-avito_user-visits.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfdd5704b7b50739e3cc8253def5ae9ea41478378542e6fe32b5754184dce243
+size 38508537

leaderboard-finetuned/finetune_rel-event_user-attendance.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee0474971761be203d9e985c22d835ddded9ffb80adb6ff44815c80d454d5900
+size 38509873

leaderboard-finetuned/finetune_rel-event_user-ignore.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b92b932daca337d2a90415e047d3d750dd562cd4dad41a8f2247cfc64fa33c9a
+size 38508537

leaderboard-finetuned/finetune_rel-event_user-repeat.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88ec8e222927e5a7e8bb246d7e41ecfb74f62f6ce050bc61eddb014caada1322
+size 38508537

leaderboard-finetuned/finetune_rel-f1_driver-dnf.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:feb846cdd957fb79e69f21c5cea2533a2eb35a808589575a17f38ec3653cee53
+size 38507201

leaderboard-finetuned/finetune_rel-f1_driver-position.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e20732ff0c4228d820cd325c6b8728f89a7f503103bcae8bd9ef78bc869d678
+size 38508871

leaderboard-finetuned/finetune_rel-f1_driver-top3.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2cf1a94c09be22270c6587a6e3c90475dda45c0d897903f81a6ce6829ae4c76
+size 38507535

leaderboard-finetuned/finetune_rel-hm_item-sales.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b18134925ce9ffa1354c59c5cfc55dd9208319913679158f413fda6596f9c8fb
+size 38507201

leaderboard-finetuned/finetune_rel-hm_user-churn.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56c8453ba2f6b03d9ea716054bef23d5936ea326ebd5cda85d6bd764dc2192ea
+size 38507201

leaderboard-finetuned/finetune_rel-stack_post-votes.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0bf66118fd2fd3c5797ada6ce62bda7095ed65d5cc4168ed70ff5f3ac7e7c8b
+size 38508203

leaderboard-finetuned/finetune_rel-stack_user-badge.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd4a02cef7f2431052d630d3c0aeded21359781b7ac963a1823100766313cc23
+size 38508203

leaderboard-finetuned/finetune_rel-stack_user-engagement.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ecb476643a8127a5647879748ac77a9065cc47e4764f78c94b8a803b90e452b
+size 38509873

leaderboard-finetuned/finetune_rel-trial_site-success.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed9b2ae52889a35984435cd50af5dbfe705ea3fe64c14e992c73ef6d39ee9d37
+size 38508871

leaderboard-finetuned/finetune_rel-trial_study-adverse.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7a8068337111c9a5afdda9160cef2edb5b8ed4047185caf74ec10486178a4e4
+size 38509205

leaderboard-finetuned/finetune_rel-trial_study-outcome.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b93a71ea55683a83dd3a76c57c03c685d2be38008007860d5e590f090956d67f
+size 38509205