nielsr HF Staff commited on
Commit
feede0e
·
verified ·
1 Parent(s): 4801987

Add model card for Relational Transformer

Browse files

This PR adds a comprehensive model card for the Relational Transformer. It includes metadata, links to the ICLR 2026 paper and official GitHub repository, installation instructions, and details on how to use the provided checkpoints.

Files changed (1) hide show
  1. README.md +61 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: other
3
+ ---
4
+
5
+ # Relational Transformer
6
+
7
+ This repository contains the official checkpoints for the **Relational Transformer (RT)**, introduced in the paper [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://arxiv.org/abs/2510.06377).
8
+
9
+ Relational Transformer is a foundation model architecture designed to be pretrained on diverse relational databases and applied to unseen datasets and tasks without task- or dataset-specific fine-tuning. It utilizes a novel Relational Attention mechanism over columns, rows, and primary-foreign key links.
10
+
11
+ - **Paper:** [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://arxiv.org/abs/2510.06377)
12
+ - **GitHub Repository:** [snap-stanford/relational-transformer](https://github.com/snap-stanford/relational-transformer)
13
+
14
+ ## Installation
15
+
16
+ The repository uses [pixi](https://pixi.sh/latest/#installation) for package management.
17
+
18
+ ```bash
19
+ git clone https://github.com/snap-stanford/relational-transformer
20
+ cd relational-transformer
21
+ pixi install
22
+ # compile and install the rust sampler
23
+ cd rustler
24
+ pixi run maturin develop --uv --release
25
+ ```
26
+
27
+ ## Checkpoints
28
+
29
+ The project provides two types of checkpoints:
30
+ - `pretrain_<dataset>_<task>.pt`: Pretrained with the specified `<dataset>` held out.
31
+ - `contd-pretrain_<dataset>_<task>.pt`: Obtained by continued pretraining on `<dataset>` with the specific `<task>` held out.
32
+
33
+ You can download specific checkpoints using the Hugging Face CLI:
34
+
35
+ ```bash
36
+ mkdir -p ~/scratch/rt_ckpts
37
+ huggingface-cli download rishabh-ranjan/relational-transformer \
38
+ --repo-type model \
39
+ --include "pretrain_rel-amazon_user-churn.pt" \
40
+ --local-dir ~/scratch/rt_ckpts \
41
+ --local-dir-use-symlinks False
42
+ ```
43
+
44
+ ## Usage
45
+
46
+ To use these checkpoints, pass the path to the `load_ckpt_path` argument in the training scripts provided in the GitHub repository. For example, to run a finetuning experiment:
47
+
48
+ ```bash
49
+ pixi run torchrun --standalone --nproc_per_node=8 scripts/example_finetune.py
50
+ ```
51
+
52
+ ## Citation
53
+
54
+ ```bibtex
55
+ @inproceedings{ranjan2025relationaltransformer,
56
+ title={{Relational Transformer:} Toward Zero-Shot Foundation Models for Relational Data},
57
+ author={Rishabh Ranjan and Valter Hudovernik and Mark Znidar and Charilaos Kanatsoulis and Roshan Upendra and Mahmoud Mohammadi and Joe Meyer and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
58
+ booktitle={The Fourteenth International Conference on Learning Representations},
59
+ year={2026}
60
+ }
61
+ ```