Add model card for Relational Transformer
Browse filesThis PR adds a comprehensive model card for the Relational Transformer. It includes metadata, links to the ICLR 2026 paper and official GitHub repository, installation instructions, and details on how to use the provided checkpoints.
README.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: other
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# Relational Transformer
|
| 6 |
+
|
| 7 |
+
This repository contains the official checkpoints for the **Relational Transformer (RT)**, introduced in the paper [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://arxiv.org/abs/2510.06377).
|
| 8 |
+
|
| 9 |
+
Relational Transformer is a foundation model architecture designed to be pretrained on diverse relational databases and applied to unseen datasets and tasks without task- or dataset-specific fine-tuning. It utilizes a novel Relational Attention mechanism over columns, rows, and primary-foreign key links.
|
| 10 |
+
|
| 11 |
+
- **Paper:** [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://arxiv.org/abs/2510.06377)
|
| 12 |
+
- **GitHub Repository:** [snap-stanford/relational-transformer](https://github.com/snap-stanford/relational-transformer)
|
| 13 |
+
|
| 14 |
+
## Installation
|
| 15 |
+
|
| 16 |
+
The repository uses [pixi](https://pixi.sh/latest/#installation) for package management.
|
| 17 |
+
|
| 18 |
+
```bash
|
| 19 |
+
git clone https://github.com/snap-stanford/relational-transformer
|
| 20 |
+
cd relational-transformer
|
| 21 |
+
pixi install
|
| 22 |
+
# compile and install the rust sampler
|
| 23 |
+
cd rustler
|
| 24 |
+
pixi run maturin develop --uv --release
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
## Checkpoints
|
| 28 |
+
|
| 29 |
+
The project provides two types of checkpoints:
|
| 30 |
+
- `pretrain_<dataset>_<task>.pt`: Pretrained with the specified `<dataset>` held out.
|
| 31 |
+
- `contd-pretrain_<dataset>_<task>.pt`: Obtained by continued pretraining on `<dataset>` with the specific `<task>` held out.
|
| 32 |
+
|
| 33 |
+
You can download specific checkpoints using the Hugging Face CLI:
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
mkdir -p ~/scratch/rt_ckpts
|
| 37 |
+
huggingface-cli download rishabh-ranjan/relational-transformer \
|
| 38 |
+
--repo-type model \
|
| 39 |
+
--include "pretrain_rel-amazon_user-churn.pt" \
|
| 40 |
+
--local-dir ~/scratch/rt_ckpts \
|
| 41 |
+
--local-dir-use-symlinks False
|
| 42 |
+
```
|
| 43 |
+
|
| 44 |
+
## Usage
|
| 45 |
+
|
| 46 |
+
To use these checkpoints, pass the path to the `load_ckpt_path` argument in the training scripts provided in the GitHub repository. For example, to run a finetuning experiment:
|
| 47 |
+
|
| 48 |
+
```bash
|
| 49 |
+
pixi run torchrun --standalone --nproc_per_node=8 scripts/example_finetune.py
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
## Citation
|
| 53 |
+
|
| 54 |
+
```bibtex
|
| 55 |
+
@inproceedings{ranjan2025relationaltransformer,
|
| 56 |
+
title={{Relational Transformer:} Toward Zero-Shot Foundation Models for Relational Data},
|
| 57 |
+
author={Rishabh Ranjan and Valter Hudovernik and Mark Znidar and Charilaos Kanatsoulis and Roshan Upendra and Mahmoud Mohammadi and Joe Meyer and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
|
| 58 |
+
booktitle={The Fourteenth International Conference on Learning Representations},
|
| 59 |
+
year={2026}
|
| 60 |
+
}
|
| 61 |
+
```
|