Add model card for Relational Transformer

README.md

@@ -0,0 +1,61 @@
+---
+pipeline_tag: other
+---
+
+# Relational Transformer
+
+This repository contains the official checkpoints for the **Relational Transformer (RT)**, introduced in the paper [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://arxiv.org/abs/2510.06377).
+
+Relational Transformer is a foundation model architecture designed to be pretrained on diverse relational databases and applied to unseen datasets and tasks without task- or dataset-specific fine-tuning. It utilizes a novel Relational Attention mechanism over columns, rows, and primary-foreign key links.
+
+- **Paper:** [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://arxiv.org/abs/2510.06377)
+- **GitHub Repository:** [snap-stanford/relational-transformer](https://github.com/snap-stanford/relational-transformer)
+
+## Installation
+
+The repository uses [pixi](https://pixi.sh/latest/#installation) for package management.
+
+```bash
+git clone https://github.com/snap-stanford/relational-transformer
+cd relational-transformer
+pixi install
+# compile and install the rust sampler
+cd rustler
+pixi run maturin develop --uv --release
+```
+
+## Checkpoints
+
+The project provides two types of checkpoints:
+- `pretrain_<dataset>_<task>.pt`: Pretrained with the specified `<dataset>` held out.
+- `contd-pretrain_<dataset>_<task>.pt`: Obtained by continued pretraining on `<dataset>` with the specific `<task>` held out.
+
+You can download specific checkpoints using the Hugging Face CLI:
+
+```bash
+mkdir -p ~/scratch/rt_ckpts
+huggingface-cli download rishabh-ranjan/relational-transformer \
+  --repo-type model \
+  --include "pretrain_rel-amazon_user-churn.pt" \
+  --local-dir ~/scratch/rt_ckpts \
+  --local-dir-use-symlinks False
+```
+
+## Usage
+
+To use these checkpoints, pass the path to the `load_ckpt_path` argument in the training scripts provided in the GitHub repository. For example, to run a finetuning experiment:
+
+```bash
+pixi run torchrun --standalone --nproc_per_node=8 scripts/example_finetune.py
+```
+
+## Citation
+
+```bibtex
+@inproceedings{ranjan2025relationaltransformer,
+    title={{Relational Transformer:} Toward Zero-Shot Foundation Models for Relational Data}, 
+    author={Rishabh Ranjan and Valter Hudovernik and Mark Znidar and Charilaos Kanatsoulis and Roshan Upendra and Mahmoud Mohammadi and Joe Meyer and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
+    booktitle={The Fourteenth International Conference on Learning Representations},
+    year={2026}
+}
+```