README.md

---
tags:
  - tabular
  - tabular-foundation-model
  - dataset-inference
  - membership-inference
  - privacy
  - auditing
  - tabpfn
  - tabdpt
  - nanotabpfn
  - sap-rpt-oss
pipeline_tag: tabular-classification
---

# TFMs: model weights and behavioural fingerprints for dataset inference on tabular foundation models

Companion artefacts for the paper

> **Dataset inference for data provenance and privacy auditing in tabular foundation models**
> Dariush Wahdany\*¹, Jesse C. Cresswell\*², Naiqing Guan², Atiyeh Ashari Ghomi², Franziska Boenisch¹, Adam Dziedzic¹
> ¹ CISPA Helmholtz Center for Information Security · ² Layer 6 AI · \* equal contribution

Code: <https://github.com/layer6ai-labs/tfm-di>

Tabular foundation models (TFMs) are trained on large collections of real tabular datasets and served through black-box APIs. The paper introduces the first **dataset inference** (DI) method for TFMs: it probes a model with input manipulations, measures how its predictions react, reduces each (manipulation, measurement) curve to a handful of scalars, and trains an auditor on the resulting per-dataset *fingerprint* to decide whether a suspect dataset was part of the model's pre-training corpus. Every number is reported against a **blind baseline**, the same pipeline run on a TFM that never saw the datasets (TabPFN-2.5 trained on synthetic data only), so distribution shift between members and non-members is not mistaken for memorisation.

This repository holds the two kinds of artefacts needed to reproduce or extend that study:

1. **Model weights** we trained ourselves: TabDPT re-trained with different seeds and on row-level member splits, and a family of NanoTabPFN models of varying training-set size, architecture size, and training data (synthetic vs. real T4).
2. **Behavioural fingerprint tables** (`signals/<model>/<pool>/datasets.parquet`): the extracted signals for every target model and every blind, so the audits in the paper can be re-run on CPU in minutes without any GPU extraction.

## Access

The repository is gated. Request access on the repository page, then authenticate once:

```bash
hf auth login
```

```python
from huggingface_hub import hf_hub_download, snapshot_download
import pandas as pd

# a weight file
ckpt = hf_hub_download("SprintML/tfms", "nanotabpfn/classifier.pt")

# a fingerprint table
df = pd.read_parquet(hf_hub_download("SprintML/tfms", "signals/tabdpt/binary_tabdpt_cls/datasets.parquet"))
df["true_label"].value_counts()   # 1 = member of the target model's pre-training corpus
```

## Model weights

### NanoTabPFN family (`nanotabpfn/`)

Small TabPFN-style in-context learners trained with [TFM-Playground](https://github.com/automl/TFM-Playground) (vendored as `tfm_playground/` in the code repository). Every `<name>.pt` has a sibling `<name>.arch.json` with the architecture and training hyper-parameters, which the loader in `di/datasets/nanotabpfn_wrapper.py` reads automatically. All models: 80 epochs, batch 50, lr 1e-4, up to 10 classes, seed 2402.

| file | layers | embed | heads | MLP | params | training tables | role in the paper |
|---|---|---|---|---|---|---|---|
| `classifier.pt` | 6 | 192 | 6 | 768 | 3.7 M | 50,000 synthetic | baseline NanoTabPFN (Tables 1, 2, 10, 11) |
| `data25k.pt` | 6 | 192 | 6 | 768 | 3.7 M | 25,000 synthetic | data-size ablation (Table 10) |
| `data10k.pt` | 6 | 192 | 6 | 768 | 3.7 M | 10,000 synthetic | data-size ablation (Table 10) |
| `data5k.pt` | 6 | 192 | 6 | 768 | 3.7 M | 5,000 synthetic | data-size ablation (Table 10) |
| `arch_medium.pt` | 8 | 384 | 8 | 1536 | 19.5 M | 50,000 synthetic | architecture ablation (Table 11) |
| `arch_large.pt` | 12 | 768 | 12 | 3072 | 115.8 M | 50,000 synthetic | architecture ablation (Table 11) |
| `real_t4_default.pt` | 6 | 192 | 6 | 768 | 3.7 M | 9,323 real T4 tables | real-data model (Table 4) |
| `real_t4_100k_default.pt` | 6 | 192 | 6 | 768 | 3.7 M | 100,000 real T4 tables | real-data model, larger corpus (not in the paper's tables) |

Synthetic training tables come from the TFM-Playground prior dump `50x3_3_100k_classification` (100,000 tables of 50 rows × 3 features; the first 50,000 are the training stream, the remaining 50,000 the held-out non-members of the `nanotabpfn_matched20k` pool). `classifier.arch.json` carries reconstructed metadata for the pre-existing baseline model; the other seven were trained by `train_nanotabpfn_variant.py`. Not uploaded: the 2,000-, 1,000-, 500- and 100-table models of Table 10 and the two synthetic-T4 models (Gaussian copula, shuffled) of Table 4.

The two real-data models were trained on tables sampled from the [T4](https://huggingface.co/datasets/mlfoundations/t4-full) corpus (50 rows × 3 features per table, tables with at least 150 rows, at most 10 target classes). Their `*.manifest.json` lists every T4 table id, target column and feature columns that were used, which is the membership ground truth for those models: `real_t4_default` draws 9,323 tables from a single T4 chunk, `real_t4_100k_default` 100,000 tables across 94 chunks.

### TabDPT re-trainings (`tabdpt/`, `split{1,2,3}/`)

[TabDPT](https://github.com/layer6ai-labs/TabDPT) re-trained from scratch with the public [TabDPT-training](https://github.com/layer6ai-labs/TabDPT-training) code.

| file | what it is |
|---|---|
| `tabdpt/seed{42,123,456}/model.safetensors` | the full TabDPT recipe and corpus, only the random seed changed. Used to test whether fingerprints transfer across seeds. |
| `tabdpt/split{1,2,3}/model.safetensors` | TabDPT trained on a **row-level** 50/50 split of every dataset in its pre-training corpus (split seed 1, 2, 3): each model saw one half of the rows of every member dataset and never the other half. Weights only. |
| `split{1,2,3}/latest.ckpt` | the corresponding full training checkpoints written by `train.py` (model, optimiser and scheduler state; load with `torch.load(..., weights_only=False)`). |
| `split{1,2,3}/member_splits.json.zst` | zstd-compressed JSON `{"member_split_seed": s, "splits": {dataset_id: {"member_indices", "non_member_indices", "total_rows"}}}`; indices refer to the original OpenML row order. |

The split models support row-level in/out experiments only; there is no dataset-level hold-out among them.

## Behavioural fingerprint tables (`signals/`)

One parquet table per model and pool, `signals/<model>/<pool>/datasets.parquet`, with the
blind's table under `signals/tabpfn25/<pool>/`. Both audits of the paper, the label-free
external audit (Table 1) and the supervised internal audit (Table 2), read the same four
pools. Each row is one probed table, in audit order, with the columns `dataset_id`,
`true_label` (1 = member of the *target* model's pre-training corpus), `task_type`,
`success`, `n_pool`, `n_features`, and one column per signal named
`<manipulation>_<measurement>_<reduction>` (e.g. `tserum_loss_early`,
`ctx_accuracy_slope`). The manipulation, measurement and reduction vocabularies are
Tables 5–7 of the paper and `di/grid/` in the code. The tables contain no feature values
and no target values of the probed datasets.

| target model | pool | rows | members / non-members | signals | pool definition |
|---|---|---:|---|---:|---|
| `sap-rpt-oss` | `binary_t4_allcls_hybrid` | 20,100 | 16,179 / 3,921 | 678 | every table of one T4 chunk as a classification task: the 8,907 natively-categorical tables with their own target, and the 11,193 regression tables re-targeted to a column with 2–10 distinct values where one exists, otherwise to the column with the fewest distinct values binned to at most 10 classes. SAP-RPT-1-OSS was pre-trained on every T4 table with more than 150 rows, which defines membership |
| `tabdpt`, `tabdpt_seed42`, `tabdpt_seed123`, `tabdpt_seed456` | `binary_tabdpt_cls` | 160 | 88 / 72 | 1,009 | the classification datasets of TabDPT's pre-training corpus vs. those of OpenML-CC18 and CTR23 it did not use; the three seed directories hold the re-trained seeds on the same datasets |
| `nanotabpfn` | `nanotabpfn_matched20k` | 20,000 | 10,000 / 10,000 | 678 | synthetic prior tables: 10,000 of `classifier.pt`'s training tables vs. 10,000 held-out tables of the same prior |
| `realtabpfn25` | `binary_realtabpfn25` | 70 | 19 / 51 | 1,018 | the OpenML datasets among Real-TabPFN-2.5's real fine-tuning data (its Kaggle datasets are not covered) vs. TabArena datasets it did not use; 55 classification + 15 regression |

The blind for every pool is TabPFN-2.5 run from its synthetic-only checkpoints
(`tabpfn-v2.5-classifier-v2.5_default-2.ckpt`, `tabpfn-v2.5-regressor-v2.5_default.ckpt`),
aligned row-for-row by `dataset_id`.

### Reproducing the tables

```bash
hf download SprintML/tfms --include "signals/**" --local-dir data_root
# in a checkout of the code repository
uv run python -m di.audit reproduce --data-root data_root --reference-dir tests/reference
```

`reproduce` recomputes every LR / XGBoost cell of Table 2 and every cell of Table 1 on CPU
and compares them with the reference result files shipped with the code; the TabPFN-2.5
meta-classifier cells of Table 2 need a GPU. Earlier revisions of this repository carry
superseded extractions and per-policy pools that no table reads; use `main`.

## Intended use and limitations

These artefacts exist to audit tabular foundation models for training-data membership and to study what makes such audits succeed. The fingerprint tables are per-table *derived statistics* of model behaviour; they contain no feature values and no target values of the probed datasets. The T4-based manifests list table identifiers and column names of T4 tables, which is why the repository stays gated like T4 itself.

The NanoTabPFN models are research-scale (3.7 M to 116 M parameters) and are not meant as predictors. The TabDPT split models deliberately saw only half of each dataset's rows and should not be used as general-purpose TabDPT checkpoints.

## Licenses

* NanoTabPFN weights and the `signals/nanotabpfn/*`, `signals/tabdpt*/*` and `signals/sap-rpt-oss/*` fingerprint tables: Apache License 2.0, like the [code](https://github.com/layer6ai-labs/tfm-di) and [TFM-Playground](https://github.com/automl/TFM-Playground).
* The `signals/tabpfn25/*` and `signals/realtabpfn25/*` tables are statistics of outputs of TabPFN-2.5 checkpoints, which Prior Labs distributes under the [TabPFN-2.5 License v1.1](https://huggingface.co/Prior-Labs/tabpfn_2_5/blob/main/LICENSE) (non-commercial). Use them for non-commercial research only; they are not covered by the Apache-2.0 statement above.
* TabDPT re-trainings and split checkpoints: derived with the [TabDPT-training](https://github.com/layer6ai-labs/TabDPT-training) code and subject to its license.
* The real-T4 manifests reference tables of the [T4 corpus](https://huggingface.co/datasets/mlfoundations/t4-full); its terms apply to the underlying data.

## Citation

```bibtex
@article{wahdany2026dataset,
  title   = {Dataset Inference for Data Provenance and Privacy Auditing in Tabular Foundation Models},
  author  = {Wahdany, Dariush and Cresswell, Jesse C. and Guan, Naiqing and Ashari Ghomi, Atiyeh and Boenisch, Franziska and Dziedzic, Adam},
  year    = {2026}
}
```