---
license: apache-2.0
tags:
  - tabular
  - in-context-learning
  - transformer
---

# TabDPT Checkpoints

Pre-trained [TabDPT](https://github.com/JesseCresswell/tfm-mia) model weights trained with three different random seeds. Each checkpoint is from epoch 2040 of production training.

## Files

| File | Training Seed |
|---|---|
| `production_seed42.safetensors` | 42 |
| `production_seed123.safetensors` | 123 |
| `production_seed456.safetensors` | 456 |

## Model Architecture

- **Embedding size:** 512
- **Attention heads:** 8
- **Layers:** 12
- **Hidden factor:** 2
- **Max features:** 100
- **Max classes:** 10

## Benchmark Results

### Classification: Breast Cancer (binary, 30 features)

| Checkpoint | Accuracy | Ensemble Accuracy |
|---|---|---|
| seed42 | 99.4% | 99.4% |
| seed123 | 98.8% | 98.8% |
| seed456 | 98.2% | 98.2% |
| HF default (Layer6/TabDPT) | — | 99.4% |

### Classification: Wine (3-class, 13 features)

| Checkpoint | Accuracy | Ensemble Accuracy |
|---|---|---|
| seed42 | 100% | 100% |
| seed123 | 100% | 100% |
| seed456 | 100% | 100% |
| HF default (Layer6/TabDPT) | — | 100% |

### Regression: Diabetes (10 features)

| Checkpoint | MSE | Correlation |
|---|---|---|
| seed42 | 2618.6 | 0.718 |
| seed123 | 2655.1 | 0.713 |
| seed456 | 2795.5 | 0.701 |
| HF default (Layer6/TabDPT) | 2673.1 | 0.711 |

## Training Stats (from checkpoint metadata)

| Metric | seed42 | seed123 | seed456 |
|---|---|---|---|
| CC18 Accuracy | 0.877 | 0.878 | 0.879 |
| CC18 F1 | 0.870 | 0.872 | 0.873 |
| CC18 AUC | 0.927 | 0.927 | 0.928 |
| CTR Correlation | 0.830 | 0.830 | 0.827 |
| CTR R² | 0.726 | 0.730 | 0.725 |

## Format

These checkpoints were converted from PyTorch Lightning `.ckpt` files (which include optimizer state, ~295MB each) to SafeTensors format (model weights only, ~103MB each). This is the same format used by the official `Layer6/TabDPT` release. The `tabdpt` package natively loads SafeTensors via the `model_weight_path` argument — no extra conversion needed.

## Usage

```python
from tabdpt import TabDPTClassifier
from huggingface_hub import hf_hub_download

# Download once (cached afterwards)
path = hf_hub_download("dwahdany/TabDPT", "production_seed42.safetensors")

# Use exactly like the default model
clf = TabDPTClassifier(model_weight_path=path)
clf.fit(X_train, y_train)
preds = clf.predict(X_test)
```

Works identically with `TabDPTRegressor`.