| --- |
| license: other |
| license_name: neuralk-seldon-license |
| license_link: LICENSE.md |
| pipeline_tag: tabular-classification |
| extra_gated_prompt: You agree to the terms of the Neuralk Seldon License. |
| extra_gated_fields: |
| Company: text |
| I agree to the license terms: checkbox |
| tags: |
| - tabular |
| - tabular-classification |
| - foundation-model |
| - in-context-learning |
| - synthetic-prior |
| --- |
| <p align="center"> |
| <img src="https://huggingface.co/Neuralk-AI/seldon-mini/resolve/main/assets/banner.png" alt="Neuralk - The Predictive AI Company" width="100%"> |
| </p> |
|
|
| # Seldon TFM Family |
| |
| **Seldon** is a tabular foundation model by [Neuralk-AI](https://www.neuralk-ai.com). It predicts on new rows through **in-context learning**: you give it labeled example rows as context, and it predicts the target for new rows in a single forward pass, with no training or hyperparameter tuning on your data. |
| |
| This repo contains **seldon** (213.5M parameters). For now, only the **classification** checkpoint is released. |
| |
| ## Model family |
| |
| | Variant | Parameters | Availability | |
| |---|---|---| |
| | [**seldon-mini**](https://huggingface.co/Neuralk-AI/seldon-mini) | 27.4M | Open weights | |
| | [**seldon**](https://huggingface.co/Neuralk-AI/seldon) | 213.5M | Open Weights | |
| | **seldon API** | – | [Neuralk API](https://prediction.neuralk.ai/register) | |
| |
| ## Quickstart |
| |
| ```bash |
| pip install neuralk-oss |
| ``` |
| |
| ```python |
| from neuralk_oss import SeldonClassifier |
| |
| clf = SeldonClassifier.from_pretrained("seldon", device="cuda") |
| clf.fit(X_train, y_train) # stores the context; no gradient updates |
| preds = clf.predict(X_test) |
| proba = clf.predict_proba(X_test) |
| ``` |
|
|
| ## Benchmark |
|
|
| Head-to-head win rate on TabArena classification (38 datasets, 594 folds), each model with its default settings. |
|
|
| | Baseline (params) | Seldon (213.5M) win rate | |
| | --- | --- | |
| | Kumo-Tabular Small (27.5M) | 75.8% | |
| | TabICLv2 (27.6M) | 82.3% | |
| | Kumo-Tabular Medium (61.5M) | 56.6% | |
| | TabPFN-3.5-Fast (83.5M) | 73.7% | |
| | Kumo-Tabular Large (213.7M) | 38.2% | |
| | TabPFN-3.5 (219.0M) | 63.5% | |
| | LimiX-2 (406.2M) | 40.6% | |
| | TabFM (1.64B) | 52.7% | |
| |
| ## Architecture |
| |
| Seldon follows the same design as [Kumo Tabular](https://build.nvidia.com/nvidia/kumo-tabular/modelcard) and [TabFM](https://huggingface.co/google/tabfm-1.0.0-pytorch): the table is first compressed into one representation per row, and in-context learning then runs on these row representations. |
| |
| - **Tokenizer:** each cell becomes a token through a Fourier feature projection of numerical values, a categorical mask, and a missing-value indicator. Features are grouped before projection. |
| - **Cell-to-row embedding:** interleaved column and row transformers (column attention across rows uses inducing points). The result is compressed into CLS readout tokens, giving one dense vector per row. |
| - **ICL layers:** a transformer stack over the compressed row representations. Labeled context rows serve as the training set, and the model predicts the query rows. |
| - **Decoder:** an MLP head maps each query row's representation to class probabilities. |
| | Part | Parameters | Shape | |
| |---|---|---| |
| | Cell/row embedding | 9.93M | 256 channels, 6 layer pairs, 256 inducing points, 4 readout tokens | |
| | ICL stack | 203.53M | 24 layers, 1024 channels, 16 heads | |
| | **Total** | **213.46M** | | |
| |
| ## Pretraining: a synthetic prior |
| |
| Seldon is pretrained **only on synthetically generated datasets** drawn from a prior developed by Neuralk. **No real-world data was used in pretraining.** The prior is built on structural causal models (SCMs): each synthetic task comes from a randomly sampled causal graph whose nodes are linked by randomly drawn functional relationships and noise, and the features and target are read off this graph. The main goal of the prior is diversity. Its new, richer generative mechanisms go beyond classical SCM priors and vary: |
| - the graph structure |
| - the kinds of functional dependencies |
| - feature types and distributions, |
| - noise, missingness, class balance and dataset shape... |
|
|
| As a result, the model sees a much wider range of tabular problems during pretraining than any collection of real datasets would provide, and learns a general way of doing in-context inference that carries over to real-world tables. |
|
|
