stable-ai/LimiX-16M

LimiX-2M

LimiX-2M is a 2M-parameter tabular foundation model (TFM) introduced in the paper LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models. It utilizes a "tokenize-and-route" framework with RaBEL tokenization and a readout-aligned routing architecture to achieve state-of-the-art performance on tabular benchmarks with significantly lower compute requirements.

• Paper: LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
• Technical Report (Original LimiX): LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence
• GitHub Repository: limix-ldm-ai/LimiX
• Project Page: limix.ai

💥 News

• 2025-11-10: LimiX-2M is officially released! Compared to LimiX-16M, this smaller variant offers significantly lower GPU memory usage and faster inference speed. The retrieval mechanism has also been enhanced, further improving model performance while reducing both inference time and memory consumption.
• 2025-08-29: LimiX V1.0 Released.

➤ Sample Usage

The following example demonstrates how to use the LimiXPredictor for a classification task.

Note: You will need to clone the official repository to access the inference.predictor module.

import torch
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from huggingface_hub import hf_hub_download
import os, sys

# Setup environment for distributed backend (required by LimiXPredictor)
os.environ["RANK"] = "0"
os.environ["WORLD_SIZE"] = "1"
os.environ["MASTER_ADDR"] = "127.0.0.1"
os.environ["MASTER_PORT"] = "29500"

# Import LimiXPredictor (requires the source code from GitHub)
from inference.predictor import LimiXPredictor

# Load data
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)

# Download the model checkpoint
model_file = hf_hub_download(repo_id="stableai-org/LimiX-2M", filename="LimiX-2M.ckpt")

# Initialize and run inference
clf = LimiXPredictor(
    device=torch.device('cuda' if torch.cuda.is_available() else 'cpu'), 
    model_path=model_file, 
    inference_config='config/cls_default_retrieval.json'
)
prediction = clf.predict(X_train, y_train, X_test)

print("roc_auc_score:", roc_auc_score(y_test, prediction[:, 1]))
print("accuracy_score:", accuracy_score(y_test, np.argmax(prediction, axis=1)))

➤ Overview

We introduce LimiX, the first installment of our LDM series. LimiX aims to push generality further: a single model that handles classification, regression, missing-value imputation, feature selection, sample selection, and causal inference under one training and inference recipe, advancing the shift from bespoke pipelines to unified, foundation-style tabular learning.

LimiX adopts a transformer architecture optimized for structured data modeling and task generalization. The model first embeds features X and targets Y from the prior knowledge base into token representations. Within the core modules, attention mechanisms are applied across both sample and feature dimensions to identify salient patterns in key samples and features.

➤ Tutorials

➩ Installation

Build manually

Install Python dependencies:

pip install scikit-learn  einops  huggingface-hub matplotlib networkx numpy pandas  scipy tqdm typing_extensions xgboost kditransform hyperopt

Download source code

git clone https://github.com/limix-ldm/LimiX.git
cd LimiX

➤ License

The code in the associated repository is open-sourced under the Apache-2.0 license, while the usage of the LimiX model weights is subject to the Model License. The LimiX weights are fully available for academic research and may be used commercially upon obtaining proper authorization.

➤ Citation

@article{LimiX-2M,
  title={LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models},
  author={Zhang, Xingxuan and others},
  journal={arXiv preprint arXiv:2606.04485},
  year={2026}
}

@article{LimiX,
  title={LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence},
  author={Zhang, Xingxuan and Ren, Gang and Yu, Han and Yuan, Hao and Wang, Hui and Li, Jiansheng and Wu, Jiayun and Mo, Lang and Mao, Li and Hao, Mingchao and others},
  journal={arXiv preprint arXiv:2509.03505},
  year={2025}
}