Upload README.md

.gitattributes

@@ -33,3 +33,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+figures/media/image1.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image10.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image11.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image2.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image3.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image4.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image5.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image6.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image7.png filter=lfs diff=lfs merge=lfs -text
+figures/media/image8.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,305 +1,331 @@
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/LimiX-Logo.png" alt="LimiX summary" width="89%">
-</div>
-
-#  💥 News
- - 2025-11-10: LimiX-2M is officially released! Compared to LimiX-16M, this smaller variant offers significantly lower GPU memory usage and faster inference speed. The retrieval mechanism has also been enhanced, further improving model performance while reducing both inference time and memory consumption.
- - 2025-08-29: LimiX V1.0 Released.
-
-#  ⚡ Latest Results Compared with SOTA Models
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/BCCO-CLS.png"  width="30%" style="display:inline-block;">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/TabArena-CLS.png"  width="30%" style="display:inline-block;">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/TabZilla-CLS.png" width="30%" style="display:inline-block;">  
-</div>
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/BCCO-REG.png"  width="30%" style="display:inline-block;">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/TabArena-REG.png" width="30%" style="display:inline-block;">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/CTR23-REG.png" width="30%" style="display:inline-block;">
-</div>
-
-
-# ➤ Overview
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/LimiX_Summary.png" alt="LimiX summary" width="89%">
-</div>
-We introduce LimiX, the first installment of our LDM series. LimiX aims to push generality further: a single model that handles classification, regression, missing-value imputation, feature selection, sample selection, and causal inference under one training and inference recipe, advancing the shift from bespoke pipelines to unified, foundation-style tabular learning.
-
-LimiX adopts a transformer architecture optimized for structured data modeling and task generalization. The model first embeds features X and targets Y from the prior knowledge base into token representations. Within the core modules, attention mechanisms are applied across both sample and feature dimensions to identify salient patterns in key samples and features. The resulting high-dimensional representations are then passed to regression and classification heads, enabling the model to support diverse predictive tasks. 
-
-For details, please refer to the technical report at the link: [LimiX:Unleashing Structured-Data Modeling Capability for Generalist Intelligence](https://arxiv.org/abs/2509.03505) or [LimiX_Technical_Report.pdf](https://github.com/limix-ldm/LimiX/blob/main/LimiX_Technical_Report.pdf).
-
-# ➤ Superior Performance 
-The LimiX model achieved SOTA performance across multiple tasks.
-
-## ➩ Classification (Tech Report)
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/Classifier.png" alt="Classification" width="80%">
-</div>
-
-## ➩ Regression (Tech Report)
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/Regression.png" alt="Regression" width="60%">
-</div>
-
-## ➩ Missing Values Imputation (Tech Report)
-<div align="center">
-  <img src="https://github.com/limix-ldm/LimiX/raw/main/doc/MissingValueImputation.png" alt="Missing value imputation" width="80%">
-</div>
-
-# ➤ Tutorials 
-## ➩ Installation
-### Option 1 (recommended): Use the Dockerfile
-Download [Dockerfile](https://github.com/limix-ldm/LimiX/blob/main/Dockerfile)
-```bash
-docker build --network=host -t limix/infe:v1 --build-arg FROM_IMAGES=nvidia/cuda:12.2.0-base-ubuntu22.04 -f Dockerfile .
-```
-
-### Option 2: Build manually
-Download the prebuilt flash_attn files
-```bash
-wget -O flash_attn-2.8.0.post2+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux_x86_64.whl https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.0.post2/flash_attn-2.8.0.post2+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
-```
-Install Python dependencies
-```bash
-pip install python==3.12.7 torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1
-pip install flash_attn-2.8.0.post2+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
-pip install scikit-learn  einops  huggingface-hub matplotlib networkx numpy pandas  scipy tqdm typing_extensions xgboost kditransform hyperopt
-```
-
-### Download source code
-```bash
-git clone https://github.com/limix-ldm/LimiX.git
-cd LimiX
-```
-
-# ➤ Inference
-LimiX supports tasks such as classification, regression, and missing value imputation
-## ➩ Model download
-| Model size | Download link | Tasks supported |
-| --- | --- | --- |
-| LimiX-16M | [LimiX-16M.ckpt](https://huggingface.co/stableai-org/LimiX-16M/tree/main) |  ✅ classification  ✅regression   ✅missing value imputation |
-| LimiX-2M | [LimiX-2M.ckpt](https://huggingface.co/stableai-org/LimiX-2M/tree/main) |  ✅ classification  ✅regression   ✅missing value imputation |
-
-## ➩ Interface description
-
-### Model Creation
-```python
-class LimiXPredictor:
-    def __init__(self,
-                 device:torch.device,
-                 model_path:str,
-                 mix_precision:bool=True,
-                 inference_config: list|str,
-                 categorical_features_indices:List[int]|None=None,
-                 outlier_remove_std: float=12,
-                 softmax_temperature:float=0.9,
-                 task_type: Literal['Classification', 'Regression']='Classification',
-                 mask_prediction:bool=False,
-                 inference_with_DDP: bool = False,
-                 seed:int=0)
-```
-| Parameter | Data Type | Description |
-|--------|----------|----------|
-| device | torch.device | The hardware that loads the model |
-| model_path | str | The path to the model that needs to be loaded |
-| mix_precision | bool | Whether to enable the mixed precision inference |
-| inference_config | list/str | Configuration file used for inference |
-| categorical_features_indices | list | The indices of categorical columns in the tabular data |
-| outlier_remove_std | float | The threshold is employed to remove outliers, defined as values that are multiples of the standard deviation |
-| softmax_temperature | float | The temperature used to control the behavior of softmax operator |
-| task_type | str | The task type which can be either "Classification" or "Regression" |
-| mask_prediction | bool | Whether to enable missing value imputation |
-| inference_with_DDP | bool | Whether to enable DDP during inference |
-| seed | int | The seed to control random states |
-### Predict
-```python
-def predict(self, x_train:np.ndarray, y_train:np.ndarray, x_test:np.ndarray) -> np.ndarray:
-```
-| Parameter   | Data Type    | Description           |
-| ------- | ---------- | ----------------- |
-| x_train  | np.ndarray  | The input features of the training set   |
-| y_train  | np.ndarray  | The target variable of the training set   |
-| x_test   | np.ndarray  | The input features of the test set   |
-
-## Inference Configuration File Description
-| Configuration File Name | Description | Difference |
-| ------- | ---------- | ----- |
-| cls_default_retrieval.json | Default **classification task** inference configuration file **with retrieval** | Better classification performance |
-| cls_default_noretrieval.json | Default **classification task** inference configuration file **without retrieval** | Faster speed, lower memory requirements |
-| reg_default_retrieval.json | Default **regression task** inference configuration file **with retrieval** | Better regression performance |
-| reg_default_noretrieval.json | Default **regression task** inference configuration file **without retrieval** | Faster speed, lower memory requirements |
-| reg_default_noretrieval_MVI.json | Default inference configuration file for **missing value imputation task** |  |
-
-## ➩ Ensemble Inference Based on Sample Retrieval
-
-For a detailed technical introduction to Ensemble Inference Based on Sample Retrieval, please refer to the [technical report](https://github.com/limix-ldm/LimiX/blob/main/LimiX_Technical_Report.pdf).
+**LimiX-hugging face**
+
+1\. **Model Introduction**
+
+**LimiX** is a new class of tabular AI model designed to overcome one of modern machine learning's longest-standing bottlenecks: structured data. With only **2M parameters**, **LimiX-2M** sets a new state-of-the-art across classification, regression, and missing-value imputation, surpassing XGBoost, CatBoost, AutoGluon, and TabPFN, and approaching the performance level of the larger LimiX-16M. Its lightweight, training-free design makes advanced tabular modeling accessible on ordinary hardware while preserving full transparency and offline deployability.
+
+  ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- -------------------------------------------------------------------------------
+  ![](./figures/media/image1.png){width="1.75in" height="1.0416666666666667in"}   ![](./figures/media/image2.png){width="1.75in" height="1.0416666666666667in"}   ![](./figures/media/image3.png){width="1.75in" height="1.0416666666666667in"}
+  ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- -------------------------------------------------------------------------------
+
+  ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- -------------------------------------------------------------------------------
+  ![](./figures/media/image4.png){width="1.75in" height="1.0416666666666667in"}   ![](./figures/media/image5.png){width="1.75in" height="1.0416666666666667in"}   ![](./figures/media/image6.png){width="1.75in" height="1.0416666666666667in"}
+  ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- -------------------------------------------------------------------------------
+
+**Key Features**
+
+**Unified Tabular Reasoning：**
+
+End-to-end designed for multi-task tabular intelligence, enabling a single model to handle classification, regression, and imputation without additional tuning, preprocessing, or task-specific fine-tuning.
+
+**Training-Free, Context-Driven Inference：**
+
+Operates directly through context learning: no training, no hyperparameters, no preprocessing pipelines. LimiX automatically interprets and processes raw tabular inputs for immediate use.
+
+**Lightweight & Efficient Deployment：**
+
+A compact 2M-parameter architecture enables fast inference and smooth operation on standard CPUs and laptops, dramatically reducing compute requirements for advanced tabular modeling.
+
+2\. **Model Architecture & Pretraining Procedures**
+
+LimiX adopts a 12-block transformer architecture with axis-wise attention to features and samples, supported by pre-normalized LayerNorm for stable scaling. The LimiX-16M variant uses an asymmetric design, two feature-axis passes and one sample-axis pass per block, to strengthen feature interaction modeling in heterogeneous schemas with minimal overhead.
+
+To learn the joint distribution of tabular variables, LimiX is pretrained through Context-Conditional Masked Modeling (CCMM). By masking table cells and conditioning predictions on a small set of context rows, the model internalizes a wide range of conditional dependencies while adapting to new datasets without training or labels.
+
+![](./figures/media/image7.png){width="5.75in" height="4.375in"}
+
+3\. **Evaluation Results**
+
+**Classification**
+
+![](./figures/media/image8.png){width="5.75in" height="3.2604166666666665in"}
+
+On the BCCO-CLS benchmark, LimiX-16M establishes leading performance by significantly outperforming AutoGluon and all PFN variants in mean AUC, Accuracy, and F1 scores, with substantially better ranks. LimiX-2M also marks a clear lead over these models in most metrics, except for its AUC rank.
+
+**Regression**
+
+![](./figures/media/image9.png){width="4.583333333333333in" height="3.4375in"}
+
+LimiX-16M achieves the best overall scores and rankings on TALENT-REG, with the PFN models and LimiX-2M emerging as close runners-up in both R² and RMSE.
+
+**Missing Value Imputation**
+
+LimiX introduces the first training-free, in-context approach for missing-value imputation on entirely new datasets. Across a wide set of real-world benchmarks, LimiX-16M delivers the best performance, achieving lower RMSE and error rates than classical and learned imputers including KNN, MICE, MissForest, GAIN, and MIWAE. Unlike all prior methods, which depend on additional fitting, LimiX performs imputation directly from context with consistently superior accuracy.
+
+![](./figures/media/image10.png){width="5.75in" height="1.4270833333333333in"}
+
+**Finetune**
+
+Using an attention-based retrieval--guided downsampling strategy, LimiX-16M fine-tunes on compact, highly relevant in-context episodes rather than full long contexts, substantially improving sample efficiency and reducing training cost. This approach enables LimiX-16M to significantly outperform strong baselines such as TabDPT and TabPFN-v2, with notable AUC gains across BCCO-CLS datasets.
+
+![](./figures/media/image11.png){width="5.75in" height="2.40625in"}
+
+4\. **Deployment**
+
+**Environment Preparation**
+
+Recommended to deploy with Docker. Download the [Dockerfile](https://github.com/limix-ldm/LimiX/blob/main/Dockerfile) from the repository and execute the following command to build the image:
+
+  -----------------------------------------------------------------------------------------------------------------------------
+  Bash\
+  docker build \--network=host -t limix/infe:v1 \--build-arg FROM\_IMAGES=nvidia/cuda:12.2.0-base-ubuntu22.04 -f Dockerfile .
+
+  -----------------------------------------------------------------------------------------------------------------------------
+
+For manual deployment, install dependencies:
+
+  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+  Bash\
+  \# Download precompiled flash\_attn file\
+  wget -O flash\_attn-2.8.0.post2+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux\_x86\_64.whl https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.0.post2/flash\_attn-2.8.0.post2+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux\_x86\_64.whl\
+  \# Install basic dependencies\
+  pip install python==3.12.7 torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1\
+  pip install flash\_attn-2.8.0.post2+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux\_x86\_64.whl\
+  pip install scikit-learn einops huggingface-hub matplotlib networkx numpy pandas scipy tqdm typing\_extensions xgboost kditransform hyperopt
+
+  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+
+**Model Download**
+
+Download model weights via Hugging Face Hub:
+
+  ---------------------------------------------------------------------------------------------------------------------------
+  Python\
+  from huggingface\_hub import hf\_hub\_download\
+  model\_file = hf\_hub\_download(repo\_id=\"stableai-org/LimiX-16M\", filename=\"LimiX-16M.ckpt\", local\_dir=\"./cache\")
+
+  ---------------------------------------------------------------------------------------------------------------------------
+
+5\. **Model Usage**
+
+**Classification Task Example**
+
+  ----------------------------------------------------------------------------------------------------------------------------
+  Python\
+  from sklearn.datasets import load\_breast\_cancer\
+  from sklearn.metrics import accuracy\_score, roc\_auc\_score\
+  from sklearn.model\_selection import train\_test\_split\
+  from huggingface\_hub import hf\_hub\_download\
+  import numpy as np\
+  import os, sys\
+  \
+  os.environ\[\"RANK\"\] = \"0\"\
+  os.environ\[\"WORLD\_SIZE\"\] = \"1\"\
+  os.environ\[\"MASTER\_ADDR\"\] = \"127.0.0.1\"\
+  os.environ\[\"MASTER\_PORT\"\] = \"29500\"\
+  \
+  ROOT\_DIR = os.path.abspath(os.path.join(os.path.dirname(\_\_file\_\_), \"..\"))\
+  if ROOT\_DIR not in sys.path:\
+  sys.path.insert(0, ROOT\_DIR)\
+  from inference.predictor import LimiXPredictor\
+  \
+  X, y = load\_breast\_cancer(return\_X\_y=True)\
+  X\_train, X\_test, y\_train, y\_test = train\_test\_split(X, y, test\_size=0.5, random\_state=42)\
+  \
+  model\_file = hf\_hub\_download(repo\_id=\"stableai-org/LimiX-16M\", filename=\"LimiX-16M.ckpt\", local\_dir=\"./cache\")\
+  \
+  clf = LimiXPredictor(device=\'cuda\', model\_path=model\_file, inference\_config=\'config/cls\_default\_retrieval.json\')\
+  prediction = clf.predict(X\_train, y\_train, X\_test)\
+  \
+  print(\"roc\_auc\_score:\", roc\_auc\_score(y\_test, prediction\[:, 1\]))\
+  print(\"accuracy\_score:\", accuracy\_score(y\_test, np.argmax(prediction, axis=1)))
+
+  ----------------------------------------------------------------------------------------------------------------------------
+
+**Regression Task Example**
+
+  ------------------------------------------------------------------------------------------------------------------------------
+  Python\
+  from functools import partial\
+  \
+  from sklearn.datasets import fetch\_california\_housing\
+  from sklearn.model\_selection import train\_test\_split\
+  from sklearn.metrics import r2\_score\
+  from huggingface\_hub import hf\_hub\_download\
+  try:\
+  from sklearn.metrics import root\_mean\_squared\_error as mean\_squared\_error\
+  except:\
+  from sklearn.metrics import mean\_squared\_error\
+  mean\_squared\_error = partial(mean\_squared\_error, squared=False)\
+  import os, sys\
+  \
+  os.environ\[\"RANK\"\] = \"0\"\
+  os.environ\[\"WORLD\_SIZE\"\] = \"1\"\
+  os.environ\[\"MASTER\_ADDR\"\] = \"127.0.0.1\"\
+  os.environ\[\"MASTER\_PORT\"\] = \"29500\"\
+  \
+  ROOT\_DIR = os.path.abspath(os.path.join(os.path.dirname(\_\_file\_\_), \"..\"))\
+  if ROOT\_DIR not in sys.path:\
+  sys.path.insert(0, ROOT\_DIR)\
+  from inference.predictor import LimiXPredictor\
+  \
+  house\_data = fetch\_california\_housing()\
+  X, y = house\_data.data, house\_data.target\
+  X\_train, X\_test, y\_train, y\_test = train\_test\_split(X, y, test\_size=0.33, random\_state=42)\
+  \
+  y\_mean = y\_train.mean()\
+  y\_std = y\_train.std()\
+  y\_train\_normalized = (y\_train - y\_mean) / y\_std\
+  y\_test\_normalized = (y\_test - y\_mean) / y\_std\
+  \
+  model\_path = hf\_hub\_download(repo\_id=\"stableai-org/LimiX-16M\", filename=\"LimiX-16M.ckpt\", local\_dir=\"./cache\")\
+  \
+  model = LimiXPredictor(device=\'cuda\', model\_path=model\_path, inference\_config=\'config/reg\_default\_retrieval.json\')\
+  y\_pred = model.predict(X\_train, y\_train\_normalized, X\_test)\
+  \
+  \# Compute RMSE and R²\
+  y\_pred = y\_pred.to(\'cpu\').numpy()\
+  rmse = mean\_squared\_error(y\_test\_normalized, y\_pred)\
+  r2 = r2\_score(y\_test\_normalized, y\_pred)\
+  \
+  print(f\'RMSE: {rmse}\')\
+  print(f\'R2: {r2}\')
+
+  ------------------------------------------------------------------------------------------------------------------------------
+
+**Ensemble Inference Based on Sample Retrieval**
+
+For a detailed technical introduction to Ensemble Inference Based on Sample Retrieval, please refer to the [[technical report]{.underline}](https://github.com/limix-ldm/LimiX/blob/main/LimiX_Technical_Report.pdf).
 
 Considering inference speed and memory requirements, ensemble inference based on sample retrieval currently only supports hardware with specifications higher than the NVIDIA RTX 4090 GPU.
 
-### Classification Task
+**Classification Task**
 
-```
-python inference_classifier.py --save_name your_save_name --inference_config_path path_to_retrieval_config --data_dir path_to_data
-```
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
+  Plain Text\
+  python inference\_classifier.py \--save\_name your\_save\_name \--inference\_config\_path path\_to\_retrieval\_config \--data\_dir path\_to\_data
 
-### Regression Task
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
 
-```
-python inference_regression.py --save_name your_save_name --inference_config_path path_to_retrieval_config --data_dir path_to_data
-```
+**Regression Task**
 
-### Customizing Data Preprocessing for Inference Tasks
-#### First, Generate the Inference Configuration File
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
+  Plain Text\
+  python inference\_regression.py \--save\_name your\_save\_name \--inference\_config\_path path\_to\_retrieval\_config \--data\_dir path\_to\_data
 
-```python
-generate_inference_config()
-```
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
 
-### Classification Task
-#### Single GPU or CPU
+**Customizing Data Preprocessing for Inference Tasks**
 
-```
-python  inference_classifier.py --save_name your_save_name --inference_config_path path_to_retrieval_config --data_dir path_to_data
-```
+**First, Generate the Inference Configuration File**
 
-#### Multi-GPU Distributed Inference
+generate\_inference\_config()
 
-```
-torchrun --nproc_per_node=8  inference_classifier.py --save_name your_save_name --inference_config_path path_to_retrieval_config --data_dir path_to_data --inference_with_DDP
-```
+**Classification Task**
 
-### Regression Task
-#### Single GPU or CPU
+**Single GPU or CPU**
 
-```
-python  inference_regression.py --save_name your_save_name --inference_config_path path_to_retrieval_config --data_dir path_to_data
-```
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
+  Plain Text\
+  python inference\_classifier.py \--save\_name your\_save\_name \--inference\_config\_path path\_to\_retrieval\_config \--data\_dir path\_to\_data
 
-#### Multi-GPU Distributed Inference
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
 
-```
-torchrun --nproc_per_node=8  inference_regression.py --save_name your_save_name --inference_config_path path_to_retrieval_config --data_dir path_to_data --inference_with_DDP
-```
+**Multi-GPU Distributed Inference**
+
+  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+  Plain Text\
+  torchrun \--nproc\_per\_node=8 inference\_classifier.py \--save\_name your\_save\_name \--inference\_config\_path path\_to\_retrieval\_config \--data\_dir path\_to\_data \--inference\_with\_DDP
+
+  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+
+**Regression Task**
+
+**Single GPU or CPU**
+
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
+  Plain Text\
+  python inference\_regression.py \--save\_name your\_save\_name \--inference\_config\_path path\_to\_retrieval\_config \--data\_dir path\_to\_data
+
+  ---------------------------------------------------------------------------------------------------------------------------------------------------
+
+**Multi-GPU Distributed Inference**
+
+  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+  Plain Text\
+  torchrun \--nproc\_per\_node=8 inference\_regression.py \--save\_name your\_save\_name \--inference\_config\_path path\_to\_retrieval\_config \--data\_dir path\_to\_data \--inference\_with\_DDP
+
+  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+
+**Retrieval Optimization Project**
 
-### Retrieval Optimization Project
 This project implements an optimized retrieval system. To achieve the best performance, we utilize Optuna for hyperparameter tuning of retrieval parameters.
-#### Installation
+
+**Installation**
+
 Ensure you have the required dependencies installed:
-```
-pip install optuna
-```
-#### Usage
+
+  --------------------
+  Plain Text\
+  pip install optuna
+
+  --------------------
+
+**Usage**
+
 For standard inference using pre-optimized parameters, refer to the code below:
-```
-searchInference = RetrievalSearchHyperparameters(
-           dict(device_id=0,model_path=model_path), X_train, y_train, X_test, y_test,
-)
-config, result = searchInference.search(n_trials=10, metric="AUC",
-              inference_config='config/cls_default_retrieval.json',task_type="cls")
-```
+
+  ------------------------------------------------------------------------------------
+  Plain Text\
+  searchInference = RetrievalSearchHyperparameters(\
+  dict(device\_id=0,model\_path=model\_path), X\_train, y\_train, X\_test, y\_test,\
+  )\
+  config, result = searchInference.search(n\_trials=10, metric=\"AUC\",\
+  inference\_config=\'config/cls\_default\_retrieval.json\',task\_type=\"cls\")
+
+  ------------------------------------------------------------------------------------
+
 This will launch an Optuna study to find the best combination of retrieval parameters for your specific dataset and use case.
 
-## ➩ Classification
-```python
-from sklearn.datasets import load_breast_cancer
-from sklearn.metrics import accuracy_score, roc_auc_score
-from sklearn.model_selection import train_test_split
-from huggingface_hub import hf_hub_download
-import numpy as np
-import os, sys
-
-os.environ["RANK"] = "0"
-os.environ["WORLD_SIZE"] = "1"
-os.environ["MASTER_ADDR"] = "127.0.0.1"
-os.environ["MASTER_PORT"] = "29500"
-
-ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
-if ROOT_DIR not in sys.path:
-    sys.path.insert(0, ROOT_DIR)
-from inference.predictor import LimiXPredictor
-
-X, y = load_breast_cancer(return_X_y=True)
-X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)
-
-model_file = hf_hub_download(repo_id="stableai-org/LimiX-16M", filename="LimiX-16M.ckpt", local_dir="./cache")
-
-clf = LimiXPredictor(device='cuda', model_path=model_file, inference_config='config/cls_default_retrieval.json')
-prediction = clf.predict(X_train, y_train, X_test)
-
-print("roc_auc_score:", roc_auc_score(y_test, prediction[:, 1]))
-print("accuracy_score:", accuracy_score(y_test, np.argmax(prediction, axis=1)))
-```
-For additional examples, refer to [inference_classifier.py](./inference_classifier.py)
-
-## ➩ Regression
-```python
-from functools import partial
-
-from sklearn.datasets import fetch_california_housing
-from sklearn.model_selection import train_test_split
-from sklearn.metrics import r2_score
-from huggingface_hub import hf_hub_download
-try:
-    from sklearn.metrics import root_mean_squared_error as mean_squared_error
-except:
-    from sklearn.metrics import mean_squared_error
-    mean_squared_error = partial(mean_squared_error, squared=False)
-import os, sys
-
-os.environ["RANK"] = "0"
-os.environ["WORLD_SIZE"] = "1"
-os.environ["MASTER_ADDR"] = "127.0.0.1"
-os.environ["MASTER_PORT"] = "29500"
-
-ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
-if ROOT_DIR not in sys.path:
-    sys.path.insert(0, ROOT_DIR)
-from inference.predictor import LimiXPredictor
-
-house_data = fetch_california_housing()
-X, y = house_data.data, house_data.target
-X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)
-
-y_mean = y_train.mean()
-y_std = y_train.std()
-y_train_normalized = (y_train - y_mean) / y_std
-y_test_normalized = (y_test - y_mean) / y_std
-
-model_path = hf_hub_download(repo_id="stableai-org/LimiX-16M", filename="LimiX-16M.ckpt", local_dir="./cache")
-
-model = LimiXPredictor(device='cuda', model_path=model_path, inference_config='config/reg_default_retrieval.json')
-y_pred = model.predict(X_train, y_train_normalized, X_test)    
-
-# Compute RMSE and R²
-y_pred = y_pred.to('cpu').numpy()
-rmse = mean_squared_error(y_test_normalized, y_pred)
-r2 = r2_score(y_test_normalized, y_pred)
-
-print(f'RMSE: {rmse}')
-print(f'R2: {r2}')
-```
-For additional examples, refer to [inference_regression.py](https://github.com/limix-ldm/LimiX/raw/main/inference_regression.py)
-
-## ➩ Missing value imputation
-For the demo file, see [demo_missing_value_imputation.py](https://github.com/limix-ldm/LimiX/raw/main/examples/inference_regression.py)
-
-# ➤ Link
- - LimiX:Unleashing Structured-Data Modeling Capability for Generalist Intelligence: [LimiX:Unleashing Structured-Data Modeling Capability for Generalist Intelligence](https://arxiv.org/abs/2509.03505)
- - LimiX Technical Report: [LimiX_Technical_Report.pdf](https://github.com/limix-ldm/LimiX/blob/main/LimiX_Technical_Report.pdf)
- - Detailed instructions for using Limix: [Visit the official Limix documentation](https://www.limix.ai/doc/)
- - Balance Comprehensive Challenging Omni-domain Classification Benchmark: [bcco_cls](https://huggingface.co/datasets/stableai-org/bcco_cls)
- - Balance Comprehensive Challenging Omni-domain Regression Benchmark: [bcco_reg](https://huggingface.co/datasets/stableai-org/bcco_reg)
-
-# ➤ License
-The code in this repository is open-sourced under the [Apache-2.0](LICENSE.txt) license, while the usage of the LimiX model weights is subject to the Model License. The LimiX weights are fully available for academic research and may be used commercially upon obtaining proper authorization.
-
-# ➤ Citation
-```
-@article{LimiX,
-  title={LimiX:Unleashing Structured-Data Modeling Capability for Generalist Intelligence},
-  author={LimiXTeam},
-  journal={arXiv preprint arXiv:2509.03505},
-  year={2025}
-}
-```
+6\. **Tool Invocation**
+
+The LimiX model can integrate with various toolchains for extended functionality:
+
+**Data Processing Tools**: Integrates with pandas and scikit-learn for data cleaning, feature engineering, and result evaluation (e.g., r2\_score, mean\_squared\_error).
+
+**Hyperparameter Optimization Tools**: Optimize retrieval parameters via the hyperopt library, example as follows:
+
+  ------------------------------------------------------------------------------------
+  Python\
+  \# Hyperparameter search example (refer to inference\_regression.py)\
+  from utils.inference\_utils import sample\_inferece\_params\
+  hyperopt\_config, base\_config = sample\_inferece\_params(rng, 2, 4)\
+  model.set\_inference\_config(inference\_config=hyperopt\_config, \*\*base\_config)
+
+  ------------------------------------------------------------------------------------
+
+**Distributed Inference**: Supports DDP (Distributed Data Parallel) mode for multi-GPU acceleration via torch.distributed.
+
+7\. **License**
+
+**Code License**: The repository code is licensed under the \[Apache-2.0 License\](LICENSE.txt), allowing commercial use and secondary development with retention of the original copyright notice.
+
+**Model Weight License**: The use of LimiX model weights is subject to a separate Model License:
+
+Fully open for academic research without authorization required.
+
+Commercial use requires official authorization (refer to the license application process on the [StableAI official website](https://www.stable-ai.ai/)).
+
+8\. **Third-Party Notices**
+
+This project uses the following third-party components, whose usage is governed by their respective licenses:
+
+**PyTorch**: BSD-style license
+
+**scikit-learn**: BSD license
+
+**flash-attention**: MIT License
+
+**Hugging Face Hub**: Apache-2.0 License
+
+For the complete list of dependencies and license information, refer to requirements.txt and the official documentation of each component.
+
+9\. **Contact Us**
+
+**Official Documentation**: <https://www.limix.ai/doc/>
+
+**GitHub Repository**: <https://github.com/limix-ldm/LimiX> (Submit issues for questions)
+
+**Official Website**: <https://www.stable-ai.ai/> (For commercial cooperation and license inquiries)
+
+**Technical Report**: [LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence](https://arxiv.org/abs/2509.03505)