Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,11 @@ license: mit
|
|
| 16 |
<img src="https://img.shields.io/badge/Discord-Join-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Discord"/>
|
| 17 |
</a>
|
| 18 |
<a href="https://github.com/Lexsi-Labs/Orion-MSP">
|
| 19 |
-
<img src="https://img.shields.io/badge/GitHub-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
</a>
|
| 21 |
</div>
|
| 22 |
|
|
@@ -25,6 +29,8 @@ license: mit
|
|
| 25 |
|
| 26 |
Orion-MSP is a tabular foundation model for in-context learning. It uses multi-scale sparse attention and Perceiver-style memory to process tabular data at multiple granularities, capturing both local feature interactions and global dataset-level patterns.
|
| 27 |
|
|
|
|
|
|
|
| 28 |
## Key Features
|
| 29 |
|
| 30 |
- **Multi-Scale Sparse Attention:** Processes features at three levels (scales 1, 4, 16) using windowed, global, and random attention patterns, reducing quadratic complexity to near-linear.
|
|
@@ -161,7 +167,7 @@ Orion-MSP consists of four main components:
|
|
| 161 |
</div>
|
| 162 |
|
| 163 |
Orion-MSP is the most consistent top performer across all three benchmarks, achieving the best overall rank.
|
| 164 |
-
- On TALENT, it ranks **
|
| 165 |
- On OpenML-CC18, Orion-MSP attains the top ACC/F1 (0.8722/0.8676), narrowly ahead of TabPFN and TabDPT.
|
| 166 |
- On TabZilla, it leads with the highest ACC/F1 and the best rank.
|
| 167 |
- Classical baselines (XGBoost/LightGBM/CatBoost/RF) trail noticeably, highlighting Orion-MSP’s robustness across diverse tabular tasks.
|
|
@@ -390,6 +396,7 @@ OrionMSP excels on narrow and stays strong on medium width, while TabPFN dominat
|
|
| 390 |
shine on wide feature spaces.
|
| 391 |
|
| 392 |
## Usage
|
|
|
|
| 393 |
|
| 394 |
```python
|
| 395 |
from orion_msp.sklearn import OrionMSPClassifier
|
|
@@ -402,9 +409,36 @@ predictions = clf.predict(X_test)
|
|
| 402 |
|
| 403 |
This code will automatically download the pre-trained model from Hugging Face and use a GPU if available.
|
| 404 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 405 |
## Installation
|
| 406 |
|
| 407 |
-
###
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 408 |
#### Option 1: From the local clone
|
| 409 |
|
| 410 |
```bash
|
|
|
|
| 16 |
<img src="https://img.shields.io/badge/Discord-Join-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Discord"/>
|
| 17 |
</a>
|
| 18 |
<a href="https://github.com/Lexsi-Labs/Orion-MSP">
|
| 19 |
+
<img src="https://img.shields.io/badge/GitHub-Orion%20MSP-181717?style=for-the-badge&logo=github&logoColor=white" alt="Orion MSP GitHub"/>
|
| 20 |
+
</a>
|
| 21 |
+
<!-- TabTune repo -->
|
| 22 |
+
<a href="https://github.com/Lexsi-Labs/TabTune">
|
| 23 |
+
<img src="https://img.shields.io/badge/GitHub-TabTune-181717?style=for-the-badge&logo=github&logoColor=white" alt="TabTune GitHub"/>
|
| 24 |
</a>
|
| 25 |
</div>
|
| 26 |
|
|
|
|
| 29 |
|
| 30 |
Orion-MSP is a tabular foundation model for in-context learning. It uses multi-scale sparse attention and Perceiver-style memory to process tabular data at multiple granularities, capturing both local feature interactions and global dataset-level patterns.
|
| 31 |
|
| 32 |
+
OrionMSP can be used either directly via its own Python package or through [TabTune](https://github.com/Lexsi-Labs/TabTune), which provides a unified interface over several tabular foundation models.
|
| 33 |
+
|
| 34 |
## Key Features
|
| 35 |
|
| 36 |
- **Multi-Scale Sparse Attention:** Processes features at three levels (scales 1, 4, 16) using windowed, global, and random attention patterns, reducing quadratic complexity to near-linear.
|
|
|
|
| 167 |
</div>
|
| 168 |
|
| 169 |
Orion-MSP is the most consistent top performer across all three benchmarks, achieving the best overall rank.
|
| 170 |
+
- On TALENT, it ranks **1** overall, while TabPFN edges the highest ACC/F1 by a hair.
|
| 171 |
- On OpenML-CC18, Orion-MSP attains the top ACC/F1 (0.8722/0.8676), narrowly ahead of TabPFN and TabDPT.
|
| 172 |
- On TabZilla, it leads with the highest ACC/F1 and the best rank.
|
| 173 |
- Classical baselines (XGBoost/LightGBM/CatBoost/RF) trail noticeably, highlighting Orion-MSP’s robustness across diverse tabular tasks.
|
|
|
|
| 396 |
shine on wide feature spaces.
|
| 397 |
|
| 398 |
## Usage
|
| 399 |
+
### Direct (OrionMSP Python package)
|
| 400 |
|
| 401 |
```python
|
| 402 |
from orion_msp.sklearn import OrionMSPClassifier
|
|
|
|
| 409 |
|
| 410 |
This code will automatically download the pre-trained model from Hugging Face and use a GPU if available.
|
| 411 |
|
| 412 |
+
### Via TabTune (unified TFM library)
|
| 413 |
+
|
| 414 |
+
```python
|
| 415 |
+
from tabtune import TabularPipeline
|
| 416 |
+
|
| 417 |
+
pipeline = TabularPipeline(
|
| 418 |
+
model_name="OrionMSP", # use OrionMSP through TabTune
|
| 419 |
+
tuning_strategy="inference", # zero-shot / in-context mode
|
| 420 |
+
tuning_params={"device": "cuda"} # or "cpu"
|
| 421 |
+
)
|
| 422 |
+
|
| 423 |
+
pipeline.fit(X_train, y_train)
|
| 424 |
+
predictions = pipeline.predict(X_test)
|
| 425 |
+
```
|
| 426 |
+
|
| 427 |
+
When used through TabTune, the OrionMSP weights are automatically downloaded from this Hugging Face repository on first use, and TabTune handles model-aware preprocessing for you.
|
| 428 |
+
|
| 429 |
+
|
| 430 |
## Installation
|
| 431 |
|
| 432 |
+
### Via TabTune (recommended if you want multiple tabular FMs)
|
| 433 |
+
|
| 434 |
+
```bash
|
| 435 |
+
pip install tabtune
|
| 436 |
+
```
|
| 437 |
+
|
| 438 |
+
This installs TabTune and its built-in OrionMSP integration; no separate orion-msp install is required.
|
| 439 |
+
|
| 440 |
+
|
| 441 |
+
### From the OrionMSP source
|
| 442 |
#### Option 1: From the local clone
|
| 443 |
|
| 444 |
```bash
|