---
license: mit
---
# Orion-BiX: Bi-Axial Meta-Learning Model for Tabular In-Context Learning
**Orion-BiX** is an advanced tabular foundation model that combines **Bi-Axial Attention** with **Meta-Learning** capabilities for few-shot tabular classification. The model extends the TabICL architecture with alternating attention patterns and episode-based training.
The model is part of **Orion**, a family of tabular foundation models with various enhancements.
### Key Innovations
1. **Bi-Axial Attention**: Alternating attention patterns (Standard → Grouped → Hierarchical → Relational) that capture multi-scale feature interactions within tabular data
2. **Meta-Learning with k-NN Support Selection**: Episode-based training with intelligent support set selection using similarity metrics
3. **Three-Component Architecture**: Column embedding (Set Transformer), Bi-Axial row interaction, and In-Context Learning prediction
### Architecture Overview
```
Input → tf_col (Set Transformer) → Bi-Axial Attention → tf_icl (ICL) → Output
```
**Component Details:**
- **tf_col (Column Embedder)**: Set Transformer for statistical distribution learning across features
- **Bi-Axial Attention**: Replaces standard RowInteraction with alternating attention patterns:
- Standard Cross-Feature Attention
- Grouped Feature Attention
- Hierarchical Feature Attention
- Relational Feature Attention
- CLS Token Aggregation
- **tf_icl (ICL Predictor)**: In-context learning module for few-shot prediction
## Usage
### Direct (OrionBiX Python package)
```python
from orion_bix.sklearn import OrionBiXClassifier
# Initialize and use
clf = OrionBiXClassifier()
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)
```
This code will automatically download the pre-trained model from Hugging Face and use a GPU if available.
### Via TabTune (unified TFM library)
OrionBix can be used either directly via its own Python package or through [TabTune](https://github.com/Lexsi-Labs/TabTune), which provides a unified interface over several tabular foundation models.
```python
from tabtune import TabularPipeline
pipeline = TabularPipeline(
model_name="OrionBix", # use OrionBix through TabTune
tuning_strategy="inference", # zero-shot / in-context mode
tuning_params={"device": "cuda"} # or "cpu"
)
pipeline.fit(X_train, y_train)
predictions = pipeline.predict(X_test)
```
When used through TabTune, the OrionBiX weights are automatically downloaded from this Hugging Face repository on first use, and TabTune handles model-aware preprocessing for you.
## Installation
### Via TabTune (recommended if you want multiple tabular FMs)
```bash
pip install tabtune
```
This installs TabTune and its built-in OrionBiX integration; no separate orion-bix install is required.
### From the OrionBiX source
#### Option 1: From the local clone
```bash
cd orion-bix
pip install -e .
```
#### Option 2: From the Git Remote
```bash
pip install git+https://github.com/Lexsi-Labs/Orion-BiX.git
```
## Citation
If you use Orion-BiX in your research, please cite our [paper](https://arxiv.org/abs/2512.00181)::
```bibtex
@misc{bouadi2025orionbix,
title={Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning},
author={Mohamed Bouadi and Pratinav Seth and Aditya Tanna and Vinay Kumar Sankarapu},
year={2025},
eprint={2512.00181},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2512.00181},
}
```
