Improve model card: Add pipeline tag, paper/project links, authors & full content
Browse filesThis PR significantly enhances the model card by incorporating rich, detailed content from the GitHub repository, making it more informative and useful for the community.
Key improvements include:
- Adding the `pipeline_tag: table-question-answering` to the metadata, which improves discoverability on the Hugging Face Hub.
- Adding a direct link to the paper: [Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning](https://huggingface.co/papers/2512.00181).
- Listing the authors of the paper: Mohamed Bouadi, Pratinav Seth, Aditya Tanna, Vinay Kumar Sankarapu.
- Including a link to the project homepage: https://www.lexsi.ai/.
- Integrating detailed sections on the model's approach, architecture, installation, usage, preprocessing, and performance metrics directly from the GitHub README.
- Preserving the `logo.png` image path to maintain existing functionality while using `figures/` paths for new performance images from the GitHub README.
- Updating the citation section with the more comprehensive BibTeX entry from the GitHub repository.
Please review these updates.
|
@@ -1,7 +1,12 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
| 3 |
---
|
| 4 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
<div align="center">
|
| 6 |
<img src="logo.png" alt="Orion-BiX Logo" width="700"/>
|
| 7 |
</div>
|
|
@@ -10,7 +15,7 @@ license: mit
|
|
| 10 |
<a href="https://lexsi.ai/">
|
| 11 |
<img src="https://img.shields.io/badge/Lexsi-Homepage-FF6B6B?style=for-the-badge" alt="Homepage"/>
|
| 12 |
</a>
|
| 13 |
-
<a href="https://huggingface.co/Lexsi">
|
| 14 |
<img src="https://img.shields.io/badge/π€%20Hugging%20Face-Lexsi AI-FFD21E?style=for-the-badge" alt="Hugging Face"/>
|
| 15 |
</a>
|
| 16 |
<a href="https://discord.gg/dSB62Q7A">
|
|
@@ -21,52 +26,58 @@ license: mit
|
|
| 21 |
</a>
|
| 22 |
</div>
|
| 23 |
|
|
|
|
|
|
|
|
|
|
| 24 |
|
|
|
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
**Orion-BiX** is an advanced tabular foundation model that combines **Bi-Axial Attention** with **Meta-Learning** capabilities for few-shot tabular classification. The model extends the TabICL architecture with alternating attention patterns and episode-based training.
|
| 29 |
|
| 30 |
-
|
| 31 |
|
| 32 |
### Key Innovations
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
-
|
| 39 |
|
| 40 |
```
|
| 41 |
-
Input β
|
| 42 |
```
|
| 43 |
-
**Component Details:**
|
| 44 |
-
- **tf_col (Column Embedder)**: Set Transformer for statistical distribution learning across features
|
| 45 |
-
- **Bi-Axial Attention**: Replaces standard RowInteraction with alternating attention patterns:
|
| 46 |
-
- Standard Cross-Feature Attention
|
| 47 |
-
- Grouped Feature Attention
|
| 48 |
-
- Hierarchical Feature Attention
|
| 49 |
-
- Relational Feature Attention
|
| 50 |
-
- CLS Token Aggregation
|
| 51 |
-
- **tf_icl (ICL Predictor)**: In-context learning module for few-shot prediction
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
```python
|
| 57 |
-
from orion_bix.sklearn import OrionBiXClassifier
|
| 58 |
|
| 59 |
-
# Initialize and use
|
| 60 |
-
clf = OrionBiXClassifier()
|
| 61 |
-
clf.fit(X_train, y_train)
|
| 62 |
-
predictions = clf.predict(X_test)
|
| 63 |
```
|
| 64 |
-
|
| 65 |
-
|
| 66 |
|
| 67 |
## Installation
|
| 68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
### From the source
|
|
|
|
| 70 |
#### Option 1: From the local clone
|
| 71 |
|
| 72 |
```bash
|
|
@@ -75,19 +86,98 @@ pip install -e .
|
|
| 75 |
```
|
| 76 |
|
| 77 |
#### Option 2: From the Git Remote
|
| 78 |
-
|
| 79 |
```bash
|
| 80 |
pip install git+https://github.com/Lexsi-Labs/Orion-BiX.git
|
| 81 |
```
|
| 82 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
## Citation
|
| 84 |
|
| 85 |
-
If you use Orion-BiX in your research, please cite:
|
| 86 |
|
| 87 |
```bibtex
|
| 88 |
-
@
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
}
|
| 93 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
pipeline_tag: table-question-answering
|
| 4 |
---
|
| 5 |
|
| 6 |
+
This model is presented in the paper [Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning](https://huggingface.co/papers/2512.00181).
|
| 7 |
+
Authors: Mohamed Bouadi, Pratinav Seth, Aditya Tanna, Vinay Kumar Sankarapu
|
| 8 |
+
Project Page: https://www.lexsi.ai/
|
| 9 |
+
|
| 10 |
<div align="center">
|
| 11 |
<img src="logo.png" alt="Orion-BiX Logo" width="700"/>
|
| 12 |
</div>
|
|
|
|
| 15 |
<a href="https://lexsi.ai/">
|
| 16 |
<img src="https://img.shields.io/badge/Lexsi-Homepage-FF6B6B?style=for-the-badge" alt="Homepage"/>
|
| 17 |
</a>
|
| 18 |
+
<a href="https://huggingface.co/Lexsi/Orion-BiX">
|
| 19 |
<img src="https://img.shields.io/badge/π€%20Hugging%20Face-Lexsi AI-FFD21E?style=for-the-badge" alt="Hugging Face"/>
|
| 20 |
</a>
|
| 21 |
<a href="https://discord.gg/dSB62Q7A">
|
|
|
|
| 26 |
</a>
|
| 27 |
</div>
|
| 28 |
|
| 29 |
+
[](https://www.python.org/downloads/)
|
| 30 |
+
[](https://pytorch.org/)
|
| 31 |
+
[](https://opensource.org/licenses/MIT)
|
| 32 |
|
| 33 |
+
# Orion-BiX: Bi-Axial Meta-Learning for Tabular In-Context Learning
|
| 34 |
|
| 35 |
+
**[Orion-BiX](https://arxiv.org/abs/2512.00181)** is an advanced tabular foundation model that combines **Bi-Axial Attention** with **Meta-Learning** capabilities for few-shot tabular classification. The model extends the TabICL architecture with alternating attention patterns and episode-based training, achieving state-of-the-art performance on domain-specific benchamrks such as Healthcare and Finance.
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
## ποΈ Approach and Architecture
|
| 38 |
|
| 39 |
### Key Innovations
|
| 40 |
|
| 41 |
+
Orion-BiX introduces three key architectural innovations:
|
| 42 |
+
|
| 43 |
+
1. **Bi-Axial Attention**: Alternating attention patterns (Standard β Grouped β Hierarchical β Relational) that capture multi-scale feature interactions
|
| 44 |
+
2. **Meta-Learning**: Episode-based training with k-NN support selection for few-shot learning
|
| 45 |
+
3. **Configurable Architecture**: Flexible design supporting various attention mechanisms and training modes
|
| 46 |
+
4. **Production Ready**: Memory optimization, distributed training support, and scikit-learn interface
|
| 47 |
+
|
| 48 |
+
### Component Details
|
| 49 |
|
| 50 |
+
Orion-BiX follows a three-component architecture:
|
| 51 |
|
| 52 |
```
|
| 53 |
+
Input β Column Embedder (Set Transformer) β Bi-Axial Attention β ICL Predictor β Output
|
| 54 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
+
1. **Column Embedder**: Set Transformer for statistical distribution learning across features from TabICL
|
| 57 |
+
2. **Bi-Axial Attention**: Replaces standard RowInteraction with alternating attention patterns:
|
| 58 |
+
- **Standard Cross-Feature Attention**: Direct attention between features
|
| 59 |
+
- **Grouped Feature Attention**: Attention within feature groups
|
| 60 |
+
- **Hierarchical Feature Attention**: Hierarchical feature patterns
|
| 61 |
+
- **Relational Feature Attention**: Full feature-to-feature attention
|
| 62 |
+
- **CLS Token Aggregation**: Multiple CLS tokens (default: 4) for feature summarization
|
| 63 |
+
3. **tf_icl ICL Predictor**: In-context learning module for few-shot prediction
|
| 64 |
|
| 65 |
+
Each `BiAxialAttentionBlock` applies four attention patterns in sequence:
|
|
|
|
|
|
|
|
|
|
| 66 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
```
|
| 68 |
+
Standard β Grouped β Hierarchical β Relational β CLS Aggregation
|
| 69 |
+
```
|
| 70 |
|
| 71 |
## Installation
|
| 72 |
|
| 73 |
+
### Prerequisites
|
| 74 |
+
|
| 75 |
+
- Python 3.9-3.12
|
| 76 |
+
- PyTorch 2.2+ (with CUDA support recommended)
|
| 77 |
+
- CUDA-capable GPU (recommended for training)
|
| 78 |
+
|
| 79 |
### From the source
|
| 80 |
+
|
| 81 |
#### Option 1: From the local clone
|
| 82 |
|
| 83 |
```bash
|
|
|
|
| 86 |
```
|
| 87 |
|
| 88 |
#### Option 2: From the Git Remote
|
|
|
|
| 89 |
```bash
|
| 90 |
pip install git+https://github.com/Lexsi-Labs/Orion-BiX.git
|
| 91 |
```
|
| 92 |
|
| 93 |
+
## Usage
|
| 94 |
+
|
| 95 |
+
Orion-BiX provides a scikit-learn compatible interface for easy integration:
|
| 96 |
+
|
| 97 |
+
```python
|
| 98 |
+
from orion_bix.sklearn import OrionBixClassifier
|
| 99 |
+
|
| 100 |
+
# Initialize and fit the classifier
|
| 101 |
+
clf = OrionBixClassifier()
|
| 102 |
+
|
| 103 |
+
# Fit the model (prepares data transformations)
|
| 104 |
+
clf.fit(X_train, y_train)
|
| 105 |
+
|
| 106 |
+
# Make predictions
|
| 107 |
+
predictions = clf.predict(X_test)
|
| 108 |
+
probabilities = clf.predict_proba(X_test)
|
| 109 |
+
```
|
| 110 |
+
|
| 111 |
+
## Preprocessing
|
| 112 |
+
|
| 113 |
+
Orion-BiX includes automatic preprocessing that handles:
|
| 114 |
+
|
| 115 |
+
1. **Categorical Encoding**: Automatically encodes categorical features using ordinal encoding
|
| 116 |
+
2. **Missing Value Imputation**: Handles missing values using median imputation for numerical features
|
| 117 |
+
3. **Feature Normalization**: Supports multiple normalization methods:
|
| 118 |
+
- `"none"`: No normalization
|
| 119 |
+
- `"power"`: Yeo-Johnson power transform
|
| 120 |
+
- `"quantile"`: Quantile transformation to normal distribution
|
| 121 |
+
- `"quantile_rtdl"`: RTDL-style quantile transform
|
| 122 |
+
- `"robust"`: Robust scaling using median and quantiles
|
| 123 |
+
4. **Outlier Handling**: Clips outliers beyond a specified Z-score threshold (default: 4.0)
|
| 124 |
+
5. **Feature Permutation**: Applies systematic feature shuffling for ensemble diversity:
|
| 125 |
+
- `"none"`: Original feature order
|
| 126 |
+
- `"shift"`: Circular shifting
|
| 127 |
+
- `"random"`: Random permutation
|
| 128 |
+
- `"latin"`: Latin square patterns (recommended)
|
| 129 |
+
|
| 130 |
+
The preprocessing is automatically applied during `fit()` and `predict()`, so no manual preprocessing is required.
|
| 131 |
+
|
| 132 |
+
## Performance
|
| 133 |
+
|
| 134 |
+
<div align="center">
|
| 135 |
+
<img src="figures/accuracy_ranking_talent.png" alt="Accuracy Ranking TALENT" width="700"/>
|
| 136 |
+
</div>
|
| 137 |
+
|
| 138 |
+
<div align="center">
|
| 139 |
+
<img src="figures/accuracy_ranking_tabzilla.png" alt="Accuracy Ranking TabZilla" width="700"/>
|
| 140 |
+
</div>
|
| 141 |
+
|
| 142 |
+
<div align="center">
|
| 143 |
+
<img src="figures/accuracy_ranking_openml-cc18.png" alt="Accuracy Ranking OPENML-CC18" width="700"/>
|
| 144 |
+
</div>
|
| 145 |
+
|
| 146 |
+
<div align="center">
|
| 147 |
+
<table>
|
| 148 |
+
<tr>
|
| 149 |
+
<td style="padding: 5px;"><img src="figures/relative_acc_improvement_over_tabzilla.png" alt="Relative Improvement over XGBoost on TabZilla" width="700"/></td>
|
| 150 |
+
</tr>
|
| 151 |
+
</table>
|
| 152 |
+
</div>
|
| 153 |
+
|
| 154 |
## Citation
|
| 155 |
|
| 156 |
+
If you use Orion-BiX in your research, please cite our [paper](https://arxiv.org/abs/2512.00181):
|
| 157 |
|
| 158 |
```bibtex
|
| 159 |
+
@article{bouadi2025orionbix,
|
| 160 |
+
title={Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning},
|
| 161 |
+
author={Mohamed Bouadi and Pratinav Seth and Aditya Tanna and Vinay Kumar Sankarapu},
|
| 162 |
+
year={2025},
|
| 163 |
+
eprint={2512.00181},
|
| 164 |
+
archivePrefix={arXiv},
|
| 165 |
+
primaryClass={cs.LG},
|
| 166 |
+
url={https://arxiv.org/abs/2512.00181},
|
| 167 |
}
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
## License
|
| 171 |
+
|
| 172 |
+
This project is released under the MIT License. See [LICENSE](LICENSE) for details.
|
| 173 |
+
|
| 174 |
+
## Contact
|
| 175 |
+
|
| 176 |
+
For questions, issues, or contributions, please:
|
| 177 |
+
- Open an issue on [GitHub](https://github.com/Lexsi-Labs/Orion-BiX/issues)
|
| 178 |
+
- Join our [Discord](https://discord.gg/dSB62Q7A) community
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
## π Acknowledgments
|
| 182 |
+
|
| 183 |
+
Orion-BiX is built on top of [TabICL](https://github.com/soda-inria/tabicl), a tabular foundation model for in-context learning. We gratefully acknowledge the TabICL authors for their foundational work and for making their codebase publicly available.
|