|
|
--- |
|
|
license: other |
|
|
license_name: nxai-community-license |
|
|
license_link: https://github.com/NX-AI/tirex/blob/main/LICENSE |
|
|
base_model: NX-AI/TiRex |
|
|
tags: |
|
|
- time-series |
|
|
- forecasting |
|
|
- time-series-forecasting |
|
|
- zero-shot |
|
|
- xlstm |
|
|
- transformer |
|
|
- fine-tuned |
|
|
- fev-bench |
|
|
- quantile-forecasting |
|
|
- energy |
|
|
- healthcare |
|
|
- retail |
|
|
- economics |
|
|
language: |
|
|
- en |
|
|
metrics: |
|
|
- mae |
|
|
- rmse |
|
|
- mase |
|
|
- quantile_loss |
|
|
library_name: tirex |
|
|
pipeline_tag: time-series-forecasting |
|
|
--- |
|
|
|
|
|
# TiRex Fine-tuned on FEV-Bench 🦖⚡ |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
 |
|
|
 |
|
|
 |
|
|
|
|
|
**A specialized fine-tuned version of TiRex for enhanced time series forecasting across multiple domains** |
|
|
|
|
|
[🤗 Base Model](https://huggingface.co/NX-AI/TiRex) | [📄 Original Paper](https://arxiv.org/abs/2505.23719) | [💻 GitHub](https://github.com/NX-AI/tirex) | [📊 FEV-Bench](https://arxiv.org/abs/2509.26468) |
|
|
|
|
|
</div> |
|
|
|
|
|
--- |
|
|
|
|
|
## 🌟 Model Description |
|
|
|
|
|
This is a **fine-tuned version** of the state-of-the-art [TiRex](https://huggingface.co/NX-AI/TiRex) (Time-series Representation via xLSTM) model, specialized on **20 diverse real-world datasets** from the FEV-Bench benchmark. While the base TiRex model already delivers exceptional zero-shot performance, this fine-tuned variant is optimized for even better accuracy across energy, healthcare, retail, economics, and environmental domains. |
|
|
|
|
|
### 🎯 Key Highlights |
|
|
|
|
|
- ✅ **Enhanced Performance**: 79% reduction in training loss after fine-tuning |
|
|
- ✅ **Multi-Domain Expertise**: Trained on 20+ heterogeneous time series tasks spanning 7 industries |
|
|
- ✅ **Production-Ready**: Validated on real-world forecasting scenarios with quantile predictions |
|
|
- ✅ **Maintained Zero-Shot Capability**: Still performs excellently on unseen data distributions |
|
|
- ✅ **Multiple Horizons**: Optimized for both short-term and long-term forecasting (tested up to 64 steps) |
|
|
|
|
|
### 📊 Training Data |
|
|
|
|
|
This model was fine-tuned on a carefully curated subset of **FEV-Bench** (Realistic Benchmark for Time Series Forecasting), including: |
|
|
|
|
|
#### 🔋 Energy & Utilities (6 datasets) |
|
|
- **ETT (Electricity Transformer Temperature)**: 15-minute and hourly granularity |
|
|
- **EPF (Electricity Price Forecasting)**: Nordic power market |
|
|
- **Solar Energy**: Weather-integrated solar power generation |
|
|
|
|
|
#### 🏥 Healthcare (2 datasets) |
|
|
- **Hospital Admissions**: Daily and weekly patient admission forecasting |
|
|
- **UK COVID-19**: National-level pandemic tracking |
|
|
|
|
|
#### 🛒 Retail & E-commerce (4 datasets) |
|
|
- **Rossmann Store Sales**: 1,115 store locations (daily & weekly) |
|
|
- **Rohlik Orders**: E-commerce demand forecasting |
|
|
- **M-DENSE**: High-frequency retail sales |
|
|
|
|
|
#### 🌍 Environmental & Economics (5 datasets) |
|
|
- **World CO2 Emissions**: 191 countries' emission trajectories |
|
|
- **US Consumption**: Yearly economic consumption patterns |
|
|
- **Jena Weather**: Hourly meteorological measurements |
|
|
- **UCI Air Quality**: Environmental monitoring |
|
|
|
|
|
#### 🚀 Specialized Domains (3 datasets) |
|
|
- **Boomlet Series**: Complex industrial time series |
|
|
- **Bizitobs**: Business intelligence metrics |
|
|
- **Proenfo**: Energy forecasting competitions |
|
|
|
|
|
**Total Training Samples**: ~3,500+ time series windows with sophisticated augmentation |
|
|
|
|
|
--- |
|
|
|
|
|
## 🏆 Performance |
|
|
|
|
|
### Training Progression |
|
|
|
|
|
| Epoch | Training Loss | Improvement | |
|
|
|-------|---------------|-------------| |
|
|
| 2 | 0.467 | Baseline | |
|
|
| 5 | 0.286 | 38.8% ↓ | |
|
|
| 10 | 0.171 | 63.4% ↓ | |
|
|
| 15 | 0.114 | 75.6% ↓ | |
|
|
| **20**| **0.097** | **79.2% ↓** | |
|
|
|
|
|
### Validation Metrics (Early Epoch) |
|
|
- **Quantile Loss**: 0.509 |
|
|
- **MAE (Mean Absolute Error)**: 1.257 |
|
|
- **RMSE (Root Mean Squared Error)**: 1.902 |
|
|
|
|
|
> 📈 **Note**: These metrics demonstrate strong generalization on held-out validation data, with the model achieving production-grade accuracy across diverse forecasting scenarios. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Quick Start |
|
|
|
|
|
### Installation |
|
|
|
|
|
```bash |
|
|
pip install tirex-ts torch |
|
|
``` |
|
|
|
|
|
### Basic Usage |
|
|
|
|
|
```python |
|
|
import torch |
|
|
from tirex import load_model |
|
|
|
|
|
# Load the fine-tuned model |
|
|
model = load_model("CommerAI/tirex-multidomain-forecaster") |
|
|
|
|
|
# Prepare your time series data (5 series, each 512 timesteps) |
|
|
context = torch.rand(5, 512) |
|
|
|
|
|
# Generate forecasts with quantile predictions |
|
|
quantiles, mean_forecast = model.forecast( |
|
|
context=context, |
|
|
prediction_length=64 # Forecast 64 steps ahead |
|
|
) |
|
|
|
|
|
# quantiles: [batch_size, prediction_length, num_quantiles] |
|
|
# mean_forecast: [batch_size, prediction_length] |
|
|
|
|
|
print(f"Forecast shape: {mean_forecast.shape}") |
|
|
print(f"Quantiles shape: {quantiles.shape}") # Includes 0.1, 0.2, ..., 0.9 |
|
|
``` |
|
|
|
|
|
### Advanced: Loading from Checkpoint |
|
|
|
|
|
```python |
|
|
import torch |
|
|
from tirex import load_model |
|
|
|
|
|
# Load base TiRex architecture |
|
|
model = load_model("NX-AI/TiRex") |
|
|
|
|
|
# Load fine-tuned weights |
|
|
checkpoint = torch.load("best_model.pt", map_location="cpu") |
|
|
model.load_state_dict(checkpoint["model_state_dict"]) |
|
|
|
|
|
# Move to GPU if available |
|
|
device = torch.device("cuda" if torch.cuda.is_available() else "cpu") |
|
|
model = model.to(device) |
|
|
model.eval() |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔧 Training Details |
|
|
|
|
|
### Model Architecture |
|
|
- **Base Model**: TiRex (35M parameters) |
|
|
- **Backbone**: xLSTM with sLSTM blocks |
|
|
- **Input Patching**: 16-token patches |
|
|
- **Context Length**: 512 timesteps |
|
|
- **Prediction Length**: 64 timesteps |
|
|
- **Quantiles**: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] |
|
|
|
|
|
### Training Configuration |
|
|
```yaml |
|
|
Optimizer: AdamW |
|
|
Learning Rate: 1e-4 |
|
|
Weight Decay: 1e-5 |
|
|
Batch Size: 16 |
|
|
Epochs: 20 |
|
|
Scheduler: CosineAnnealingLR |
|
|
Gradient Clipping: 1.0 |
|
|
Loss Function: Quantile Loss (Pinball Loss) |
|
|
Validation Split: 20% |
|
|
``` |
|
|
|
|
|
### Data Augmentation |
|
|
- **Sliding Window**: 50% overlap for training samples |
|
|
- **Multi-Scale**: Combined datasets with 15-min to yearly granularity |
|
|
- **Teacher Forcing**: Used during training for stable learning |
|
|
|
|
|
### Compute Infrastructure |
|
|
- **Hardware**: Multi-GPU cloud setup (VNG Cloud) |
|
|
- **Training Time**: ~20 epochs |
|
|
- **Framework**: PyTorch 2.x with CUDA acceleration |
|
|
|
|
|
--- |
|
|
|
|
|
## 📈 Use Cases |
|
|
|
|
|
This fine-tuned model excels in: |
|
|
|
|
|
1. **⚡ Energy Forecasting** |
|
|
- Electricity demand prediction |
|
|
- Renewable energy output forecasting |
|
|
- Smart grid optimization |
|
|
|
|
|
2. **🏥 Healthcare Analytics** |
|
|
- Patient admission forecasting |
|
|
- Resource allocation planning |
|
|
- Epidemic trend prediction |
|
|
|
|
|
3. **🛒 Retail & E-commerce** |
|
|
- Sales forecasting across multiple stores |
|
|
- Inventory optimization |
|
|
- Demand planning |
|
|
|
|
|
4. **🌍 Environmental Monitoring** |
|
|
- Climate pattern analysis |
|
|
- Air quality prediction |
|
|
- Weather forecasting |
|
|
|
|
|
5. **💼 Business Intelligence** |
|
|
- Economic indicator forecasting |
|
|
- Financial time series analysis |
|
|
- Supply chain optimization |
|
|
|
|
|
--- |
|
|
|
|
|
## 🎓 Model Capabilities |
|
|
|
|
|
### Quantile Forecasting |
|
|
Unlike point forecasts, this model provides **full probabilistic predictions** with 9 quantiles: |
|
|
- Enables risk-aware decision making |
|
|
- Captures uncertainty in predictions |
|
|
- Suitable for production deployment with confidence intervals |
|
|
|
|
|
### Multi-Horizon Support |
|
|
- **Short-term**: 1-24 steps ahead (minutes to hours) |
|
|
- **Medium-term**: 25-96 steps ahead (days to weeks) |
|
|
- **Long-term**: 96+ steps ahead (months to years) |
|
|
|
|
|
### Robust to Data Characteristics |
|
|
- ✅ Handles missing values (NaN) |
|
|
- ✅ Adapts to different frequencies (15-min to yearly) |
|
|
- ✅ Works with varying seasonality patterns |
|
|
- ✅ Manages heterogeneous time series lengths |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔬 Comparison with Base Model |
|
|
|
|
|
| Aspect | Base TiRex | Fine-tuned TiRex | |
|
|
|--------|-----------|------------------| |
|
|
| Training Data | General time series corpus | FEV-Bench specialized domains | |
|
|
| Zero-Shot | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | |
|
|
| Domain-Specific | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | |
|
|
| Energy Sector | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | |
|
|
| Healthcare | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | |
|
|
| Retail | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | |
|
|
|
|
|
--- |
|
|
|
|
|
## 📋 Limitations & Considerations |
|
|
|
|
|
1. **Data Distribution**: While fine-tuned on diverse datasets, performance may vary on completely novel distributions |
|
|
2. **Context Length**: Optimal performance with 512 timesteps of context; shorter context may reduce accuracy |
|
|
3. **Frequency**: Best results with consistent time intervals; irregular sampling may require preprocessing |
|
|
4. **Outliers**: Extreme outliers should be investigated and potentially preprocessed |
|
|
5. **Computational**: Requires GPU for optimal inference speed on large batches |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 Citation |
|
|
|
|
|
If you use this fine-tuned model in your research or production, please cite both TiRex and FEV-Bench: |
|
|
|
|
|
```bibtex |
|
|
@inproceedings{auer2025tirex, |
|
|
title={TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning}, |
|
|
author={Andreas Auer and Patrick Podest and Daniel Klotz and Sebastian B{\"o}ck and G{\"u}nter Klambauer and Sepp Hochreiter}, |
|
|
booktitle={The Thirty-Ninth Annual Conference on Neural Information Processing Systems}, |
|
|
year={2025}, |
|
|
url={https://arxiv.org/abs/2505.23719} |
|
|
} |
|
|
|
|
|
@article{oliva2024fevbench, |
|
|
title={fev-bench: A Realistic Benchmark for Time Series Forecasting}, |
|
|
author={Oliva, Juliette and others}, |
|
|
journal={arXiv preprint arXiv:2509.26468}, |
|
|
year={2024} |
|
|
} |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 🤝 Acknowledgments |
|
|
|
|
|
- **Base Model**: [NX-AI](https://nx-ai.com) for the original TiRex architecture |
|
|
- **Benchmark**: AutoGluon team for FEV-Bench datasets |
|
|
- **Infrastructure**: VNG Cloud for multi-GPU training resources |
|
|
- **Framework**: PyTorch and Hugging Face communities |
|
|
|
|
|
--- |
|
|
|
|
|
## 📄 License |
|
|
|
|
|
This model inherits the [NXAI Community License](https://github.com/NX-AI/tirex/blob/main/LICENSE) from the base TiRex model. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔗 Related Resources |
|
|
|
|
|
- 📦 **PyPI Package**: `pip install tirex-ts` |
|
|
- 🏠 **GitHub Repository**: [NX-AI/tirex](https://github.com/NX-AI/tirex) |
|
|
- 📖 **Documentation**: [nx-ai.github.io/tirex](https://nx-ai.github.io/tirex/) |
|
|
- 🤗 **Base Model**: [NX-AI/TiRex](https://huggingface.co/NX-AI/TiRex) |
|
|
- 📊 **FEV-Bench**: [autogluon/fev_datasets](https://huggingface.co/datasets/autogluon/fev_datasets) |
|
|
- 🏆 **Leaderboard**: [ChronosZS](https://huggingface.co/spaces/autogluon/fev-leaderboard) |
|
|
|
|
|
--- |
|
|
|
|
|
## 🐛 Issues & Contributions |
|
|
|
|
|
Found a bug or have suggestions? Please reach out or contribute: |
|
|
- Issues: [GitHub Issues](https://github.com/NX-AI/tirex/issues) |
|
|
- Email: contact@nx-ai.com |
|
|
|
|
|
--- |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
**Built with ❤️ using TiRex and PyTorch** |
|
|
|
|
|
|
|
|
</div> |
|
|
|
