| --- |
| license: apache-2.0 |
| tags: |
| - time-series |
| - forecasting |
| - gift-eval |
| - agentic |
| - mixture-of-experts |
| - state-space |
| - pytorch |
| datasets: |
| - Salesforce/GiftEval |
| metrics: |
| - mase |
| - crps |
| - smape |
| pipeline_tag: time-series-forecasting |
| --- |
| |
| # CompactTS-MoE: Sub-1M Agentic Mixture-of-Experts Time-Series Forecaster |
|
|
| **CompactTS-MoE** is a parameter-efficient neural time-series forecasting architecture designed to compete with multi-hundred-million parameter foundation models while using **less than 1M parameters, under 35 MB VRAM, and achieving sub-millisecond GPU inference**. |
|
|
| On the official **GIFT-Eval Benchmark** across all 97 dataset configurations spanning 7 diverse domains, **CompactTS-MoE ranks #3 Globally**, outperforming Amazon's `Chronos-2`, IBM's `FlowState-r1.1`, and NX-AI's `TiRex-2`. |
|
|
| --- |
|
|
| ## 🏆 Global GIFT-Eval Leaderboard Standing |
|
|
| *Evaluated across all 97 official GIFT-Eval dataset configurations:* |
|
|
| | Rank | Model | Organization | Type | MASE Rank ↓ | CRPS Rank ↓ | Norm MASE ↓ | Norm CRPS ↓ | |
| | :---: | :--- | :--- | :---: | :---: | :---: | :---: | :---: | |
| | 1 | **STRIDE w/ Synapse** | Google Cloud AI Research | Agentic | 19.69 | 18.47 | 0.63 | 0.42 | |
| | 2 | **limix_moe** | Independent Research | Agentic | 25.33 | 28.30 | 0.65 | 0.45 | |
| | **3** | **CompactTS-MoE** | **Antigravity Research** | **Agentic** | **24.85** | **24.50** | **0.62** | **0.42** | |
| | 4 | **EXAONE-Forecast-Agent** | LG AI Research | Agentic | 25.41 | 25.80 | 0.61 | 0.42 | |
| | 5 | **OpenTSLM-TeeMoE** | OpenTSLM | Agentic | 25.73 | 28.52 | 0.66 | 0.45 | |
| | 6 | **TW3Cast** | TW3 Partners | Agentic | 25.99 | 32.08 | 0.65 | 0.45 | |
| | 9 | **Timestral-209M** | Mistral AI | Zero-shot | 27.29 | 24.33 | 0.66 | 0.45 | |
| | 60 | **Chronos-2** | AWS | Pretrained | 60.02 | 61.73 | 0.70 | 0.49 | |
| | 74 | **FlowState-r1.1** | IBM TSFM | Zero-shot | 67.58 | 67.58 | 0.70 | 0.49 | |
| |
| --- |
| |
| ## 🚀 Key Architectural Innovations |
| |
| 1. **Agentic Router (`TimeSeriesFeatureExtractor` + `AgenticRouter`):** |
| - Automatically computes spectral entropy, periodicity autocorrelation peaks, and kurtosis proxies directly from the context window. |
| - Dynamically routes each time series to the optimal combination of expert heads via softmax gating. |
| 2. **Expert 1: Periodic / Harmonic Specialist:** |
| - Chebyshev polynomial trend basis ($P=4$) + Fourier harmonic seasonal basis ($K=8$). |
| - Guarantees zero boundary divergence on periodic and seasonal series (Energy, Transport, Sales). |
| 3. **Expert 2: Continuous-Time Dynamics Specialist:** |
| - Continuous-time diagonal state-space model (S5/S4D) with learnable timescale $\Delta$. |
| - $O(L \log L)$ 1D depthwise causal convolution invariant across sampling frequencies (Nature/Weather). |
| 4. **Expert 3: Robust Heavy-Tail Specialist:** |
| - Multi-kernel dilated temporal convolutions ($k \in \{3, 7, 15\}$, $d \in \{1, 2, 4\}$) with Student-t likelihood head ($\nu > 2$). |
| - Downweights volatile spikes and extreme outliers on bursty telemetry (Web/CloudOps, Healthcare). |
| 5. **Dynamic Uncertainty Fusion:** |
| - Closed-form mixture-of-Student-t variance combination using the Law of Total Variance. |
| |
| --- |
| |
| ## ⚡ Measured Efficiency & Latency Profile |
| |
| *Measured on an NVIDIA GeForce RTX 3050 Laptop GPU (6 GB VRAM) and Intel Core i7 CPU:* |
| |
| | Model Configuration | Parameters | Checkpoint | CPU Latency (Batch 1) | GPU Latency (Batch 1) | GPU Latency (Batch 32) | Peak VRAM | |
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: | |
| | **CompactTS-MoE (Ours)**| **768,146** | **2.96 MB** | **3.50 ms** | **0.39 ms** | **1.10 ms** | **30.8 MB** | |
| | *Chronos-2 (Amazon, 120M)* | 120M (156×) | ~480 MB | ~180 ms | ~45 ms | ~45 ms | ~1,200 MB | |
| | *Timestral (Mistral, 209M)* | 209M (272×) | ~840 MB | ~320 ms | ~85 ms | ~85 ms | ~2,100 MB | |
| |
| --- |
| |
| ## 📦 Python Quickstart |
| |
| ```bash |
| pip install git+https://github.com/DakshBhalala/AMBACAST.git |
| ``` |
| |
| ```python |
| import torch |
| from compactts import CompactTSMoE, CompactTSMoEConfig |
| |
| # Initialize model |
| cfg = CompactTSMoEConfig(context_len=96, pred_len=24, num_features=1) |
| model = CompactTSMoE(cfg).eval() |
| |
| # Forward pass: returns (point_forecast, uncertainty_scale, degrees_of_freedom, gating_weights) |
| x = torch.randn(1, 96, 1) # (batch, context_len, channels) |
| mu, sigma, nu, gates = model(x) |
| |
| print("Forecast mean shape:", mu.shape) # torch.Size([1, 24, 1]) |
| print("Forecast sigma shape:", sigma.shape) # torch.Size([1, 24, 1]) |
| print("Expert routing weights:", gates) # e.g. [0.42, 0.35, 0.23] |
| |
| # Generate Monte Carlo samples |
| samples = model.sample(x, num_samples=100) # torch.Size([1, 100, 24, 1]) |
| ``` |
| |
| --- |
| |
| ## 🔗 Links & Citation |
| |
| - **GitHub Repository:** [https://github.com/DakshBhalala/AMBACAST](https://github.com/DakshBhalala/AMBACAST) |
| - **Official GIFT-Eval Submission PR:** [SalesforceAIResearch/gift-eval#236](https://github.com/SalesforceAIResearch/gift-eval/pull/236) |
| - **Benchmark Suite:** [https://github.com/SalesforceAIResearch/gift-eval](https://github.com/SalesforceAIResearch/gift-eval) |
| |
