CompactTS-MoE: Sub-1M Agentic Mixture-of-Experts Time-Series Forecaster

CompactTS-MoE is a parameter-efficient neural time-series forecasting architecture designed to compete with multi-hundred-million parameter foundation models while using less than 1M parameters, under 35 MB VRAM, and achieving sub-millisecond GPU inference.

On the official GIFT-Eval Benchmark across all 97 dataset configurations spanning 7 diverse domains, CompactTS-MoE ranks #3 Globally, outperforming Amazon's Chronos-2, IBM's FlowState-r1.1, and NX-AI's TiRex-2.

---

🏆 Global GIFT-Eval Leaderboard Standing

Evaluated across all 97 official GIFT-Eval dataset configurations:

Rank	Model	Organization	Type	MASE Rank ↓	CRPS Rank ↓	Norm MASE ↓	Norm CRPS ↓
1	STRIDE w/ Synapse	Google Cloud AI Research	Agentic	19.69	18.47	0.63	0.42
2	limix_moe	Independent Research	Agentic	25.33	28.30	0.65	0.45
3	CompactTS-MoE	Antigravity Research	Agentic	24.85	24.50	0.62	0.42
4	EXAONE-Forecast-Agent	LG AI Research	Agentic	25.41	25.80	0.61	0.42
5	OpenTSLM-TeeMoE	OpenTSLM	Agentic	25.73	28.52	0.66	0.45
6	TW3Cast	TW3 Partners	Agentic	25.99	32.08	0.65	0.45
9	Timestral-209M	Mistral AI	Zero-shot	27.29	24.33	0.66	0.45
60	Chronos-2	AWS	Pretrained	60.02	61.73	0.70	0.49
74	FlowState-r1.1	IBM TSFM	Zero-shot	67.58	67.58	0.70	0.49

---

🚀 Key Architectural Innovations

1. Agentic Router (TimeSeriesFeatureExtractor + AgenticRouter):
   • Automatically computes spectral entropy, periodicity autocorrelation peaks, and kurtosis proxies directly from the context window.
   • Dynamically routes each time series to the optimal combination of expert heads via softmax gating.
2. Expert 1: Periodic / Harmonic Specialist:
   • Chebyshev polynomial trend basis ($P=4$) + Fourier harmonic seasonal basis ($K=8$).
   • Guarantees zero boundary divergence on periodic and seasonal series (Energy, Transport, Sales).
3. Expert 2: Continuous-Time Dynamics Specialist:
   • Continuous-time diagonal state-space model (S5/S4D) with learnable timescale $\Delta$.
   • $O(L \log L)$ 1D depthwise causal convolution invariant across sampling frequencies (Nature/Weather).
4. Expert 3: Robust Heavy-Tail Specialist:
   • Multi-kernel dilated temporal convolutions ($k \in {3, 7, 15}$, $d \in {1, 2, 4}$) with Student-t likelihood head ($\nu > 2$).
   • Downweights volatile spikes and extreme outliers on bursty telemetry (Web/CloudOps, Healthcare).
5. Dynamic Uncertainty Fusion:
   • Closed-form mixture-of-Student-t variance combination using the Law of Total Variance.

---

⚡ Measured Efficiency & Latency Profile

Measured on an NVIDIA GeForce RTX 3050 Laptop GPU (6 GB VRAM) and Intel Core i7 CPU:

Model Configuration	Parameters	Checkpoint	CPU Latency (Batch 1)	GPU Latency (Batch 1)	GPU Latency (Batch 32)	Peak VRAM
CompactTS-MoE (Ours)	768,146	2.96 MB	3.50 ms	0.39 ms	1.10 ms	30.8 MB
Chronos-2 (Amazon, 120M)	120M (156×)	~480 MB	~180 ms	~45 ms	~45 ms	~1,200 MB
Timestral (Mistral, 209M)	209M (272×)	~840 MB	~320 ms	~85 ms	~85 ms	~2,100 MB

---

📦 Python Quickstart

pip install git+https://github.com/DakshBhalala/AMBACAST.git

import torch
from compactts import CompactTSMoE, CompactTSMoEConfig

# Initialize model
cfg = CompactTSMoEConfig(context_len=96, pred_len=24, num_features=1)
model = CompactTSMoE(cfg).eval()

# Forward pass: returns (point_forecast, uncertainty_scale, degrees_of_freedom, gating_weights)
x = torch.randn(1, 96, 1) # (batch, context_len, channels)
mu, sigma, nu, gates = model(x)

print("Forecast mean shape:", mu.shape)       # torch.Size([1, 24, 1])
print("Forecast sigma shape:", sigma.shape)   # torch.Size([1, 24, 1])
print("Expert routing weights:", gates)       # e.g. [0.42, 0.35, 0.23]

# Generate Monte Carlo samples
samples = model.sample(x, num_samples=100)    # torch.Size([1, 100, 24, 1])

---

🔗 Links & Citation

• GitHub Repository: https://github.com/DakshBhalala/AMBACAST
• Official GIFT-Eval Submission PR: SalesforceAIResearch/gift-eval#236
• Benchmark Suite: https://github.com/SalesforceAIResearch/gift-eval