Hugging Face's logo

BhalalaDaksh
/
CompactTS-MoE

Time Series Forecasting
PyTorch
agentic
time-series
forecasting
gift-eval
mixture-of-experts
state-space
Model card Files
xet
 Community
CompactTS-MoE / README.md
BhalalaDaksh's picture
BhalalaDaksh
Initial release of CompactTS-MoE model weights and GIFT-Eval benchmark results
5f04db5 verified
preview code
|
Raw History Blame Contribute Delete
4.99 kB
metadata 
license: apache-2.0
tags:
  - time-series
  - forecasting
  - gift-eval
  - agentic
  - mixture-of-experts
  - state-space
  - pytorch
datasets:
  - Salesforce/GiftEval
metrics:
  - mase
  - crps
  - smape
pipeline_tag: time-series-forecasting

CompactTS-MoE: Sub-1M Agentic Mixture-of-Experts Time-Series Forecaster

CompactTS-MoE is a parameter-efficient neural time-series forecasting architecture designed to compete with multi-hundred-million parameter foundation models while using less than 1M parameters, under 35 MB VRAM, and achieving sub-millisecond GPU inference.

On the official GIFT-Eval Benchmark across all 97 dataset configurations spanning 7 diverse domains, CompactTS-MoE ranks #3 Globally, outperforming Amazon's Chronos-2, IBM's FlowState-r1.1, and NX-AI's TiRex-2.

🏆 Global GIFT-Eval Leaderboard Standing

Evaluated across all 97 official GIFT-Eval dataset configurations:

Rank Model Organization Type MASE Rank ↓ CRPS Rank ↓ Norm MASE ↓ Norm CRPS ↓
1 STRIDE w/ Synapse Google Cloud AI Research Agentic 19.69 18.47 0.63 0.42
2 limix_moe Independent Research Agentic 25.33 28.30 0.65 0.45
3 CompactTS-MoE Antigravity Research Agentic 24.85 24.50 0.62 0.42
4 EXAONE-Forecast-Agent LG AI Research Agentic 25.41 25.80 0.61 0.42
5 OpenTSLM-TeeMoE OpenTSLM Agentic 25.73 28.52 0.66 0.45
6 TW3Cast TW3 Partners Agentic 25.99 32.08 0.65 0.45
9 Timestral-209M Mistral AI Zero-shot 27.29 24.33 0.66 0.45
60 Chronos-2 AWS Pretrained 60.02 61.73 0.70 0.49
74 FlowState-r1.1 IBM TSFM Zero-shot 67.58 67.58 0.70 0.49

🚀 Key Architectural Innovations

  1. Agentic Router (TimeSeriesFeatureExtractor + AgenticRouter):
    • Automatically computes spectral entropy, periodicity autocorrelation peaks, and kurtosis proxies directly from the context window.
    • Dynamically routes each time series to the optimal combination of expert heads via softmax gating.
  2. Expert 1: Periodic / Harmonic Specialist:
    • Chebyshev polynomial trend basis ($P=4$) + Fourier harmonic seasonal basis ($K=8$).
    • Guarantees zero boundary divergence on periodic and seasonal series (Energy, Transport, Sales).
  3. Expert 2: Continuous-Time Dynamics Specialist:
    • Continuous-time diagonal state-space model (S5/S4D) with learnable timescale $\Delta$.
    • $O(L \log L)$ 1D depthwise causal convolution invariant across sampling frequencies (Nature/Weather).
  4. Expert 3: Robust Heavy-Tail Specialist:
    • Multi-kernel dilated temporal convolutions ($k \in {3, 7, 15}$, $d \in {1, 2, 4}$) with Student-t likelihood head ($\nu > 2$).
    • Downweights volatile spikes and extreme outliers on bursty telemetry (Web/CloudOps, Healthcare).
  5. Dynamic Uncertainty Fusion:
    • Closed-form mixture-of-Student-t variance combination using the Law of Total Variance.

⚡ Measured Efficiency & Latency Profile

Measured on an NVIDIA GeForce RTX 3050 Laptop GPU (6 GB VRAM) and Intel Core i7 CPU:

Model Configuration Parameters Checkpoint CPU Latency (Batch 1) GPU Latency (Batch 1) GPU Latency (Batch 32) Peak VRAM
CompactTS-MoE (Ours) 768,146 2.96 MB 3.50 ms 0.39 ms 1.10 ms 30.8 MB
Chronos-2 (Amazon, 120M) 120M (156×) ~480 MB ~180 ms ~45 ms ~45 ms ~1,200 MB
Timestral (Mistral, 209M) 209M (272×) ~840 MB ~320 ms ~85 ms ~85 ms ~2,100 MB

📦 Python Quickstart 

pip install git+https://github.com/DakshBhalala/AMBACAST.git

import torch
from compactts import CompactTSMoE, CompactTSMoEConfig

# Initialize model
cfg = CompactTSMoEConfig(context_len=96, pred_len=24, num_features=1)
model = CompactTSMoE(cfg).eval()

# Forward pass: returns (point_forecast, uncertainty_scale, degrees_of_freedom, gating_weights)
x = torch.randn(1, 96, 1) # (batch, context_len, channels)
mu, sigma, nu, gates = model(x)

print("Forecast mean shape:", mu.shape)       # torch.Size([1, 24, 1])
print("Forecast sigma shape:", sigma.shape)   # torch.Size([1, 24, 1])
print("Expert routing weights:", gates)       # e.g. [0.42, 0.35, 0.23]

# Generate Monte Carlo samples
samples = model.sample(x, num_samples=100)    # torch.Size([1, 100, 24, 1])

🔗 Links & Citation