README.md

---
license: apache-2.0
library_name: pytorch
pipeline_tag: time-series-forecasting
tags:
  - time-series
  - forecasting
  - time-series-forecasting
  - knowledge-distillation
  - foundation-model
  - probabilistic-forecasting
  - gift-eval
---

# TimeTron-v2-33M

A **33.5M-parameter** time-series foundation model, distilled from **Chronos-2** by
**latent-space knowledge distillation** — the student matches the teacher's internal
representations rather than its forecasts.

**GIFT-Eval: 0.8794 normalized MASE / 0.6122 normalized CRPS** across all 97 configurations
(official Salesforce harness). On probabilistic accuracy it beats models up to **21× larger**.

| model | params | nMASE | nCRPS |
|---|---|---|---|
| **TimeTron-v2-33M** | **33.5M** | **0.8794** | **0.6122** |
| Chronos-Small | 46M | 0.8917 | 0.6630 |
| Moirai-Base | 91M | 0.9010 | 0.6095 |
| Chronos-Base | 200M | 0.8758 | 0.6521 |
| Chronos-Large | 710M | 0.8696 | 0.6473 |
| *Chronos-2 (teacher)* | *~120M* | *0.6978* | *0.4854* |

Trained on **8.25B points** (7.5B latent distillation + 750M output distillation) on a single
rented RTX 4090.

## Usage

```python
import numpy as np
from modeling_timetron import TimeTron

model = TimeTron.from_pretrained("CorteriIntelligence/TimeTron-v2-33M")

context = np.random.randn(4, 512).astype("float32")     # (batch, history)
quantiles = model.predict(context, prediction_length=128)   # (4, 128, 21), input's own scale
median = model.predict_median(context, prediction_length=128)  # (4, 128)
```

- **21 quantile levels**: `[0.01, 0.05, 0.1, 0.15 … 0.9, 0.95, 0.99]` — index 10 is the median
- **384 steps decoded natively**; longer horizons extend autoregressively in whole chunks
- **Any context length** — cropped to a multiple of 32 and left-padded as needed, up to 2048
- Output is in the **input's own scale**; no manual normalization required

Requires `torch`, `numpy`, and `safetensors`. `modeling_timetron.py` is self-contained.

## Model details

| | |
|---|---|
| parameters | 33.52M |
| hidden / layers / heads | 512 / 12 / 16 |
| patch length | 32 |
| max context | 2048 (trained at ≤512) |
| native horizon | 384 |
| attention | bidirectional, RoPE + QK-norm |
| normalization | causal patch-norm, asinh-compressed, μ-anchored |
| output | 21 quantiles per future patch |
| teacher | Chronos-2 (~120M) |

Forecasts are produced in `z = asinh((y − μ)/σ)` space and inverted as `ŷ = μ + σ·sinh(z)`,
which is why the model handles wide dynamic ranges without manual scaling.

### Training

Latent KD at three taps (student layers 4/8/12 → teacher layers 4/8/final) with SmoothL1
against LayerNorm'd, pooled teacher representations, plus a pinball loss on 21 quantiles and a
masked-view consistency term. The final checkpoint is a weight average of one latent-distilled
checkpoint and two output-distilled variants (`W_OUT` 0.2 and 0.5) — averaging over
*objective-diverse* checkpoints was the single most reliable source of gain in the project.

## Limitations

- **Test-data leakage: declared `Yes`.** Training pools contain series that are GIFT-Eval test
  datasets (m4 family, LOOP_SEATTLE, SZ_TAXI, hierarchical_sales, restaurant,
  temperature_rain). Long series used a 10% tail holdout and short series a last-window
  holdout, but partial overlap with official test horizons remains possible. We declare it
  rather than argue the edge case.
- **Univariate.** The architecture contains a group-attention branch for in-context learning
  across related series, and `predict(..., group_ids=...)` exposes it — but **on this
  checkpoint it is untrained and passing `group_ids` is a no-op**. Training it on a frozen
  backbone measured **3.4% worse** on GIFT's 43 multivariate configurations; the capability
  turned out to be inseparable from the backbone it co-adapts with.
- **Known-future covariates** are supported by the architecture (`future_values`) with zero
  additional parameters, but are only lightly trained (~5% of rows).
- **Trained at context ≤512.** Longer contexts are accepted but untested.
- **Horizons beyond 384** use autoregressive chunk extension, not a native long-horizon head.

## Evaluation

Numbers above are the official GIFT-Eval harness (gluonts metric engine, official windowing
and seasonality), aggregated as the geometric mean of per-configuration
`model_MASE / seasonal_naive_MASE`. Submission artifacts are in `gifteval_submission/`.

On **IEX** Indian electricity spot prices (private data, clean for every model compared),
TimeTron scores **1.189 MASE / 862 CRPS** versus TimesFM-2.5's 1.232 / 908 — better on both at
1/7 the size.

## Citation

```bibtex
@misc{srivastava2026timetron,
  title  = {TimeTron: Latent-Space Distillation of a Time-Series Foundation Model at 33M Parameters},
  author = {Srivastava, Aditya},
  year   = {2026},
  note   = {Corteri Intelligence}
}
```

## Acknowledgements

Distilled from [amazon/chronos-2](https://huggingface.co/amazon/chronos-2). Evaluated with
[GIFT-Eval](https://huggingface.co/datasets/Salesforce/GiftEval). Pretraining corpora: LOTSA,
Time-300B, GIFT-Eval-Pretrain.