---
language: en
license: apache-2.0
library_name: pytorch
pipeline_tag: time-series-forecasting
tags:
- time-series
- forecasting
- probabilistic-forecasting
- quantile-regression
- bitnet
- ternary
- quantization
datasets:
- Salesforce/lotsa_data
metrics:
- mase
---

# TriChronos-50M

**TriChronos-50M** is a ~50M-parameter, encoder-only Transformer for **probabilistic**
time-series forecasting. Weights are trained with **1.58-bit ternary quantisation**
(BitNet-style `{-1, 0, +1}`), and the model outputs **21 quantiles** per future step
rather than a single point forecast.

It was trained from scratch on a strict compute budget (single NVIDIA L40S, ~$15) as a
study in *how far a small, quantised model can go* on general time-series forecasting —
not as a state-of-the-art benchmark entry.

> **TL;DR** — On the datasets it does well, it does *genuinely* well: it **beats the
> naïve baseline on Weather (MASE 0.83) and M3-Monthly (0.81)**. Performance is strongly
> **frequency-dependent**: solid on monthly/high-frequency series, weak on quarterly, and
> poor on yearly (which have very few observations). Read the per-frequency breakdown
> below rather than the headline aggregate.

---

## Highlights

| | Dataset | MASE | Meaning |
|---|---|---:|---|
| 🟢 | **Weather** | **0.83** | Beats naïve — strongest result |
| 🟢 | **M3-Monthly** | **0.81** | Beats naïve |
| 🟡 | Traffic | 2.13 | Moderate |
| 🟡 | M1-Monthly | 1.66 | Moderate |
| 🔴 | Quarterly / Yearly | 4–43 | Weak → poor (few observations, long horizons) |

*MASE < 1 = better than the naïve baseline; lower is better.*

---

## Architecture

| Property | Value |
|---|---|
| Parameters | **50,081,016** (~50M) |
| Type | Encoder-only Transformer |
| d_model | 768 |
| Layers | 6 |
| Heads | 12 |
| FFN dim | 2304 |
| Patch size | 8 timesteps |
| Forecast horizon | 24 timesteps |
| Weight precision | 1.58-bit ternary (`{-1, 0, +1}`, BitLinear) in attention + FFN |
| Activation precision | 8-bit per-token |
| Training precision | BF16 autocast |
| Output | 21 quantiles (τ = 0.025, 0.05, 0.10 … 0.90, 0.95, 0.975) |

Each encoder block applies **temporal self-attention**, then **cross-series ("group")
attention** over the batch, then a BitLinear FFN. The input series is split into
non-overlapping 8-step patches; patch embeddings and the quantile head stay in full
precision.

---

## Training

- **Data:** [Salesforce/lotsa_data](https://huggingface.co/datasets/Salesforce/lotsa_data),
  streamed per-subset (Bronze→Silver→Gold pipeline: asinh z-score normalisation → 8-step patches).
- **Hardware / budget:** 1× NVIDIA L40S, ~$15 total compute.
- **Steps:** ~105k (single session; cosine LR annealed toward 10% of peak).
- **Optimiser:** AdamW, `lr=3e-4`, `wd=1e-2`, β=(0.9, 0.95), 2k-step warmup.
- **Loss:** pinball / quantile loss over all 21 quantiles.

---

## Evaluation

Zero-shot MASE on Monash Time Series Forecasting datasets (via the Parquet mirror
`autogluon/chronos_datasets`). **Two protocols are reported for honesty:**

- **Full** — up to a few hundred series per dataset.
- **Diagnostic (10-series)** — a fixed small sample per dataset; faster, but higher variance.

> ⚠️ **Metric caveat.** MASE here is computed in the model's *normalised (asinh z-score)*
> space, for both the model and the naïve baseline. The model-vs-baseline comparison is
> therefore fair and internally consistent, but these numbers are **not directly
> comparable to published Monash leaderboards**, which report MASE on raw values with
> per-dataset seasonal-naïve denominators.

### Per-dataset MASE

| Dataset | Full | Diagnostic (10-series) | Read |
|---|---:|---:|---|
| **Weather** | **0.868** | **0.831** | 🟢 beats naïve |
| **M3 Monthly** | **0.808** | 1.012 | 🟢 beats naïve (full) |
| M1 Monthly | 1.656 | 2.168 | 🟡 moderate |
| Traffic | 2.133 | 1.865 | 🟡 moderate |
| M4 Monthly | 2.413 | 2.283 | 🟡 moderate |
| Electricity Hourly | 3.377 | 1.833 | 🟡 moderate |
| Tourism Monthly | 3.236 | 2.567 | 🟠 weak |
| Tourism Quarterly | 4.062 | 2.911 | 🟠 weak |
| M4 Quarterly | 4.086 | 1.501 | 🟠 weak (full) |
| M3 Quarterly | 4.478 | — | 🟠 weak |
| M1 Quarterly | 4.906 | 3.735 | 🔴 weak |
| M4 Yearly | 14.007 | — | 🔴 poor (long-horizon, few obs.) |
| M1 Yearly | 42.972 | — | 🔴 poor (**only 3 series**) |

### Aggregates

| Slice | Mean MASE | Note |
|---|---:|---|
| Full benchmark (all 13) | **6.846** | inflated by yearly outliers |
| Excluding yearly | **2.91** | fairer central estimate |
| **High-frequency subset** (monthly + hourly) | **≈ 2.07** | where the model is designed to work |
| Diagnostic 10-series | **2.071** | small-sample sanity check |

**How to read this:** the full-benchmark mean is dominated by two low-frequency datasets
(M1-Yearly = 42.97 across just **3 series**, M4-Yearly = 14.01). Those series have very
few observations and a long forecast horizon — intrinsically hard, and statistically
noisy at this sample size. The high-frequency subset (~2.07) is the number that best
reflects the model's actual behaviour.

### Observed pattern: frequency dependence

```
Monthly   → good        (M3 0.81, M1 1.66, M4 2.41)
Hourly    → moderate    (Traffic 2.13, Electricity 3.38)
Quarterly → weak        (~4.0–4.9)
Yearly    → poor        (14–43, small N)
```

This is consistent with a model that has learned **local, high-frequency temporal
structure** but degrades when a series is short and the horizon is long relative to the
available history — a data/generalisation limitation more than an obvious capacity ceiling.

---

## Intended use & limitations

**Intended:** research on small / quantised time-series foundation models; probabilistic
forecasting on **monthly and higher-frequency** univariate series; a lightweight baseline.

**Not recommended (as-is):** yearly or very short series; long-horizon forecasting far
beyond 24 steps; any setting needing calibrated leaderboard-grade MASE without re-running
evaluation on raw values.

**Known limitations**
- Frequency-dependent quality (above).
- MASE reported in normalised space (above) — recompute on raw values for cross-paper comparison.
- Trained ~105k steps on a single small budget; not converged to SOTA.
- The forecast head mean-pools patch representations before projecting the horizon, which
  can flatten fine temporal detail on long horizons.

---

## Usage

```python
import torch
from model import TriChronos   # from this repo

model = TriChronos()           # d_model=768, n_layers=6, n_heads=12, ffn_dim=2304
model.load_state_dict(torch.load("model_state.pt", map_location="cpu"))
model.eval()

# patches: (batch, n_patches, patch_size=8) — asinh z-scored, most-recent-last
patches = torch.randn(1, 64, 8)
with torch.no_grad():
    quantiles = model(patches)   # (1, 24, 21) → horizon × quantile levels
median = quantiles[..., 9]       # τ = 0.50
```

Preprocessing (asinh z-score → 8-step patches) and the quantile levels are defined in
`data_pipeline.py` / `model.py`, both included in this repo.

## Reproducing the evaluation

```bash
python evaluate.py --checkpoint model_state.pt --max-series 200
```

## Citation

```bibtex
@misc{trichronos2026,
  title  = {TriChronos-50M: Ternary-Quantised Probabilistic Time-Series Forecasting},
  year   = {2026},
  url     = {https://huggingface.co/iravikr/trichronos-50M}
}
```

## License

Apache 2.0