TriChronos-50M

TriChronos-50M is a ~50M-parameter, encoder-only Transformer for probabilistic time-series forecasting. Weights are trained with 1.58-bit ternary quantisation (BitNet-style {-1, 0, +1}), and the model outputs 21 quantiles per future step rather than a single point forecast.

It was trained from scratch on a strict compute budget (single NVIDIA L40S, ~$15) as a study in how far a small, quantised model can go on general time-series forecasting — not as a state-of-the-art benchmark entry.

TL;DR — On the datasets it does well, it does genuinely well: it beats the naïve baseline on Weather (MASE 0.83) and M3-Monthly (0.81). Performance is strongly frequency-dependent: solid on monthly/high-frequency series, weak on quarterly, and poor on yearly (which have very few observations). Read the per-frequency breakdown below rather than the headline aggregate.

---

Highlights

	Dataset	MASE	Meaning
🟢	Weather	0.83	Beats naïve — strongest result
🟢	M3-Monthly	0.81	Beats naïve
🟡	Traffic	2.13	Moderate
🟡	M1-Monthly	1.66	Moderate
🔴	Quarterly / Yearly	4–43	Weak → poor (few observations, long horizons)

MASE < 1 = better than the naïve baseline; lower is better.

---

Architecture

Property	Value
Parameters	50,081,016 (~50M)
Type	Encoder-only Transformer
d_model	768
Layers	6
Heads	12
FFN dim	2304
Patch size	8 timesteps
Forecast horizon	24 timesteps
Weight precision	1.58-bit ternary ({-1, 0, +1}, BitLinear) in attention + FFN
Activation precision	8-bit per-token
Training precision	BF16 autocast
Output	21 quantiles (τ = 0.025, 0.05, 0.10 … 0.90, 0.95, 0.975)

Each encoder block applies temporal self-attention, then cross-series ("group") attention over the batch, then a BitLinear FFN. The input series is split into non-overlapping 8-step patches; patch embeddings and the quantile head stay in full precision.

---

Training

• Data: Salesforce/lotsa_data, streamed per-subset (Bronze→Silver→Gold pipeline: asinh z-score normalisation → 8-step patches).
• Hardware / budget: 1× NVIDIA L40S, ~$15 total compute.
• Steps: ~105k (single session; cosine LR annealed toward 10% of peak).
• Optimiser: AdamW, lr=3e-4, wd=1e-2, β=(0.9, 0.95), 2k-step warmup.
• Loss: pinball / quantile loss over all 21 quantiles.

---

Evaluation

Zero-shot MASE on Monash Time Series Forecasting datasets (via the Parquet mirror autogluon/chronos_datasets). Two protocols are reported for honesty:

• Full — up to a few hundred series per dataset.
• Diagnostic (10-series) — a fixed small sample per dataset; faster, but higher variance.

⚠️ Metric caveat. MASE here is computed in the model's normalised (asinh z-score) space, for both the model and the naïve baseline. The model-vs-baseline comparison is therefore fair and internally consistent, but these numbers are not directly comparable to published Monash leaderboards, which report MASE on raw values with per-dataset seasonal-naïve denominators.

Per-dataset MASE

Dataset	Full	Diagnostic (10-series)	Read
Weather	0.868	0.831	🟢 beats naïve
M3 Monthly	0.808	1.012	🟢 beats naïve (full)
M1 Monthly	1.656	2.168	🟡 moderate
Traffic	2.133	1.865	🟡 moderate
M4 Monthly	2.413	2.283	🟡 moderate
Electricity Hourly	3.377	1.833	🟡 moderate
Tourism Monthly	3.236	2.567	🟠 weak
Tourism Quarterly	4.062	2.911	🟠 weak
M4 Quarterly	4.086	1.501	🟠 weak (full)
M3 Quarterly	4.478	—	🟠 weak
M1 Quarterly	4.906	3.735	🔴 weak
M4 Yearly	14.007	—	🔴 poor (long-horizon, few obs.)
M1 Yearly	42.972	—	🔴 poor (only 3 series)

Aggregates

Slice	Mean MASE	Note
Full benchmark (all 13)	6.846	inflated by yearly outliers
Excluding yearly	2.91	fairer central estimate
High-frequency subset (monthly + hourly)	≈ 2.07	where the model is designed to work
Diagnostic 10-series	2.071	small-sample sanity check

How to read this: the full-benchmark mean is dominated by two low-frequency datasets (M1-Yearly = 42.97 across just 3 series, M4-Yearly = 14.01). Those series have very few observations and a long forecast horizon — intrinsically hard, and statistically noisy at this sample size. The high-frequency subset (~2.07) is the number that best reflects the model's actual behaviour.

Observed pattern: frequency dependence

Monthly   → good        (M3 0.81, M1 1.66, M4 2.41)
Hourly    → moderate    (Traffic 2.13, Electricity 3.38)
Quarterly → weak        (~4.0–4.9)
Yearly    → poor        (14–43, small N)

This is consistent with a model that has learned local, high-frequency temporal structure but degrades when a series is short and the horizon is long relative to the available history — a data/generalisation limitation more than an obvious capacity ceiling.

---

Intended use & limitations

Intended: research on small / quantised time-series foundation models; probabilistic forecasting on monthly and higher-frequency univariate series; a lightweight baseline.

Not recommended (as-is): yearly or very short series; long-horizon forecasting far beyond 24 steps; any setting needing calibrated leaderboard-grade MASE without re-running evaluation on raw values.

Known limitations

• Frequency-dependent quality (above).
• MASE reported in normalised space (above) — recompute on raw values for cross-paper comparison.
• Trained ~105k steps on a single small budget; not converged to SOTA.
• The forecast head mean-pools patch representations before projecting the horizon, which can flatten fine temporal detail on long horizons.

---

Usage

import torch
from model import TriChronos   # from this repo

model = TriChronos()           # d_model=768, n_layers=6, n_heads=12, ffn_dim=2304
model.load_state_dict(torch.load("model_state.pt", map_location="cpu"))
model.eval()

# patches: (batch, n_patches, patch_size=8) — asinh z-scored, most-recent-last
patches = torch.randn(1, 64, 8)
with torch.no_grad():
    quantiles = model(patches)   # (1, 24, 21) → horizon × quantile levels
median = quantiles[..., 9]       # τ = 0.50

Preprocessing (asinh z-score → 8-step patches) and the quantile levels are defined in data_pipeline.py / model.py, both included in this repo.

Reproducing the evaluation

python evaluate.py --checkpoint model_state.pt --max-series 200

Citation

@misc{trichronos2026,
  title  = {TriChronos-50M: Ternary-Quantised Probabilistic Time-Series Forecasting},
  year   = {2026},
  url     = {https://huggingface.co/iravikr/trichronos-50M}
}

License

Apache 2.0