TinyCast v1.0.0: weights and model card

.gitattributes

@@ -0,0 +1 @@
+*.safetensors filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+__pycache__/

README.md

@@ -0,0 +1,210 @@
+---
+license: apache-2.0
+library_name: tinycast
+pipeline_tag: time-series-forecasting
+datasets:
+  - Salesforce/GiftEvalPretrain
+tags:
+  - time-series
+  - time-series-forecasting
+  - foundation-model
+  - attention-free
+  - gift-eval
+  - edge
+  - streaming
+  - quantization
+---
+
+# TinyCast
+
+**Probabilistic zero-shot forecasting at 146,505 parameters: the smallest GIFT-Eval entry that
+publishes per-configuration results and declares no leakage.**
+
+TinyCast forecasts a series it has never seen, with no fitting and no fine-tuning, and returns nine
+quantiles rather than a single number. It is attention-free, and it computes each context's
+periodicity instead of learning it, so no capacity is spent rediscovering seasonality.
+
+**Below 1.4 M parameters it is the only zero-shot model on the GIFT-Eval board that emits a
+predictive distribution**, and it fits on a microcontroller.
+
+| | |
+|---|---|
+| **Paper** | https://arxiv.org/abs/2608.15767 |
+| **Code, training and replication** | https://github.com/raws-labs/tinycast |
+| **Parameters** | 146,505 (fp32 weights, about 0.6 MB) |
+| **GIFT-Eval, zero-shot** | 0.774 nGMASE, 0.545 nWQL, 0.554 nMSIS over 97 configurations |
+| **On a Cortex-M7** | 138.1 KiB INT8 weights, 730.7 KiB peak RAM, 4.08 s per call ([firmware profile](#on-device), 0.833 nGMASE) |
+| **License** | Apache-2.0 |
+
+## GIFT-Eval results (zero-shot)
+
+| Metric | Value |
+|--------|-------|
+| **nGMASE** (point accuracy) | **0.774** |
+| **nWQL** (probabilistic accuracy) | **0.545** |
+| **nMSIS** (interval score) | **0.554** |
+
+All three are geometric means, over the 97 benchmark configurations, of the ratio between the
+model's metric and the seasonal-naive reference's; 1.000 is parity with seasonal naive. Every number
+on this page comes from one profile: bf16 autocast at compute, flip-invariance symmetrization and
+period-alignment downsampling, which the reproduction command below runs.
+
+**Below 1.4 M parameters, no zero-shot model on the board emits a predictive distribution** (counting
+models with a public per-configuration result and no declared test-data leakage, as the table below
+does). TinyCast emits nine quantiles at 146,505 parameters, so below that size the probabilistic
+frontier does not exist until this model defines it. Every board model that scores below TinyCast's
+nWQL of 0.545 carries at least 1.4 M parameters: TTM-R3 spends about 10 times the budget for 0.026 of
+nWQL, and FlowState-9.1M spends 62 times for 0.043. The two smaller models emit point forecasts only,
+and the nearest model in that family that emits quantiles carries eighteen times TinyCast's
+parameters.
+
+Point accuracy extends the same frontier by scaling. TinyCast is smaller than every zero-shot model
+on the board with a public per-configuration result and no declared test-data leakage, and it is
+level with the smallest of them at three quarters of its parameters: 0.774 nGMASE against
+Reverso-Nano's 0.760, a gap that does not survive resampling by base dataset.
+
+Against every zero-shot model on the board up to 10 M parameters with a public per-configuration
+result and no declared leakage, recomputed from one pinned snapshot against the same reference:
+
+| Model | Params | nGMASE | nWQL | Predictive distribution |
+|---|---:|---:|---:|:---:|
+| **TinyCast** | **146 K** | 0.774 | **0.545** | yes |
+| Reverso-Nano | 200 K | 0.760 | (0.661) | no |
+| Reverso-Small | 550 K | 0.726 | (0.626) | no |
+| TTM-R3 | 1.4 M | 0.724 | 0.520 | yes |
+| Reverso | 2.6 M | 0.711 | (0.610) | no |
+| Toto-2.0-4m | 4.1 M | 0.757 | 0.524 | yes |
+| YingLong-6m | 7.3 M | 0.880 | 0.609 | yes |
+| FlowState-9.1M | 9.1 M | 0.726 | 0.502 | yes |
+| Kairos-10m | 9.9 M | 0.753 | 0.554 | yes |
+
+The Reverso models emit no predictive distribution, so their nWQL column is a point error and is not
+comparable with the rest; it is parenthesized for that reason.
+
+**Other benchmarks.** On Chronos-ZS (27 tasks) TinyCast reaches relative MASE 0.880 and relative WQL
+0.722; on fev-bench (100 tasks) relative MASE 0.819, relative WQL 0.658 and a skill score of 0.304.
+On both, every neural model ahead of it carries at least 28 times its parameters. Each benchmark
+normalizes over its own dataset set, so these aggregates are not comparable with the GIFT-Eval
+figures above.
+
+## Model description
+
+| | |
+|---|---|
+| Backbone | 10 dilated causal Conv1d blocks, kernel 3, dilations doubling from 1 to 512 (receptive field 2047 over a context of 2048) |
+| Efficiency | depthwise-separable convolutions, one SwiGLU feed-forward ALBERT-tied across all ten blocks |
+| Structural prior | Fisher's significance test for harmonic analysis on the normalized periodogram, alpha 0.05, up to four periods, zero parameters, then a 16-bin phase fold |
+| Decoder | pooled summary, phase-gather seasonal profile and a causal future-conv correction; nine decile quantiles |
+| Normalization | per-context min-max over the observed history, inverted on the output |
+| Context / horizon | context 2048; forecasts in blocks of 48 steps, rolled out autoregressively (evaluated to 720) |
+| Attention | none |
+| Working memory | bounded per-layer ring buffers that do not grow as the model runs; causal padding, so a per-step streaming variant is exact rather than approximate |
+
+Every learned operation is a convolution, a matrix multiplication or a normalization, so the model
+exports to a static INT8 graph and runs a forecast end to end on an embedded device.
+
+### On device
+
+Deployed on an STM32H753 (Arm Cortex-M7) development board as a static W8A8 graph with quantization
+scales calibrated once and frozen. Single core, no neural accelerator, no off-chip memory.
+
+| | |
+|---|---|
+| INT8 matrix and convolution coefficients | 138.1 KiB |
+| Complete firmware image, including an 8 KiB context | 365.5 KiB |
+| Peak RAM (statics, heap, stack high-water) | 730.7 KiB |
+| One core call at a 2048-step context | 4.08 s |
+
+The board runs the **firmware configuration**, which scores **0.833 nGMASE and 0.581 nWQL**, not the
+0.774 / 0.545 headlined above: it executes INT8 and drops both inference-time strategies.
+Quantization alone, with the strategies kept, costs 2.1% of aggregate point accuracy over all 97
+configurations.
+
+**Training data.** Pretrained once on GIFT-Eval-Pretrain with every dataset overlapping the GIFT-Eval
+and Chronos-ZS test sets removed, plus Chronos KernelSynth and four synthetic shards. No per-dataset
+fine-tuning: all results above are zero-shot.
+
+## Files
+
+- `model.safetensors`: 146,505 fp32 parameters, about 0.6 MB. The weight-tied feed-forward is stored
+  once, so counting parameters by summing a loaded `state_dict()` overcounts; instantiate from the
+  config instead.
+- `config.json`: the `TinyCastConfig` the loader rebuilds the architecture from.
+
+## Usage
+
+There is no PyPI package. Install from this repository:
+
+```bash
+git clone https://github.com/raws-labs/tinycast.git
+cd tinycast && pip install -e .
+```
+
+Load the weights:
+
+```python
+from huggingface_hub import hf_hub_download
+from tinycast import load_model
+
+weights = hf_hub_download("raws-labs/tinycast", "model.safetensors")
+hf_hub_download("raws-labs/tinycast", "config.json")   # sibling, picked up automatically
+
+model, config = load_model(weights)   # TinyCastForPrediction, 146,505 params
+```
+
+Forecast with the gluonts predictor these results were produced with:
+
+```python
+from tinycast import TinyCastPredictor
+
+predictor = TinyCastPredictor(prediction_length=48, checkpoint_path=weights,
+                              freq="H", domain="Energy", device="cpu")
+forecasts = predictor.predict(test_input)   # QuantileForecasts, nine deciles
+```
+
+Reproduce the table above. The GIFT-Eval data loader is a separate install, needed only by the
+benchmark driver; note that its distribution name and its import name differ:
+
+```bash
+pip install "salesforce-gift-eval @ git+https://github.com/SalesforceAIResearch/gift-eval.git"
+
+export GIFT_EVAL=/path/to/gift-eval
+python -m tinycast.eval --ckpt model.safetensors --flip \
+    --device cuda --output all_results.csv
+```
+
+The repository README documents the thread caps that keep this from running many times slower, and
+everything else that moves the numbers. It also covers the training recipe, checkpoint averaging and
+export, and the synthetic-corpus builder, all of which ship in the same package.
+
+## Intended use and limitations
+
+TinyCast is built for forecasting a univariate signal on hardware that was not chosen for machine
+learning, where a per-signal model would have to be fitted and maintained for every deployment.
+
+- It is univariate. It uses no covariates and no cross-series structure, so a task that supplies
+  either will be forecast without them.
+- The model emits no signal when its input leaves the regime its pretraining covers, so degradation
+  there is silent.
+- The period comes from a rounded FFT bin, so its resolution falls as the ratio of window to period
+  falls: about seven percent for a weekly cycle in hourly data.
+- Min-max normalization is per context window and therefore sensitive to a single extreme value.
+- Fresh deployments degrade toward the seasonal-naive baseline rather than failing: parity at 64
+  observed samples, and two thirds of the way back to full-context accuracy by 512.
+
+## License
+
+Apache-2.0. Third-party attributions are in the code repository's `NOTICE`.
+
+## Citation
+
+```bibtex
+@misc{tinycast2026,
+  title         = {TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicity},
+  author        = {Armin Steinhauser},
+  year          = {2026},
+  eprint        = {2608.15767},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.LG}
+}
+```

config.json

@@ -0,0 +1,31 @@
+{
+  "quantiles": [
+    0.1,
+    0.2,
+    0.3,
+    0.4,
+    0.5,
+    0.6,
+    0.7,
+    0.8,
+    0.9
+  ],
+  "seq_len": 2048,
+  "output_token_len": 48,
+  "top_k_periods": 4,
+  "significance_alpha": 0.05,
+  "n_harmonics": 1,
+  "conv_dim": 64,
+  "n_layers": 10,
+  "kernel_size": 3,
+  "ffn_mult": 1.0,
+  "pool_kind": "mean_last",
+  "causal": true,
+  "phase_bins": 16,
+  "decoder_depth": 1,
+  "separable_conv": true,
+  "share_ffn": true,
+  "future_conv": true,
+  "future_conv_layers": 6,
+  "future_conv_seed": 128
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e436c98bd9c8b62b866ccbdfc5ceea4552eac26e05928b57b269ff2813312e81
+size 602044