cascade r8-challenger checkpoint: best public-bench artifact (GIFT-Eval CRPS 0.6009)
b428368 verified | pipeline_tag: time-series-forecasting | |
| tags: | |
| - time-series | |
| - forecasting | |
| - toto2 | |
| - bittensor | |
| - cascade | |
| - synthetic-data | |
| library_name: safetensors | |
| # cascade Β· Toto2-4M β best public-benchmark checkpoint (round 8 challenger) | |
| The strongest checkpoint (on public benchmarks) yet produced by **[cascade](https://github.com/TensorLink-AI/cascade)**, a Bittensor subnet (netuid 91) where miners compete on **data, not models**: each miner submits a synthetic time-series *data generator* (pure code, no weights), and the subnet trains this fixed 4M-parameter Toto2 **from random initialization** on each generator's corpus in a single ~3 h round, then evaluates on private, rotating real-world data the miners never see. | |
| This model was trained on the corpus of the round-8 challenger generator (miner uid 72). It **won the public benchmarks but lost its duel** on the private eval β kept here as the best public-bench artifact the subnet has produced to date (as of 2026-08-10). | |
| ## Benchmark results | |
| Scored with the official [gift-eval](https://github.com/SalesforceAIResearch/gift-eval) harness (full 97-config GIFT-Eval suite and BOOM, official Seasonal-Naive-normalized aggregation, leaderboard-comparable) plus the TIME benchmark: | |
| | suite | CRPS | MASE | | |
| |---|---|---| | |
| | GIFT-Eval | **0.6009** | **0.8993** | | |
| | BOOM | 0.4304 | 0.6951 | | |
| | TIME | 0.6736 | 0.9200 | | |
| (Lower is better; values are ratios vs the Seasonal-Naive baseline under the official shifted-geometric-mean aggregation.) `bench_report.json` in this repo is the trainer-signed score record. | |
| For scale: this is a **4M-parameter model trained for ~3 GPU-hours from scratch**. The point is not to rival large foundation models β it is that data quality alone moved a fixed tiny model from ~0.68 (subnet genesis) to 0.60 GIFT-Eval CRPS in eight competitive rounds. | |
| ## Files | |
| - `weights.safetensors` β trained Toto2-4M parameters | |
| - `config.json` β architecture config | |
| - `model.py` β model implementation | |
| - `forecast_wrapper.py` β inference entry point (`forecast_quantiles_batch(histories, horizon)` quantile head); the same code path the subnet's validator and benchmark sidecar score through | |
| - `bench_report.json` β signed benchmark record published by the subnet trainer | |
| ## Provenance | |
| Content-addressed original on the Hippius Hub registry: | |
| ``` | |
| metro-v1:trained:hippius:cascade/ckpt-r13786693137342042853-challenger-toto2-4m@sha256:702bf6c7924dd1c58af273347c87a41e43f9a83ce76f9983ae6f99ba9014cd39 | |
| ``` | |
| Round 13786693137342042853 (2026-08-08, netuid 91). Training is deterministic under the subnet's reproducibility contract (pinned torch 2.4.1+cu124, Python 3.11, fixed seeds): anyone can re-derive this checkpoint from the on-chain round data and the miner's revealed generator. | |