tensorlink-dev's picture
cascade r8-challenger checkpoint: best public-bench artifact (GIFT-Eval CRPS 0.6009)
b428368 verified
|
Raw
History Blame Contribute Delete
2.79 kB
---
pipeline_tag: time-series-forecasting
tags:
- time-series
- forecasting
- toto2
- bittensor
- cascade
- synthetic-data
library_name: safetensors
---
# cascade Β· Toto2-4M β€” best public-benchmark checkpoint (round 8 challenger)
The strongest checkpoint (on public benchmarks) yet produced by **[cascade](https://github.com/TensorLink-AI/cascade)**, a Bittensor subnet (netuid 91) where miners compete on **data, not models**: each miner submits a synthetic time-series *data generator* (pure code, no weights), and the subnet trains this fixed 4M-parameter Toto2 **from random initialization** on each generator's corpus in a single ~3 h round, then evaluates on private, rotating real-world data the miners never see.
This model was trained on the corpus of the round-8 challenger generator (miner uid 72). It **won the public benchmarks but lost its duel** on the private eval β€” kept here as the best public-bench artifact the subnet has produced to date (as of 2026-08-10).
## Benchmark results
Scored with the official [gift-eval](https://github.com/SalesforceAIResearch/gift-eval) harness (full 97-config GIFT-Eval suite and BOOM, official Seasonal-Naive-normalized aggregation, leaderboard-comparable) plus the TIME benchmark:
| suite | CRPS | MASE |
|---|---|---|
| GIFT-Eval | **0.6009** | **0.8993** |
| BOOM | 0.4304 | 0.6951 |
| TIME | 0.6736 | 0.9200 |
(Lower is better; values are ratios vs the Seasonal-Naive baseline under the official shifted-geometric-mean aggregation.) `bench_report.json` in this repo is the trainer-signed score record.
For scale: this is a **4M-parameter model trained for ~3 GPU-hours from scratch**. The point is not to rival large foundation models β€” it is that data quality alone moved a fixed tiny model from ~0.68 (subnet genesis) to 0.60 GIFT-Eval CRPS in eight competitive rounds.
## Files
- `weights.safetensors` β€” trained Toto2-4M parameters
- `config.json` β€” architecture config
- `model.py` β€” model implementation
- `forecast_wrapper.py` β€” inference entry point (`forecast_quantiles_batch(histories, horizon)` quantile head); the same code path the subnet's validator and benchmark sidecar score through
- `bench_report.json` β€” signed benchmark record published by the subnet trainer
## Provenance
Content-addressed original on the Hippius Hub registry:
```
metro-v1:trained:hippius:cascade/ckpt-r13786693137342042853-challenger-toto2-4m@sha256:702bf6c7924dd1c58af273347c87a41e43f9a83ce76f9983ae6f99ba9014cd39
```
Round 13786693137342042853 (2026-08-08, netuid 91). Training is deterministic under the subnet's reproducibility contract (pinned torch 2.4.1+cu124, Python 3.11, fixed seeds): anyone can re-derive this checkpoint from the on-chain round data and the miner's revealed generator.