| --- |
| license: apache-2.0 |
| tags: [time-series, forecasting, foundation-model, tokenizer-ablation, toto-2-recipe] |
| --- |
| # TSFM tokenizer ablation — Toto-2-4m-recipe clone (v10) |
|
|
| Context-tokenizer ablation on a ~3.6M-param decoder-only patched transformer |
| trained on a TempoPFN-style synthetic prior ensemble with contiguous patch |
| masking, a 9-level quantile head, NorMuon+AdamW, and index RoPE at |
| time-scaled positions. Horizon decoding is fixed patch-32 in every arm; only |
| the CONTEXT tokenizer varies: |
|
|
| | arm | tokenizer | history | ctx tokens | |
| |-----|-----------|---------|-----------| |
| | T0 | fixed-32 (control) | 4,096 | 128 | |
| | T1 | pyramid, iso-context | 4,096 | 44 | |
| | T2 | pyramid, iso-token | 16,384 | 128 | |
| | T3 | adaptive equal-surprise | 16,384 | 128 | |
|
|
| Each subfolder is one (arm, seed) run: `model.pt` (final), `ckpt_15000.pt` |
| (rank-stability snapshot), `config.json`, `results.json` (dev-GIFT CRPS + |
| long-horizon probe). Dev metrics use a fixed 14-task GIFT-Eval subset — NOT |
| the full leaderboard; treat numbers as ablation-internal, not comparable to |
| published GIFT scores. Generated by the v10 experiment notebook. |
|
|
| ## Results |
| ``` |
| gm_crps_mean gm_crps_std gm_crps_15k long_season other probe_r probe_amp n |
| arm |
| T0_fixed 0.1949 0.0113 0.1875 0.2570 0.1352 -0.0067 0.2647 3 |
| T1_pyr_ctx 0.2070 0.0089 0.2015 0.2673 0.1473 -0.0020 0.3043 3 |
| T2_pyr_tok 0.2166 0.0143 0.2030 0.2664 0.1647 0.0000 0.2563 3 |
| T3_adaptive 0.2324 0.0145 0.2175 0.2588 0.2021 0.0057 0.2533 3 |
| |
| --- tripwire checks --- |
| seed noise floor (max arm sigma): 0.0145 |
| T0_fixed vs T1_pyr_ctx: gap=0.0121 -> UNRESOLVED (< 2*sigma) — add seeds/steps |
| T1_pyr_ctx vs T2_pyr_tok: gap=0.0095 -> UNRESOLVED (< 2*sigma) — add seeds/steps |
| T2_pyr_tok vs T3_adaptive: gap=0.0158 -> UNRESOLVED (< 2*sigma) — add seeds/steps |
| rank stable 15k->30k: ['T0_fixed', 'T1_pyr_ctx', 'T2_pyr_tok', 'T3_adaptive'] |
| H1 compression tax (T1 vs T0): +6.23% (FAILS H1 gate — read T2 as gain minus tax) |
| |
| ``` |
|
|
