| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1" /> |
| <title>FUTURE-TS — Future-only benchmark for time-series foundation models</title> |
| <meta |
| name="description" |
| content="FUTURE-TS is a v0.1.0 public-preview benchmark for time-series foundation models: future-only evaluation, strict manifests, sealed-runner execution, and multi-budget scoring." |
| /> |
| <link rel="preconnect" href="https://fonts.googleapis.com" /> |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin /> |
| <link |
| href="https://fonts.googleapis.com/css2?family=Sora:wght@400;500;600&family=Manrope:wght@400;500;600&family=IBM+Plex+Mono:wght@400;500&display=swap" |
| rel="stylesheet" |
| /> |
| <link rel="stylesheet" href="styles.css" /> |
| </head> |
| <body> |
| <header class="site-header"> |
| <a class="brand" href="#top" aria-label="FUTURE-TS home"> |
| <svg class="brand-mark" viewBox="0 0 24 24" fill="none" aria-hidden="true"> |
| <polyline points="3 17 9 12 14 15 21 5" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" /> |
| <circle cx="21" cy="5" r="2.4" class="mark-dot" /> |
| </svg> |
| <span class="brand-name">FUTURE-TS</span> |
| <span class="brand-tag">TSFM.ai</span> |
| </a> |
| <nav class="site-nav" aria-label="Sections"> |
| <a href="#contract">Benchmark</a> |
| <a href="#results">Results</a> |
| <a href="#submit">Submit</a> |
| <a href="#scope">Scope</a> |
| </nav> |
| <a class="header-link" href="docs/benchmark-card.md">Docs</a> |
| </header> |
|
|
| <main id="top"> |
| <section class="hero"> |
| <p class="eyebrow">v0.1.0 — public preview</p> |
| <h1>Future-only evaluation for time-series foundation models.</h1> |
| <p class="lede"> |
| FUTURE-TS ranks models only on labels that were unavailable when the |
| model was frozen — with strict task cards, pretraining manifests, |
| sealed-runner execution, and multi-budget context scoring. |
| </p> |
| <div class="hero-actions"> |
| <a class="btn" href="docs/benchmark-card.md">Benchmark card</a> |
| <a class="link-arrow" href="docs/submission-guide.md">Submit a model<span aria-hidden="true">→</span></a> |
| </div> |
| </section> |
|
|
| <figure class="series" aria-label="Illustration: an observed series and a future-only forecast across the freeze point"> |
| <svg id="series-svg" viewBox="0 0 1200 220" preserveAspectRatio="none" aria-hidden="true"></svg> |
| <figcaption class="series-caption"> |
| <span class="mono">observed</span> |
| <span class="mono accent">freeze</span> |
| <span class="mono">future-only horizon</span> |
| </figcaption> |
| </figure> |
|
|
| <section class="stats" aria-label="Release metrics"> |
| <div><span class="stat-num">25</span><span class="stat-label mono">strict-surface tasks</span></div> |
| <div><span class="stat-num">15</span><span class="stat-label mono">empirical v2 tasks</span></div> |
| <div><span class="stat-num">47</span><span class="stat-label mono">scored public models</span></div> |
| <div><span class="stat-num">3</span><span class="stat-label mono">context budgets</span></div> |
| </section> |
|
|
| <section id="contract" class="block"> |
| <div class="block-head"> |
| <p class="kicker mono">The contract</p> |
| <h2>Evaluation as a program, not a static bundle.</h2> |
| <p class="block-sub"> |
| FUTURE-TS separates clean future-only evidence from accidental |
| leakage, catalog convenience, and post-hoc tuning. |
| </p> |
| </div> |
| <div class="grid-4"> |
| <article> |
| <span class="idx mono">01</span> |
| <h3>Future-only ranking</h3> |
| <p>Ranked labels must be unavailable at freeze time. Task cards carry issue times, target timestamps, and availability metadata.</p> |
| </article> |
| <article> |
| <span class="idx mono">02</span> |
| <h3>Strict manifest mode</h3> |
| <p><code>benchmarks/v1</code> rejects manifestless submissions, so declared-clean and undeclared models are never conflated.</p> |
| </article> |
| <article> |
| <span class="idx mono">03</span> |
| <h3>Sealed execution</h3> |
| <p>The local sealed runner stamps platform timestamps, applies resource caps, and supports network isolation on Linux.</p> |
| </article> |
| <article> |
| <span class="idx mono">04</span> |
| <h3>Multi-budget context</h3> |
| <p>The empirical run evaluates zero-shot, few-shot, and s16 budgets at 96, 192, and 288 observations.</p> |
| </article> |
| </div> |
| </section> |
|
|
| <section id="results" class="block"> |
| <div class="block-head"> |
| <p class="kicker mono">Canonical public run</p> |
| <h2>Empirical v2 — public catalog, three budgets.</h2> |
| <p class="block-sub"> |
| The current artifact set excludes internal evaluator-only entries and |
| reports 47 scored public models. Useful evidence, not a permanent |
| universal ranking. |
| </p> |
| </div> |
| <table class="board"> |
| <thead> |
| <tr><th class="mono">Rank</th><th class="mono">Model</th><th class="mono">Tier</th><th class="num mono">Score</th></tr> |
| </thead> |
| <tbody> |
| <tr><td class="rank mono">01</td><td class="model">Datadog/Toto-2.0-1B</td><td class="tier mono">T1</td><td class="num mono">0.2369</td></tr> |
| <tr><td class="rank mono">02</td><td class="model">Datadog/Toto-2.0-313m <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T2</td><td class="num mono">0.2214</td></tr> |
| <tr><td class="rank mono">03</td><td class="model">NX-AI/TiRex <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T3</td><td class="num mono">0.2024</td></tr> |
| <tr><td class="rank mono">04</td><td class="model">Salesforce/moirai-2.0-R-small <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T4</td><td class="num mono">0.1860</td></tr> |
| <tr><td class="rank mono">05</td><td class="model">amazon/chronos-2 <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T4</td><td class="num mono">0.1851</td></tr> |
| <tr><td class="rank mono">06</td><td class="model">NX-AI/TiRex-1.1-gifteval</td><td class="tier mono">T4</td><td class="num mono">0.1833</td></tr> |
| <tr><td class="rank mono">07</td><td class="model">google/timesfm-2.5-200m-pytorch</td><td class="tier mono">T4</td><td class="num mono">0.1772</td></tr> |
| <tr><td class="rank mono">08</td><td class="model">google/timesfm-2.0-500m-pytorch <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T4</td><td class="num mono">0.1706</td></tr> |
| <tr><td class="rank mono">09</td><td class="model">Datadog/Toto-2.0-22m</td><td class="tier mono">T4</td><td class="num mono">0.1640</td></tr> |
| <tr><td class="rank mono">10</td><td class="model">Datadog/Toto-2.0-4m <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T4</td><td class="num mono">0.1557</td></tr> |
| <tr><td class="rank mono">11</td><td class="model">cisco-ai/cisco-time-series-model-1.0 <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T5</td><td class="num mono">0.1437</td></tr> |
| <tr><td class="rank mono">12</td><td class="model">Salesforce/moirai-1.1-R-large</td><td class="tier mono">T6</td><td class="num mono">0.1277</td></tr> |
| <tr><td class="rank mono">13</td><td class="model">amazon/chronos-t5-large</td><td class="tier mono">T7</td><td class="num mono">0.1068</td></tr> |
| <tr><td class="rank mono">14</td><td class="model">Salesforce/moirai-1.1-R-base</td><td class="tier mono">T7</td><td class="num mono">0.1067</td></tr> |
| <tr><td class="rank mono">15</td><td class="model">amazon/chronos-bolt-mini</td><td class="tier mono">T7</td><td class="num mono">0.1030</td></tr> |
| <tr><td class="rank mono">16</td><td class="model">Salesforce/moirai-1.1-R-small <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T8</td><td class="num mono">0.0882</td></tr> |
| <tr><td class="rank mono">17</td><td class="model">mldi-lab/Kairos_50m</td><td class="tier mono">T9</td><td class="num mono">0.0746</td></tr> |
| <tr><td class="rank mono">18</td><td class="model">ibm-research/granite-timeseries-flowstate-r1.1 <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T9</td><td class="num mono">0.0738</td></tr> |
| <tr><td class="rank mono">19</td><td class="model">amazon/chronos-bolt-small</td><td class="tier mono">T9</td><td class="num mono">0.0645</td></tr> |
| <tr><td class="rank mono">20</td><td class="model">thuml/sundial-base-128m</td><td class="tier mono">T9</td><td class="num mono">0.0618</td></tr> |
| <tr><td class="rank mono">21</td><td class="model">Datadog/Toto-Open-Base-1.0</td><td class="tier mono">T9</td><td class="num mono">0.0574</td></tr> |
| <tr><td class="rank mono">22</td><td class="model">amazon/chronos-bolt-base <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T9</td><td class="num mono">0.0518</td></tr> |
| <tr><td class="rank mono">23</td><td class="model">mldi-lab/Kairos_23m</td><td class="tier mono">T9</td><td class="num mono">0.0464</td></tr> |
| <tr><td class="rank mono">24</td><td class="model">Maple728/TimeMoE-50M <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T9</td><td class="num mono">0.0406</td></tr> |
| <tr><td class="rank mono">25</td><td class="model">bytedance-research/Timer-S1</td><td class="tier mono">T9</td><td class="num mono">0.0369</td></tr> |
| <tr><td class="rank mono">26</td><td class="model">Salesforce/moirai-1.0-R-large</td><td class="tier mono">T10</td><td class="num mono">0.0268</td></tr> |
| <tr><td class="rank mono">27</td><td class="model">amazon/chronos-bolt-tiny <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T10</td><td class="num mono">0.0268</td></tr> |
| <tr><td class="rank mono">28</td><td class="model">Salesforce/moirai-1.0-R-base</td><td class="tier mono">T10</td><td class="num mono">0.0220</td></tr> |
| <tr><td class="rank mono">29</td><td class="model">NeoQuasar/Kronos-base</td><td class="tier mono">T10</td><td class="num mono">0.0207</td></tr> |
| <tr><td class="rank mono">30</td><td class="model">ibm-research/granite-timeseries-flowstate-r1</td><td class="tier mono">T10</td><td class="num mono">0.0176</td></tr> |
| <tr><td class="rank mono">31</td><td class="model">Salesforce/moirai-1.0-R-small</td><td class="tier mono">T11</td><td class="num mono neg">-0.0331</td></tr> |
| <tr><td class="rank mono">32</td><td class="model">Maple728/TimeMoE-200M <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T12</td><td class="num mono neg">-0.0452</td></tr> |
| <tr><td class="rank mono">33</td><td class="model">qcw2333/YingLong_300m</td><td class="tier mono">T12</td><td class="num mono neg">-0.0546</td></tr> |
| <tr><td class="rank mono">34</td><td class="model">mldi-lab/Kairos_10m</td><td class="tier mono">T12</td><td class="num mono neg">-0.0566</td></tr> |
| <tr><td class="rank mono">35</td><td class="model">ibm-research/granite-timeseries-ttm-v1</td><td class="tier mono">T12</td><td class="num mono neg">-0.0648</td></tr> |
| <tr><td class="rank mono">36</td><td class="model">ibm-research/granite-timeseries-ttm-r2 <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T13</td><td class="num mono neg">-0.1112</td></tr> |
| <tr><td class="rank mono">37</td><td class="model">qcw2333/YingLong_110m <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T14</td><td class="num mono neg">-0.1403</td></tr> |
| <tr><td class="rank mono">38</td><td class="model">qcw2333/YingLong_50m</td><td class="tier mono">T15</td><td class="num mono neg">-0.2224</td></tr> |
| <tr><td class="rank mono">39</td><td class="model">qcw2333/YingLong_6m</td><td class="tier mono">T16</td><td class="num mono neg">-0.3691</td></tr> |
| <tr><td class="rank mono">40</td><td class="model">time-series-foundation-models/Lag-Llama</td><td class="tier mono">T17</td><td class="num mono neg">-0.6597</td></tr> |
| <tr><td class="rank mono">41</td><td class="model">ibm-research/ttm-r3 <span class="pareto" title="Pareto-optimal" aria-label="Pareto-optimal">◆</span></td><td class="tier mono">T18</td><td class="num mono neg">-0.7236</td></tr> |
| <tr><td class="rank mono">42</td><td class="model">Salesforce/moirai-moe-1.0-R-base</td><td class="tier mono">T19</td><td class="num mono neg">-1.0577</td></tr> |
| <tr><td class="rank mono">43</td><td class="model">Salesforce/moirai-moe-1.0-R-small</td><td class="tier mono">T19</td><td class="num mono neg">-1.0644</td></tr> |
| <tr><td class="rank mono">44</td><td class="model">AutonLab/MOMENT-1-large</td><td class="tier mono">T20</td><td class="num mono neg">-1.2300</td></tr> |
| <tr><td class="rank mono">45</td><td class="model">ibm-research/granite-timeseries-patchtst-fm-r1</td><td class="tier mono">T21</td><td class="num mono neg">-1.6944</td></tr> |
| <tr><td class="rank mono">46</td><td class="model">AutonLab/MOMENT-1-small</td><td class="tier mono">T22</td><td class="num mono neg">-7.0476</td></tr> |
| <tr><td class="rank mono">47</td><td class="model">AutonLab/MOMENT-1-base</td><td class="tier mono">T23</td><td class="num mono neg">-1281.3842</td></tr> |
| </tbody> |
| </table> |
| <p class="board-note mono"><span class="pareto-key">◆</span> Pareto-optimal across budget & cost · tiers group models whose rank intervals overlap (not statistically separable).</p> |
| <div class="board-links"> |
| <a href="reports/tsfm_ai_empirical_v2_multi_budget/leaderboard.json">Leaderboard JSON</a> |
| <a href="paper/future_ts_empirical.pdf">Empirical paper</a> |
| </div> |
| </section> |
|
|
| <section id="submit" class="block"> |
| <div class="block-head"> |
| <p class="kicker mono">Model entries</p> |
| <h2>Submit through the sealed-runner path.</h2> |
| </div> |
| <ol class="steps"> |
| <li><span class="idx mono">01</span><span>Declare model identity, artifact URI, and pretraining sources.</span></li> |
| <li><span class="idx mono">02</span><span>Implement a deterministic task-window to predictions script.</span></li> |
| <li><span class="idx mono">03</span><span>Open a pull request and pass sealed-runner smoke validation.</span></li> |
| <li><span class="idx mono">04</span><span>A reviewer triggers the full run and publishes the report.</span></li> |
| </ol> |
| </section> |
|
|
| <section id="scope" class="block scope"> |
| <div class="block-head"> |
| <p class="kicker mono">Scope boundary</p> |
| <h2>Public preview — the methodology can still change.</h2> |
| <p class="block-sub"> |
| v0.1.0 is a runnable local benchmark package and sealed-runner MVP. |
| Hosted attestation, immutable submission windows, repeated live waves, |
| wider task coverage, and stronger manifest evidence are future work. |
| </p> |
| </div> |
| <div class="scope-links"> |
| <a href="paper/future_ts_design.pdf">Design paper</a> |
| <a href="paper/future_ts_empirical.pdf">Empirical paper</a> |
| <a href="docs/validity-envelope.md">Validity envelope</a> |
| <a href="docs/submission-guide.md">Submission guide</a> |
| </div> |
| </section> |
| </main> |
|
|
| <footer> |
| <span class="mono">FUTURE-TS v0.1.0</span> |
| <span class="mono">Apache-2.0</span> |
| <span class="mono">Built by <a href="https://tsfm.ai">TSFM.ai</a> · <a href="https://parisi-labs.com">Parisi Labs</a></span> |
| </footer> |
| <script src="app.js"></script> |
| </body> |
| </html> |
|
|
