src/about.py

TITLE = """ TSFM Realworld Bench Leaderboard """

INTRODUCTION_TEXT = """
**TSFM Realworld Bench** evaluates time series foundation models on **live TS-Bench
real-world data** with **zero-shot API inference** via [TSFM.ai](https://tsfm.ai/).
Following [GIFT-Eval](https://huggingface.co/spaces/Salesforce/GIFT-Eval), the leaderboard
reports **absolute metric values** and **per-dataset ranks**. The Overall tab also reports
the latest overall snapshot plus full 24-hour, 7-day, and 30-day metric tables and a
dataset-balanced **pairwise historical ranking** over all shared releases through the
current cutoff. The
**GIFT-style Aggregates** tab provides Seasonal-Naive-normalized MSE, CRPS,
and mean CRPS rank grouped by actual prediction length, domain, and frequency. Each
subsequent domain tab retains the original absolute per-dataset results.
"""

LLM_BENCHMARKS_TEXT = """
## How to participate (Submit a New Model)

Community models are evaluated through owner-operated HTTPS endpoints. Copy the
portable FastAPI template in `examples/community_endpoint/`, replace its
`forecast_one` function, and deploy it on your own inference server or existing
cloud service. A paid Hugging Face Space is optional and a Static Space cannot
run the forecasting API.

The endpoint receives only causal history, prediction length, frequency, and
requested quantiles. It never receives future labels. Servers behind NAT can use
a stable named HTTPS tunnel; temporary Quick Tunnel URLs are not accepted.

Before submitting, run `scripts/validate_external_model_endpoint.py` against the
public `/forecast` URL and generate `community_model.yaml` with
`scripts/generate_community_model_metadata.py`. Submit the public model card,
endpoint-code URL, endpoint URL, metadata, and successful validator receipt
through the community model request form. Accepted entries are evaluated only
on future live releases.

## Metrics

- **MSE** — Mean Squared Error on the mean forecast (absolute)
- **RMSE** — Root Mean Squared Error on the mean forecast
- **MAPE** — Mean Absolute Percentage Error, reported only away from zero
- **CRPS** — quantile approximation of the Continuous Ranked Probability Score
- **RTG** — normalized real-time MSE gain over causal Seasonal-Naive (higher is better)
- **Stability** — standard deviation of release-level MSE (lower is better)
- **Improvement** — Kendall trend statistic over release-level MSE (more negative is better)
- **Pairwise Win Rate** — dataset-balanced MSE/CRPS wins over shared future releases;
  official ranks require sufficient shared releases, datasets, time span, opponents,
  and membership in the main comparison component
- **RankScore** — Elo-style aggregate from per-dataset MSE and CRPS ranks (higher is better)
- **MSE_Rank** / **CRPS_Rank** — per-dataset rank (lower is better)
- **Grouped MSE / CRPS** — geometric mean after per-configuration normalization against
  Seasonal-Naive (lower is better; 1.0 equals the baseline)
- **Grouped Rank** — mean per-configuration CRPS rank (lower is better)
"""

CITATION_BUTTON_LABEL = "Copy citation"
CITATION_BUTTON_TEXT = r"""
@misc{tsfm_realworld_bench,
  title={TSFM Realworld Bench: A Benchmark for Time Series Foundation Models},
  author={TSFM Realworld Bench Team},
  year={2026},
  howpublished={\url{https://huggingface.co/spaces/CityMindDev/TSFM-Realworld-Bench}}
}
"""