Solves 500 Errors For Some Users

.gitattributes

@@ -25,7 +25,6 @@
 *.safetensors filter=lfs diff=lfs merge=lfs -text
 saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
 *.wasm filter=lfs diff=lfs merge=lfs -text
@@ -33,3 +32,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+scale-hf-logo.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -1 +1,13 @@
-__pycache__
+auto_evals/
+venv/
+__pycache__/
+.env
+.ipynb_checkpoints
+*ipynb
+.vscode/
+
+eval-queue/
+eval-results/
+eval-queue-bk/
+eval-results-bk/
+logs/

.pre-commit-config.yaml

@@ -0,0 +1,53 @@
+# Copyright (c) 2022, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+default_language_version:
+  python: python3
+
+ci:
+  autofix_prs: true
+  autoupdate_commit_msg: '[pre-commit.ci] pre-commit suggestions'
+  autoupdate_schedule: quarterly
+
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.3.0
+    hooks:
+      - id: check-yaml
+      - id: check-case-conflict
+      - id: detect-private-key
+      - id: check-added-large-files
+        args: ['--maxkb=1000']
+      - id: requirements-txt-fixer
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+
+  - repo: https://github.com/PyCQA/isort
+    rev: 5.12.0
+    hooks:
+      - id: isort
+        name: Format imports
+
+  - repo: https://github.com/psf/black
+    rev: 22.12.0
+    hooks:
+      - id: black
+        name: Format code
+        additional_dependencies: ['click==8.0.2']
+
+  - repo: https://github.com/charliermarsh/ruff-pre-commit
+    # Ruff version.
+    rev: 'v0.0.267'
+    hooks:
+      - id: ruff

.streamlit/config.toml

@@ -1,2 +0,0 @@
-[theme]
-base = "light"

CLAUDE.md

@@ -1,82 +0,0 @@
-# CLAUDE.md
-
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
-
-## Project Overview
-
-fev-bench Leaderboard is a Streamlit web application displaying time series forecasting model evaluation results from the fev-bench benchmark. It evaluates 30+ forecasting models using multiple metrics (SQL, MASE, WQL, WAPE) across 100 benchmark tasks.
-
-## Common Commands
-
-```bash
-# Run the Streamlit app locally
-uv run streamlit run fev-leaderboard-app.py --server.port=8501 --server.address=0.0.0.0
-
-# Regenerate leaderboard tables from autogluon/fev repo (defaults to main branch)
-uv run python save_tables.py [commit]  # e.g., uv run python save_tables.py abc123
-
-# Docker build and run
-docker build -t fev-leaderboard .
-docker run -p 8501:8501 fev-leaderboard
-```
-
-Note: Use `uv run` prefix for all Python commands in this project.
-
-No test or lint frameworks are configured.
-
-## Architecture
-
-```
-fev-leaderboard-app.py     # Main entry point (Streamlit multi-page router)
-save_tables.py             # Generates pre-computed CSV tables from raw summaries
-pages/
-├── fev_bench.py           # Main leaderboard (100 tasks, loads from tables/)
-├── chronos_bench_ii.py    # Alternative leaderboard (27 tasks, fetches from GitHub)
-└── about.py               # Help page with links
-src/
-├── utils.py               # Visualization, formatting, MODEL_CONFIG, color palette
-├── strings.py             # UI text, metric descriptions, paper citations
-└── task_groups.py         # Task groupings by frequency and domain
-tables/                    # Pre-generated CSVs
-├── pivot_*.csv            # Full pivot tables (filtered in app by task group)
-├── summaries.csv          # Raw evaluation summaries
-└── {group}/               # Subdirectories for each task group (full, mini, frequency_*, domain_*)
-    ├── leaderboard_*.csv  # Leaderboard tables per metric
-    └── pairwise_*.csv     # Pairwise comparison tables per metric
-```
-
-**Data flow**: GitHub (autogluon/fev) → `save_tables.py` → pre-computed tables → `fev_bench.py` visualization
-
-## Key Modules
-
-**`src/utils.py`**: Core module containing:
-- `MODEL_CONFIG`: Dict mapping model names to (huggingface_url, organization, is_zero_shot, model_type)
-- `ALL_METRICS`: Dict with SQL, MASE, WQL, WAPE definitions
-- `format_leaderboard()`, `construct_bar_chart()`, `construct_pairwise_chart()`, `construct_pivot_table()`: Styling functions
-- `COLORS`: Custom palette (purple, gold, silver, bronze)
-
-**`src/strings.py`**: Documentation strings for metric formulas, win rate/skill score calculations, imputation strategies
-
-## Metrics
-
-| Metric | Type | Description |
-|--------|------|-------------|
-| SQL | Probabilistic | Scaled Quantile Loss (scale-invariant) |
-| MASE | Point | Mean Absolute Scaled Error (scale-invariant) |
-| WQL | Probabilistic | Weighted Quantile Loss (scale-dependent) |
-| WAPE | Point | Weighted Absolute Percentage Error (scale-dependent) |
-
-## Model Types
-
-Models are categorized as DL (deep learning) or ST (statistical) in `MODEL_CONFIG`. This affects color-coding in visualizations (blue vs. orange).
-
-## Imputation Strategy
-
-- **Failed tasks**: Replaced with Seasonal Naive scores
-- **Leaky tasks** (training corpus overlap for zero-shot models): Replaced with Chronos-Bolt scores
-
-## External References
-
-- fev-bench paper: https://arxiv.org/abs/2509.26468
-- fev library docs: https://autogluon.github.io/fev/latest/
-- GitHub: https://github.com/autogluon/fev

Dockerfile

@@ -1,21 +0,0 @@
-FROM python:3.13.5-slim
-
-RUN useradd -m -u 1000 user
-WORKDIR /app
-
-RUN apt-get update && apt-get install -y \
-    build-essential \
-    curl \
-    git \
-    && rm -rf /var/lib/apt/lists/*
-
-COPY --chown=user ./requirements.txt requirements.txt
-COPY --chown=user . /app
-
-RUN pip3 install -r requirements.txt
-
-EXPOSE 8501
-
-HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health
-
-ENTRYPOINT ["streamlit", "run", "fev-leaderboard-app.py", "--server.port=8501", "--server.address=0.0.0.0"]

Makefile

@@ -0,0 +1,13 @@
+.PHONY: style format
+
+
+style:
+	python -m black --line-length 119 .
+	python -m isort .
+	ruff check --fix .
+
+
+quality:
+	python -m black --check --line-length 119 .
+	python -m isort --check-only .
+	ruff check .

README.md

@@ -1,20 +1,44 @@
 ---
-title: fev-bench
-emoji: 🏆
+title: Fev Leaderboard
+emoji: 🥇
 colorFrom: green
 colorTo: indigo
-sdk: docker
-app_port: 8501
-tags:
-- streamlit
-pinned: false
-short_description: Forecast evaluation benchmark
+sdk: gradio
+app_file: app.py
+pinned: true
 license: apache-2.0
 ---
 
-# Welcome to Streamlit!
+# Start the configuration
 
-Edit `/src/streamlit_app.py` to customize this app to your heart's desire. :heart:
+Most of the variables to change for a default leaderboard are in `src/env.py` (replace the path for your leaderboard) and `src/about.py` (for tasks).
 
-If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
-forums](https://discuss.streamlit.io).
+Results files should have the following format and be stored as json files:
+```json
+{
+    "config": {
+        "model_dtype": "torch.float16", # or torch.bfloat16 or 8bit or 4bit
+        "model_name": "path of the model on the hub: org/model",
+        "model_sha": "revision on the hub",
+    },
+    "results": {
+        "task_name": {
+            "metric_name": score,
+        },
+        "task_name2": {
+            "metric_name": score,
+        }
+    }
+}
+```
+
+Request files are created automatically by this tool.
+
+If you encounter problem on the space, don't hesitate to restart it to remove the create eval-queue, eval-queue-bk, eval-results and eval-results-bk created folder.
+
+# Code logic for more complex edits
+
+You'll find 
+- the main table' columns names and properties in `src/display/utils.py`
+- the logic to read all results and request files, then convert them in dataframe lines, in `src/leaderboard/read_evals.py`, and `src/populate.py`
+- the logic to allow or filter submissions in `src/submission/submit.py` and `src/submission/check_validity.py`

pages/chronos_bench_ii.py → app.py

@@ -1,41 +1,18 @@
-import sys
-from pathlib import Path
-
-sys.path.append(str(Path(__file__).parent.parent))
-
 import fev
+import gradio as gr
 import pandas as pd
-import streamlit as st
-from streamlit.elements.lib.column_types import ColumnConfig
-
-from src.strings import (
-    CHRONOS_BENCHMARK_BASIC_INFO,
-    CHRONOS_BENCHMARK_DETAILS,
-    CITATION_CHRONOS,
-    CITATION_FEV,
-    CITATION_HEADER,
-    PAIRWISE_BENCHMARK_DETAILS,
-    get_pivot_legend,
-)
-from src.utils import (
-    construct_bar_chart,
-    construct_pairwise_chart,
-    construct_pivot_table,
-    format_leaderboard,
-    format_metric_name,
-    get_metric_description,
+
+from src import about
+from src.custom_html_js import custom_css
+from src.formatting import make_clickable_model
+
+# Load the CSV data into a pandas DataFrame
+df = pd.read_csv(
+    "https://raw.githubusercontent.com/autogluon/fev/refs/heads/main/benchmarks/chronos_zeroshot/results/seasonal_naive.csv"
 )
 
-st.set_page_config(layout="wide", page_title="FEV Benchmark Leaderboard", page_icon=":material/trophy:")
 
-TITLE = "<h1 style='text-align: center; font-size: 350%;'>Chronos Benchmark II</h1>"
-BASELINE_MODEL = "seasonal_naive"
-LEAKAGE_IMPUTATION_MODEL = "chronos_bolt_base"
-SORT_COL = "win_rate"
-N_RESAMPLES_FOR_CI = 1000
-TOP_K_MODELS_TO_PLOT = 15
-AVAILABLE_METRICS = ["WQL", "MASE"]
-SUMMARY_URLS = [
+summary_urls = [
     "https://raw.githubusercontent.com/autogluon/fev/refs/heads/main/benchmarks/chronos_zeroshot/results/auto_arima.csv",
     "https://raw.githubusercontent.com/autogluon/fev/refs/heads/main/benchmarks/chronos_zeroshot/results/auto_ets.csv",
     "https://raw.githubusercontent.com/autogluon/fev/refs/heads/main/benchmarks/chronos_zeroshot/results/auto_theta.csv",
@@ -58,122 +35,63 @@ SUMMARY_URLS = [
     "https://raw.githubusercontent.com/autogluon/fev/refs/heads/main/benchmarks/chronos_zeroshot/results/tirex.csv",
 ]
 
-
-@st.cache_data()
-def load_summaries():
-    summaries = []
-    for url in SUMMARY_URLS:
-        df = pd.read_csv(url)
-        summaries.append(df)
-    return pd.concat(summaries, ignore_index=True)
-
-
-@st.cache_data()
-def get_leaderboard(metric_name: str) -> pd.DataFrame:
-    summaries = load_summaries()
-    lb = fev.analysis.leaderboard(
-        summaries=summaries,
-        metric_column=metric_name,
-        missing_strategy="impute",
-        baseline_model=BASELINE_MODEL,
-        leakage_imputation_model=LEAKAGE_IMPUTATION_MODEL,
-    )
-    lb = lb.astype("float64").reset_index()
-
-    lb["skill_score"] = lb["skill_score"] * 100
-    lb["win_rate"] = lb["win_rate"] * 100
-    lb["num_failures"] = lb["num_failures"] / summaries["task_name"].nunique() * 100
-    return lb
-
-
-@st.cache_data()
-def get_pairwise(metric_name: str, included_models: list[str]) -> pd.DataFrame:
-    if BASELINE_MODEL not in included_models:
-        included_models = included_models + [BASELINE_MODEL]
-    summaries = load_summaries()
-    return (
-        fev.analysis.pairwise_comparison(
-            summaries,
-            included_models=included_models,
-            metric_column=metric_name,
-            baseline_model=BASELINE_MODEL,
-            missing_strategy="impute",
-            n_resamples=N_RESAMPLES_FOR_CI,
-            leakage_imputation_model=LEAKAGE_IMPUTATION_MODEL,
-        )
-        .round(3)
-        .reset_index()
-    )
-
-
-with st.sidebar:
-    selected_metric = st.selectbox("Evaluation Metric", options=AVAILABLE_METRICS, format_func=format_metric_name)
-    st.caption(get_metric_description(selected_metric))
-
-cols = st.columns(spec=[0.025, 0.95, 0.025])
-
-with cols[1] as main_container:
-    st.markdown(TITLE, unsafe_allow_html=True)
-
-    metric_df = get_leaderboard(selected_metric).sort_values(by=SORT_COL, ascending=False)
-    top_k_models = metric_df.head(TOP_K_MODELS_TO_PLOT)["model_name"].tolist()
-    pairwise_df = get_pairwise(selected_metric, included_models=top_k_models)
-
-    st.markdown("## :material/trophy: Leaderboard", unsafe_allow_html=True)
-    st.markdown(CHRONOS_BENCHMARK_BASIC_INFO, unsafe_allow_html=True)
-    df_styled = format_leaderboard(metric_df)
-    st.dataframe(
-        df_styled,
-        use_container_width=True,
-        hide_index=True,
-        column_config={
-            "model_name": ColumnConfig(label="Model Name", alignment="left"),
-            "win_rate": st.column_config.NumberColumn(label="Avg. win rate (%)", format="%.1f"),
-            "skill_score": st.column_config.NumberColumn(label="Skill score (%)", format="%.1f"),
-            "median_inference_time_s": st.column_config.NumberColumn(label="Median runtime (s)", format="%.1f"),
-            "training_corpus_overlap": st.column_config.NumberColumn(label="Leakage (%)", format="%d"),
-            "num_failures": st.column_config.NumberColumn(label="Failed tasks (%)", format="%.0f"),
-            "zero_shot": ColumnConfig(label="Zero-shot", alignment="center"),
-            "org": ColumnConfig(label="Organization", alignment="left"),
-            "link": st.column_config.LinkColumn(label="Link", display_text=":material/open_in_new:"),
-        },
-    )
-
-    with st.expander("See details"):
-        st.markdown(CHRONOS_BENCHMARK_DETAILS, unsafe_allow_html=True)
-
-    st.markdown("## :material/bar_chart: Pairwise comparison", unsafe_allow_html=True)
-    chart_col_1, _, chart_col_2 = st.columns(spec=[0.45, 0.1, 0.45])
-
-    with chart_col_1:
-        st.altair_chart(
-            construct_pairwise_chart(pairwise_df, col="win_rate", metric_name=selected_metric),
-            use_container_width=True,
-        )
-
-    with chart_col_2:
-        st.altair_chart(
-            construct_pairwise_chart(pairwise_df, col="skill_score", metric_name=selected_metric),
-            use_container_width=True,
-        )
-
-    with st.expander("See details"):
-        st.markdown(PAIRWISE_BENCHMARK_DETAILS, unsafe_allow_html=True)
-
-    st.markdown("## :material/table_chart: Results for individual tasks", unsafe_allow_html=True)
-    with st.expander("Show detailed results"):
-        st.markdown(get_pivot_legend(BASELINE_MODEL, LEAKAGE_IMPUTATION_MODEL), unsafe_allow_html=True)
-        st.dataframe(
-            construct_pivot_table(
-                summaries=load_summaries(),
-                metric_name=selected_metric,
-                baseline_model=BASELINE_MODEL,
-                leakage_imputation_model=LEAKAGE_IMPUTATION_MODEL,
-            )
-        )
-
-    st.divider()
-    st.markdown("### :material/format_quote: Citation", unsafe_allow_html=True)
-    st.markdown(CITATION_HEADER)
-    st.markdown(CITATION_FEV)
-    st.markdown(CITATION_CHRONOS)
+rename_cols = {
+    "gmean_relative_error": "Average relative error",
+    "avg_rank": "Average rank",
+    "median_inference_time_s": "Median inference time (s)",
+    "training_corpus_overlap": "Training corpus overlap (%)",
+}
+selected_cols = list(rename_cols.keys())
+
+
+def highlight_zeroshot(styler):
+    """Highlight training overlap for zero-shot models with bold green."""
+
+    def style_func(val):
+        if val == 0:
+            return "color: green; font-weight: bold"
+        else:
+            return "color: black"
+
+    return styler.map(style_func, subset=["Training corpus overlap (%)"])
+
+
+leaderboards = {}
+for metric in ["WQL", "MASE"]:
+    lb = fev.leaderboard(summary_urls, metric_column=metric)[selected_cols].rename(columns=rename_cols)
+    lb = lb.astype("float64").round(3).reset_index()
+    lb["Training corpus overlap (%)"] = (lb["Training corpus overlap (%)"] * 100).round(1)
+    lb["model_name"] = lb["model_name"].apply(make_clickable_model)
+    leaderboards[metric] = highlight_zeroshot(lb.style).format(precision=3)
+
+
+with gr.Blocks(css=custom_css) as demo:
+    gr.HTML(about.TITLE)
+    gr.Markdown(about.INTRODUCTION_TEXT, elem_classes="markdown-text")
+
+    with gr.Tabs(elem_classes="tab-buttons"):
+        with gr.Tab("🏅 Chronos Benchmark II", id=0):
+            with gr.Column():
+                gr.Markdown(about.CHRONOS_BENCHMARK, elem_classes="markdown-text")
+            with gr.Tabs():
+                with gr.Tab("📊 Probabilistic forecast (WQL)"):
+                    gr.Markdown("""Forecast accuracy measured by Weighted Quantile Loss.""")
+                    gr.Dataframe(
+                        value=leaderboards["WQL"],
+                        datatype=["markdown", "number", "number", "number"],
+                        interactive=False,
+                    )
+
+                with gr.Tab("📈 Point forecast (MASE)"):
+                    gr.Markdown("""Forecast accuracy measured by Mean Absolute Scaled Error.""")
+                    gr.Dataframe(
+                        value=leaderboards["MASE"],
+                        datatype=["markdown", "number", "number", "number"],
+                        interactive=False,
+                    )
+
+        with gr.Tab("📝 About", id=1):
+            gr.Markdown(about.ABOUT_LEADERBOARD)
+
+if __name__ == "__main__":
+    demo.launch(ssr_mode=False)

fev-leaderboard-app.py

@@ -1,9 +0,0 @@
-import streamlit as st
-
-pages = [
-    st.Page("pages/fev_bench.py", title="fev-bench", icon=":material/trophy:"),
-    st.Page("pages/about.py", title="About", icon=":material/info:"),
-]
-
-page = st.navigation(pages)
-page.run()

pages/about.py

@@ -1,19 +0,0 @@
-import streamlit as st
-
-ABOUT_LEADERBOARD = """
-## About
-
-[**fev**](https://github.com/autogluon/fev) is a lightweight wrapper around the 🤗 [datasets](https://huggingface.co/docs/datasets/en/index) library designed to streamline
- benchmarking of time series forecasting models.
-
-### 📚 Resources
-- **Documentation**: [Official docs](https://autogluon.github.io/fev/latest/)
-- **Publication**: ["fev-bench: A Realistic Benchmark for Time Series Forecasting"](https://arxiv.org/abs/2509.26468)
-- **Source Code**: [GitHub repository](https://github.com/autogluon/fev)
-- **Issues & Questions**: [GitHub Issues](https://github.com/autogluon/fev/issues)
-
-### 🚀 Submit Your Model
-Ready to add your model to the leaderboard? Follow this [tutorial](https://autogluon.github.io/fev/latest/tutorials/05-add-your-model/) to evaluate your model with fev and contribute your results.
-"""
-st.set_page_config(layout="wide", page_title="About FEV", page_icon=":material/info:")
-st.markdown(ABOUT_LEADERBOARD)

pages/fev_bench.py

@@ -1,219 +0,0 @@
-import sys
-from pathlib import Path
-
-sys.path.append(str(Path(__file__).parent))
-
-import fev
-import pandas as pd
-import streamlit as st
-from streamlit.elements.lib.column_types import ColumnConfig
-
-from src.strings import (
-    CITATION_FEV,
-    CITATION_HEADER,
-    FEV_BENCHMARK_DETAILS,
-    PAIRWISE_BENCHMARK_DETAILS,
-    get_pivot_legend,
-)
-from src.task_groups import (
-    ALL_TASKS,
-    DOMAIN_GROUPS,
-    FREQUENCY_GROUPS,
-    MINI_TASKS,
-    get_task_group,
-)
-from src.utils import (
-    COLORS,
-    construct_pairwise_chart,
-    format_leaderboard,
-    format_metric_name,
-    get_metric_description,
-)
-
-st.set_page_config(layout="wide", page_title="fev leaderboard", page_icon=":material/trophy:")
-
-TITLE = "<h1 style='text-align: center; font-size: 350%;'>fev-bench</h1>"
-SORT_COL = "win_rate"
-AVAILABLE_METRICS = ["SQL", "MASE", "WQL", "WAPE"]
-
-# Group type options
-GROUP_TYPES = ["Full (100 tasks)", "Mini (20 tasks)", "By frequency", "By domain"]
-FREQUENCY_OPTIONS = list(FREQUENCY_GROUPS.keys())
-DOMAIN_OPTIONS = list(DOMAIN_GROUPS.keys())
-
-def get_subset_description(group_type: str, subgroup: str | None, num_tasks: int) -> str:
-    """Generate a description of the current subset."""
-    base = f"Results for various forecasting models on **{num_tasks} tasks**"
-    if group_type == "Full (100 tasks)":
-        subset_desc = "from the full **fev-bench** benchmark"
-    elif group_type == "Mini (20 tasks)":
-        subset_desc = "from the **fev-bench-mini** subset"
-    elif group_type == "By frequency":
-        subset_desc = f"with **{subgroup.lower()}** frequency"
-    else:  # By domain
-        subset_desc = f"from the **{subgroup}** domain"
-    paper_link = "[fev-bench: A Realistic Benchmark for Time Series Forecasting](https://arxiv.org/abs/2509.26468)"
-    return f"{base} {subset_desc}, as described in the paper {paper_link}."
-
-
-# Mapping from UI selections to table directory names
-GROUP_DIR_MAPPING = {
-    "Full (100 tasks)": "full",
-    "Mini (20 tasks)": "mini",
-    "Sub-hourly": "frequency_sub_hourly",
-    "Hourly": "frequency_hourly",
-    "Daily": "frequency_daily",
-    "Weekly": "frequency_weekly",
-    "Monthly+": "frequency_monthly_plus",
-    "Energy": "domain_energy",
-    "Nature": "domain_nature",
-    "Cloud": "domain_cloud",
-    "Mobility": "domain_mobility",
-    "Econ": "domain_econ",
-    "Health": "domain_health",
-    "Retail": "domain_retail",
-}
-
-
-@st.cache_data()
-def get_leaderboard(metric_name: str, group_dir: str) -> pd.DataFrame:
-    return pd.read_csv(f"tables/{group_dir}/leaderboard_{metric_name}.csv")
-
-
-@st.cache_data()
-def get_pairwise(metric_name: str, group_dir: str) -> pd.DataFrame:
-    return pd.read_csv(f"tables/{group_dir}/pairwise_{metric_name}.csv")
-
-
-@st.cache_data()
-def get_pivot_table(metric_name: str) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
-    pivot_df = pd.read_csv(f"tables/pivot_{metric_name}.csv")
-    baseline_imputed = pd.read_csv(f"tables/pivot_{metric_name}_baseline_imputed.csv")
-    leakage_imputed = pd.read_csv(f"tables/pivot_{metric_name}_leakage_imputed.csv")
-    return pivot_df, baseline_imputed, leakage_imputed
-
-
-with st.sidebar:
-    # Task group selection
-    selected_group_type = st.selectbox("Subset", options=GROUP_TYPES)
-
-    # Conditional sub-selection for frequency/domain
-    selected_subgroup = None
-    if selected_group_type == "By frequency":
-        selected_subgroup = st.selectbox("Frequency", options=FREQUENCY_OPTIONS)
-    elif selected_group_type == "By domain":
-        selected_subgroup = st.selectbox("Domain", options=DOMAIN_OPTIONS)
-
-    # Determine the directory to load tables from
-    if selected_group_type in ["Full (100 tasks)", "Mini (20 tasks)"]:
-        group_dir = GROUP_DIR_MAPPING[selected_group_type]
-        task_list = ALL_TASKS if selected_group_type == "Full (100 tasks)" else MINI_TASKS
-    else:
-        group_dir = GROUP_DIR_MAPPING[selected_subgroup]
-        if selected_group_type == "By frequency":
-            task_list = FREQUENCY_GROUPS[selected_subgroup]
-        else:
-            task_list = DOMAIN_GROUPS[selected_subgroup]
-
-    st.caption(f"{len(task_list)} tasks")
-
-    st.divider()
-
-    selected_metric = st.selectbox("Evaluation Metric", options=AVAILABLE_METRICS, format_func=format_metric_name)
-    st.caption(get_metric_description(selected_metric))
-
-cols = st.columns(spec=[0.025, 0.95, 0.025])
-
-with cols[1] as main_container:
-    st.markdown(TITLE, unsafe_allow_html=True)
-
-    metric_df = get_leaderboard(selected_metric, group_dir).sort_values(by=SORT_COL, ascending=False)
-    pairwise_df = get_pairwise(selected_metric, group_dir)
-
-    st.markdown("## :material/trophy: Leaderboard", unsafe_allow_html=True)
-    st.markdown(get_subset_description(selected_group_type, selected_subgroup, len(task_list)), unsafe_allow_html=True)
-    df_styled = format_leaderboard(metric_df)
-    st.dataframe(
-        df_styled,
-        width="stretch",
-        hide_index=True,
-        column_config={
-            "model_name": ColumnConfig(label="Model Name", alignment="left"),
-            "win_rate": st.column_config.NumberColumn(label="Avg. win rate (%)", format="%.1f"),
-            "skill_score": st.column_config.NumberColumn(label="Skill score (%)", format="%.1f"),
-            "median_inference_time_s_per100": st.column_config.NumberColumn(label="Median runtime (s / 100 series)", format="%.1f"),
-            "training_corpus_overlap": st.column_config.NumberColumn(label="Leakage (%)", format="%d"),
-            "num_failures": st.column_config.NumberColumn(label="Failed tasks (%)", format="%.0f"),
-            "zero_shot": ColumnConfig(label="Zero-shot", alignment="center"),
-            "org": ColumnConfig(label="Organization", alignment="left"),
-            "link": st.column_config.LinkColumn(label="Link", display_text="🔗"),
-        },
-    )
-
-    with st.expander("See details"):
-        st.markdown(FEV_BENCHMARK_DETAILS, unsafe_allow_html=True)
-
-    st.markdown("## :material/bar_chart: Pairwise comparison", unsafe_allow_html=True)
-    chart_col_1, _, chart_col_2 = st.columns(spec=[0.45, 0.1, 0.45])
-
-    with chart_col_1:
-        st.altair_chart(
-            construct_pairwise_chart(pairwise_df, col="win_rate", metric_name=selected_metric),
-            use_container_width=True,
-        )
-
-    with chart_col_2:
-        st.altair_chart(
-            construct_pairwise_chart(pairwise_df, col="skill_score", metric_name=selected_metric),
-            use_container_width=True,
-        )
-
-    with st.expander("See details"):
-        st.markdown(PAIRWISE_BENCHMARK_DETAILS, unsafe_allow_html=True)
-
-    st.markdown("## :material/table_chart: Results for individual tasks", unsafe_allow_html=True)
-    with st.expander("Show detailed results"):
-        st.markdown(get_pivot_legend("Seasonal Naive", "Chronos-Bolt"), unsafe_allow_html=True)
-        pivot_df, baseline_imputed, leakage_imputed = get_pivot_table(selected_metric)
-        pivot_df = pivot_df.set_index("Task name")
-        baseline_imputed = baseline_imputed.set_index("Task name")
-        leakage_imputed = leakage_imputed.set_index("Task name")
-
-        # Filter pivot table to only show tasks in the selected group
-        available_tasks = [t for t in task_list if t in pivot_df.index]
-        pivot_df = pivot_df.loc[available_tasks]
-        baseline_imputed = baseline_imputed.loc[available_tasks]
-        leakage_imputed = leakage_imputed.loc[available_tasks]
-
-        def style_pivot_table(errors, is_baseline_imputed, is_leakage_imputed):
-            rank_colors = {1: COLORS["gold"], 2: COLORS["silver"], 3: COLORS["bronze"]}
-
-            def highlight_by_position(styler):
-                for row_idx in errors.index:
-                    row_ranks = errors.loc[row_idx].rank(method="min")
-                    for col_idx in errors.columns:
-                        rank = row_ranks[col_idx]
-                        style_parts = []
-                        if rank <= 3:
-                            style_parts.append(f"background-color: {rank_colors[rank]}")
-                        if is_leakage_imputed.loc[row_idx, col_idx]:
-                            style_parts.append(f"color: {COLORS['leakage_impute']}")
-                        elif is_baseline_imputed.loc[row_idx, col_idx]:
-                            style_parts.append(f"color: {COLORS['failure_impute']}")
-                        elif not style_parts:
-                            style_parts.append(f"color: {COLORS['text_default']}")
-                        if style_parts:
-                            styler = styler.map(
-                                lambda x, s="; ".join(style_parts): s,
-                                subset=pd.IndexSlice[row_idx:row_idx, col_idx:col_idx],
-                            )
-                return styler
-
-            return highlight_by_position(errors.style).format(precision=3)
-
-        st.dataframe(style_pivot_table(pivot_df, baseline_imputed, leakage_imputed))
-
-    st.divider()
-    st.markdown("### :material/format_quote: Citation", unsafe_allow_html=True)
-    st.markdown(CITATION_HEADER)
-    st.markdown(CITATION_FEV)

pyproject.toml

@@ -1,12 +1,13 @@
-[project]
-name = "hf-leaderboard"
-version = "0.1.0"
-requires-python = ">=3.11"
-dependencies = [
-    "altair>=6.0.0",
-    "fev>=0.7.0",
-    "numpy<2.2",
-    "pyarrow<21",
-    "scipy<1.15",
-    "streamlit>=1.53.1",
-]
+[tool.ruff]
+# Enable pycodestyle (`E`) and Pyflakes (`F`) codes by default.
+select = ["E", "F"]
+ignore = ["E501"] # line too long (black is taking care of this)
+line-length = 119
+fixable = ["A", "B", "C", "D", "E", "F", "G", "I", "N", "Q", "S", "T", "W", "ANN", "ARG", "BLE", "COM", "DJ", "DTZ", "EM", "ERA", "EXE", "FBT", "ICN", "INP", "ISC", "NPY", "PD", "PGH", "PIE", "PL", "PT", "PTH", "PYI", "RET", "RSE", "RUF", "SIM", "SLF", "TCH", "TID", "TRY", "UP", "YTT"]
+
+[tool.isort]
+profile = "black"
+line_length = 119
+
+[tool.black]
+line-length = 119

requirements.txt

@@ -1,7 +1,11 @@
+APScheduler
+black
+datasets
+gradio
+gradio[oauth]
+gradio_client
+huggingface-hub>=0.18.0
 matplotlib
 numpy
 pandas
-requests
-streamlit==1.49.1
-fev>=0.6.0
-altair>=5.5.0
+fev==0.4.0

save_tables.py

@@ -1,212 +0,0 @@
-#!/usr/bin/env python3
-
-import argparse
-import io
-import sys
-from pathlib import Path
-
-import requests
-
-sys.path.append(str(Path(__file__).parent))
-
-import fev
-import pandas as pd
-
-from src.task_groups import ALL_TASKS, DOMAIN_GROUPS, FREQUENCY_GROUPS, MINI_TASKS
-from src.utils import format_leaderboard
-
-GITHUB_REPO = "autogluon/fev"
-RESULTS_PATH = "benchmarks/fev_bench/results"
-
-# Constants from the main app
-BASELINE_MODEL = "Seasonal Naive"
-LEAKAGE_IMPUTATION_MODEL = "Chronos-Bolt"
-SORT_COL = "win_rate"
-N_RESAMPLES_FOR_CI = 1000
-TOP_K_MODELS_TO_PLOT = 15
-AVAILABLE_METRICS = ["SQL", "MASE", "WQL", "WAPE"]
-
-# All task groups to generate tables for
-TASK_GROUPS = {
-    "full": ALL_TASKS,
-    "mini": MINI_TASKS,
-    "frequency_sub_hourly": FREQUENCY_GROUPS["Sub-hourly"],
-    "frequency_hourly": FREQUENCY_GROUPS["Hourly"],
-    "frequency_daily": FREQUENCY_GROUPS["Daily"],
-    "frequency_weekly": FREQUENCY_GROUPS["Weekly"],
-    "frequency_monthly_plus": FREQUENCY_GROUPS["Monthly+"],
-    "domain_energy": DOMAIN_GROUPS["Energy"],
-    "domain_nature": DOMAIN_GROUPS["Nature"],
-    "domain_cloud": DOMAIN_GROUPS["Cloud"],
-    "domain_mobility": DOMAIN_GROUPS["Mobility"],
-    "domain_econ": DOMAIN_GROUPS["Econ"],
-    "domain_health": DOMAIN_GROUPS["Health"],
-    "domain_retail": DOMAIN_GROUPS["Retail"],
-}
-
-
-def get_csv_files_from_github(commit: str) -> list[str]:
-    """Get list of CSV file paths from the GitHub repo at a specific commit."""
-    api_url = f"https://api.github.com/repos/{GITHUB_REPO}/contents/{RESULTS_PATH}?ref={commit}"
-    response = requests.get(api_url)
-    response.raise_for_status()
-
-    files = response.json()
-    csv_files = [f["path"] for f in files if f["name"].endswith(".csv")]
-
-    if not csv_files:
-        raise FileNotFoundError(f"No CSV files found in {RESULTS_PATH} at commit {commit}")
-
-    return csv_files
-
-
-def load_summaries_from_github(commit: str) -> pd.DataFrame:
-    """Load and concatenate all CSV summaries from the GitHub repo at a specific commit."""
-    csv_files = get_csv_files_from_github(commit)
-    print(f"Found {len(csv_files)} CSV files")
-
-    dfs = []
-    for file_path in csv_files:
-        raw_url = f"https://raw.githubusercontent.com/{GITHUB_REPO}/{commit}/{file_path}"
-        response = requests.get(raw_url)
-        response.raise_for_status()
-        df = pd.read_csv(io.StringIO(response.text))
-        dfs.append(df)
-        print(f"  Loaded: {Path(file_path).name}")
-
-    return pd.concat(dfs, ignore_index=True)
-
-
-def compute_leaderboard(summaries: pd.DataFrame, metric_name: str) -> pd.DataFrame:
-    lb = fev.analysis.leaderboard(
-        summaries=summaries,
-        metric_column=metric_name,
-        missing_strategy="impute",
-        baseline_model=BASELINE_MODEL,
-        leakage_imputation_model=LEAKAGE_IMPUTATION_MODEL,
-        normalize_time_per_n_forecasts=100,
-    )
-    lb = lb.astype("float64").reset_index()
-
-    lb["skill_score"] = lb["skill_score"] * 100
-    lb["win_rate"] = lb["win_rate"] * 100
-    lb["num_failures"] = lb["num_failures"] / summaries["task_name"].nunique() * 100
-    return lb
-
-
-def compute_pairwise(summaries: pd.DataFrame, metric_name: str, included_models: list[str]) -> pd.DataFrame:
-    if BASELINE_MODEL not in included_models:
-        included_models = included_models + [BASELINE_MODEL]
-
-    return (
-        fev.analysis.pairwise_comparison(
-            summaries,
-            included_models=included_models,
-            metric_column=metric_name,
-            baseline_model=BASELINE_MODEL,
-            missing_strategy="impute",
-            n_resamples=N_RESAMPLES_FOR_CI,
-            leakage_imputation_model=LEAKAGE_IMPUTATION_MODEL,
-        )
-        .round(3)
-        .reset_index()
-    )
-
-
-def compute_pivot_table(summaries: pd.DataFrame, metric_name: str) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
-    errors = fev.pivot_table(summaries=summaries, metric_column=metric_name, task_columns=["task_name"])
-    train_overlap = (
-        fev.pivot_table(summaries=summaries, metric_column="trained_on_this_dataset", task_columns=["task_name"])
-        .fillna(False)
-        .astype(bool)
-    )
-
-    is_imputed_baseline = errors.isna()
-    is_leakage_imputed = train_overlap
-
-    # Handle imputations
-    errors = errors.mask(train_overlap, errors[LEAKAGE_IMPUTATION_MODEL], axis=0)
-    for col in errors.columns:
-        if col != BASELINE_MODEL:
-            errors[col] = errors[col].fillna(errors[BASELINE_MODEL])
-
-    errors = errors[errors.rank(axis=1).mean().sort_values().index]
-    is_imputed_baseline = is_imputed_baseline[errors.columns]
-    is_leakage_imputed = is_leakage_imputed[errors.columns]
-
-    errors.index.rename("Task name", inplace=True)
-    is_imputed_baseline.index.rename("Task name", inplace=True)
-    is_leakage_imputed.index.rename("Task name", inplace=True)
-
-    return errors.reset_index(), is_imputed_baseline.reset_index(), is_leakage_imputed.reset_index()
-
-
-def main():
-    parser = argparse.ArgumentParser(description="Generate leaderboard tables from CSV summaries in the fev repo")
-    parser.add_argument(
-        "commit",
-        nargs="?",
-        default="main",
-        help=f"Git commit SHA or branch name in the {GITHUB_REPO} repository (default: main)",
-    )
-    args = parser.parse_args()
-
-    # Create tables directory
-    tables_dir = Path("tables")
-    tables_dir.mkdir(exist_ok=True)
-
-    print(f"Loading summaries from {GITHUB_REPO} at commit {args.commit}...")
-    summaries = load_summaries_from_github(args.commit)
-
-    # Save raw summaries for on-the-fly subset computation
-    summaries.to_csv(tables_dir / "summaries.csv", index=False)
-    print("Saved: summaries.csv")
-
-    # Generate pivot tables (full version only, at root level)
-    for metric in AVAILABLE_METRICS:
-        print(f"Processing pivot table for {metric}...")
-        pivot_df, baseline_imputed, leakage_imputed = compute_pivot_table(summaries, metric)
-        pivot_df.to_csv(tables_dir / f"pivot_{metric}.csv", index=False)
-        baseline_imputed.to_csv(tables_dir / f"pivot_{metric}_baseline_imputed.csv", index=False)
-        leakage_imputed.to_csv(tables_dir / f"pivot_{metric}_leakage_imputed.csv", index=False)
-        print(f"  Saved: pivot_{metric}.csv")
-
-    # Generate leaderboard and pairwise tables for each task group
-    for group_name, task_list in TASK_GROUPS.items():
-        print(f"\nProcessing group: {group_name} ({len(task_list)} tasks)...")
-
-        # Create subdirectory for this group
-        group_dir = tables_dir / group_name
-        group_dir.mkdir(exist_ok=True)
-
-        # Filter summaries to only include tasks in this group
-        group_summaries = summaries[summaries["task_name"].isin(task_list)]
-
-        if group_summaries.empty:
-            print(f"  WARNING: No matching tasks found for group {group_name}")
-            continue
-
-        actual_tasks = group_summaries["task_name"].nunique()
-        print(f"  Found {actual_tasks} tasks in summaries")
-
-        for metric in AVAILABLE_METRICS:
-            # Compute leaderboard for this group
-            leaderboard_df = compute_leaderboard(group_summaries, metric)
-            leaderboard_df.to_csv(group_dir / f"leaderboard_{metric}.csv", index=False)
-
-            # Get top models for pairwise comparison
-            top_k_models = (
-                leaderboard_df.sort_values(by=SORT_COL, ascending=False).head(TOP_K_MODELS_TO_PLOT)["model_name"].tolist()
-            )
-
-            # Compute pairwise comparison
-            pairwise_df = compute_pairwise(group_summaries, metric, top_k_models)
-            pairwise_df.to_csv(group_dir / f"pairwise_{metric}.csv", index=False)
-
-            print(f"  Saved: {group_name}/leaderboard_{metric}.csv, {group_name}/pairwise_{metric}.csv")
-
-    print(f"\nAll tables saved to {tables_dir}/")
-
-
-if __name__ == "__main__":
-    main()

src/about.py

@@ -0,0 +1,50 @@
+TITLE = """<h1 align="center" id="space-title">Forecast evaluation leaderboard</h1>"""
+
+# What does your leaderboard evaluate?
+INTRODUCTION_TEXT = """
+This space hosts evaluation results for time series forecasting models.
+
+The results are obtained using [fev](https://github.com/autogluon/fev) - a lightweight library for evaluating time series forecasting models.
+"""
+
+ABOUT_LEADERBOARD = """
+## What is `fev`?
+
+[`fev`](https://github.com/autogluon/fev) is a lightweight wrapper around the 🤗 [`datasets`](https://huggingface.co/docs/datasets/en/index) library that makes it easy to benchmark time series forecasting models.
+
+For more information about `fev`, please check out [github.com/autogluon/fev](https://github.com/autogluon/fev).
+
+Currently, the results in this space are a minimal proof of concept. We plan to add new benchmark datasets and tasks in the future.
+
+## How is `fev` different from other benchmarking tools?
+Existing forecasting benchmarks usually fall into one of two categories:
+
+- Standalone datasets without any supporting infrastructure. These provide no guarantees that the results obtained by different users are comparable. For example, changing the start date or duration of the forecast horizon totally changes the meaning of the scores.
+- Bespoke end-to-end systems that combine models, datasets and forecasting tasks. Such packages usually come with lots of dependencies and assumptions, which makes extending or integrating these libraries into existing systems difficult.
+
+`fev` aims for the middle ground - it provides the core benchmarking functionality without introducing unnecessary constraints or bloated dependencies. The library supports point & probabilistic forecasting, different types of covariates, as well as all popular forecasting metrics.
+
+
+## Submitting your model
+For instructions on how to evaluate your model using `fev` and contribute your results to the leaderboard, please follow the [instructions in the GitHub repo](https://github.com/autogluon/fev/blob/main/docs/04-models.ipynb).
+"""
+
+CHRONOS_BENCHMARK = """
+## Chronos Benchmark II results
+
+This tab contains results for various forecasting models on the 27 datasets used in Benchmark II in the publication [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815).
+
+These datasets were used for zero-shot evaluation of Chronos models (i.e., Chronos models were not trained on these datasets), but some other models did include certain datasets in their training corpus.
+
+Each table contains the following information:
+
+* **Average relative error**: Geometric mean of the relative errors for each task. The relative error for each task is computed as `model_error / baseline_error`.
+* **Average rank**: Arithmetic mean of the ranks achieved by each model on each task.
+* **Median inference time (s)**: Median of the times required to make predictions for the entire dataset (in seconds).
+* **Training corpus overlap (%)**: Percentage of the datasets used in the benchmark that were included in the model's training corpus. Zero-shot models are highlighted in <span style="color:green; font-weight:bold;">green</span>.
+
+Lower values are better for all of the above metrics.
+
+Task definitions and the detailed results are available on [GitHub](https://github.com/autogluon/fev/tree/main/benchmarks/chronos_zeroshot). More information for the datasets is available in [Table 3 of the paper](https://arxiv.org/abs/2403.07815).
+
+"""

src/colors.py

@@ -1,6 +0,0 @@
-# Legacy colors - kept for backward compatibility if needed elsewhere
-VERY_PALE_PURPLE = "#e8d9f3"
-VERY_PALE_GREEN = "#cffdbc"
-VERY_PALE_BLUE = "#d6fffe"
-DEEP_LAVENDER = "#8d5eb7"
-GRASS_GREEN = "#3f9b0b"

src/custom_html_js.py

@@ -0,0 +1,99 @@
+custom_css = """
+
+.markdown-text {
+    font-size: 20px !important;
+}
+
+"""
+
+
+# .tab-buttons button {
+#     font-size: 20px;
+# }
+
+# #citation-button span {
+#     font-size: 16px !important;
+# }
+
+# #citation-button textarea {
+#     font-size: 16px !important;
+# }
+
+# #citation-button > label > button {
+#     margin: 6px;
+#     transform: scale(1.3);
+# }
+
+
+# #leaderboard-table-lite {
+#     margin-top: 15px
+# }
+
+# #search-bar-table-box > div:first-child {
+#     background: none;
+#     border: none;
+# }
+
+# #search-bar {
+#     padding: 0px;
+# }
+
+# /* Hides the final AutoEvalColumn */
+# #llm-benchmark-tab-table table td:last-child,
+# #llm-benchmark-tab-table table th:last-child {
+#     display: none;
+# }
+
+# /* Limit the width of the first AutoEvalColumn so that names don't expand too much */
+# table td:first-child,
+# table th:first-child {
+#     max-width: 400px;
+#     overflow: auto;
+#     white-space: nowrap;
+# }
+
+
+# #scale-logo {
+#     border-style: none !important;
+#     box-shadow: none;
+#     display: block;
+#     margin-left: auto;
+#     margin-right: auto;
+#     max-width: 600px;
+# }
+
+# #scale-logo .download {
+#     display: none;
+# }
+# #filter_type{
+#     border: 0;
+#     padding-left: 0;
+#     padding-top: 0;
+# }
+# #filter_type label {
+#     display: flex;
+# }
+# #filter_type label > span{
+#     margin-top: var(--spacing-lg);
+#     margin-right: 0.5em;
+# }
+# #filter_type label > .wrap{
+#     width: 103px;
+# }
+# #filter_type label > .wrap .wrap-inner{
+#     padding: 2px;
+# }
+# #filter_type label > .wrap .wrap-inner input{
+#     width: 1px
+# }
+# #filter-columns-type{
+#     border:0;
+#     padding:0.5;
+# }
+# #filter-columns-size{
+#     border:0;
+#     padding:0.5;
+# }
+# #box-filter > .form{
+#     border: 0
+# }

src/formatting.py

@@ -0,0 +1,31 @@
+def model_hyperlink(link, model_name):
+    return f'<a target="_blank" href="{link}" style="color: var(--link-text-color); text-decoration: underline;text-decoration-style: dotted;">{model_name}</a>'
+
+
+MODEL_URLS = {
+    "chronos_tiny": "amazon/chronos-t5-tiny",
+    "chronos_mini": "amazon/chronos-t5-mini",
+    "chronos_small": "amazon/chronos-t5-small",
+    "chronos_base": "amazon/chronos-t5-base",
+    "chronos_large": "amazon/chronos-t5-large",
+    "chronos_bolt_tiny": "amazon/chronos-bolt-tiny",
+    "chronos_bolt_mini": "amazon/chronos-bolt-mini",
+    "chronos_bolt_small": "amazon/chronos-bolt-small",
+    "chronos_bolt_base": "amazon/chronos-bolt-base",
+    "moirai_large": "Salesforce/moirai-1.1-R-large",
+    "moirai_base": "Salesforce/moirai-1.1-R-base",
+    "moirai_small": "Salesforce/moirai-1.1-R-small",
+    "timesfm": "google/timesfm-1.0-200m-pytorch",
+    "timesfm-2.0": "google/timesfm-2.0-500m-pytorch",
+    "ttm-r2": "ibm-granite/granite-timeseries-ttm-r2",
+    "tirex": "NX-AI/TiRex",
+}
+
+
+def make_clickable_model(model_name):
+    if model_name in MODEL_URLS:
+        model_path = MODEL_URLS.get(model_name)
+        link = f"https://huggingface.co/{model_path}"
+        return model_hyperlink(link, model_name)
+    else:
+        return model_name

src/streamlit_app.py

@@ -1,9 +0,0 @@
-import streamlit as st
-
-pages = [
-    st.Page("../pages/fev_bench.py", title="fev-bench", icon=":material/trophy:"),
-    st.Page("../pages/about.py", title="About", icon=":material/info:"),
-]
-
-page = st.navigation(pages)
-page.run()

src/strings.py

@@ -1,114 +0,0 @@
-from src.utils import COLORS
-
-INTRODUCTION_TEXT = """
-This space hosts evaluation results for time series forecasting models. The results are obtained using [fev](https://github.com/autogluon/fev) - a lightweight library for evaluating time series forecasting models.
-"""
-
-LEGEND = """
-"""
-
-TABLE_INFO = f"""
-The leaderboard summarizes the performance of all models evaluated on the 100 tasks comprising **fev-bench**. More details available in the [paper](https://arxiv.org/abs/2509.26468).
-
-Model names are colored by type: <span style='color: {COLORS["dl_text"]}; font-weight: bold;'>Deep Learning</span> and <span style='color: {COLORS["st_text"]}; font-weight: bold;'>Statistical</span>.
-
-The full matrix $E_{{rj}}$ with the error of each model $j$ on task $r$ is available at the bottom of the page.
-
-* **Avg. win rate (%)**: Fraction of all possible model pairs and tasks where this model achieves lower error than the competing model. For model $j$, defined as $W_j = \\frac{{1}}{{R(M-1)}} \\sum_{{r=1}}^{{R}} \\sum_{{k \\neq j}} (\\mathbf{{1}}(E_{{rj}} < E_{{rk}}) + 0.5 \\cdot \\mathbf{{1}}(E_{{rj}} = E_{{rk}}))$ where $R$ is number of tasks, $M$ is number of models. Ties count as half-wins.
-
-    Ranges from 0% (worst) to 100% (best). Higher values are better. This value changes as new models are added to the benchmark.
-
-* **Skill score (%)**: Measures how much the model reduces forecasting error compared to the Seasonal Naive baseline. Computed as $S_j = 100 \\times (1 - \\sqrt[R]{{\\prod_{{r=1}}^{{R}} E_{{rj}}/E_{{r\\beta}}}})$, where $E_{{r\\beta}}$ is baseline error on task $r$. Relative errors are clipped between 0.01 and 100 before aggregation to avoid extreme outliers. Positive values indicate better-than-baseline performance, negative values indicate worse-than-baseline performance.
-
-    Higher values are better. This value does not change as new models are added to the benchmark.
-
-* **Median runtime (s)**: Median end-to-end time (training + prediction across all evaluation windows) in seconds. Note that inference times depend on hardware, batch sizes, and implementation details, so these serve as a rough guide rather than definitive performance benchmarks.
-
-* **Leakage (%)**: For zero-shot models, percentage of benchmark datasets included in the model's training corpus. Results for tasks with reported overlap are replaced with Chronos-Bolt (Base) performance to prevent data leakage.
-
-* **Failed tasks (%)**: Percentage of tasks where the model failed to produce a forecast. Results for failed tasks are replaced with Seasonal Naive performance.
-
-* **Zero-shot**: Indicates whether the model can make predictions without task-specific training (✓ = zero-shot, × = task-specific).
-"""
-
-CHRONOS_BENCHMARK_BASIC_INFO = f"""
-**Chronos Benchmark II** contains results for various forecasting models on the 27 datasets used in Benchmark II in the paper [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815). {LEGEND}
-"""
-
-CHRONOS_BENCHMARK_DETAILS = f"""
-{TABLE_INFO}
-
-Task definitions and the detailed results are available on [GitHub](https://github.com/autogluon/fev/tree/main/benchmarks/chronos_zeroshot). More information for the datasets is available in [Table 3 of the paper](https://arxiv.org/abs/2403.07815).
-"""
-
-FEV_BENCHMARK_BASIC_INFO = f"""
-Results for various forecasting models on 100 tasks of the **fev-bench** benchmark, as described in the paper [fev-bench: A Realistic Benchmark for Time Series Forecasting](https://arxiv.org/abs/2509.26468). {LEGEND}
-"""
-
-FEV_BENCHMARK_DETAILS = f"""
-{TABLE_INFO}
-
-Task definitions and the detailed results are available on [GitHub](https://github.com/autogluon/fev/tree/main/benchmarks/). Datasets used for evaluation are available on [Hugging Face](https://huggingface.co/datasets/autogluon/fev_datasets).
-"""
-
-CITATION_HEADER = """
-If you find this leaderboard useful for your research, please consider citing the associated paper(s):
-
-"""
-CITATION_FEV = """
-```
-@article{shchur2025fev,
-  title={{fev-bench}: A Realistic Benchmark for Time Series Forecasting},
-  author={Shchur, Oleksandr and Ansari, Abdul Fatir and Turkmen, Caner and Stella, Lorenzo and Erickson, Nick and Guerron, Pablo and Bohlke-Schneider, Michael and Wang, Yuyang},
-  year={2025},
-  eprint={2509.26468},
-  archivePrefix={arXiv},
-  primaryClass={cs.LG}
-}
-```
-"""
-
-
-def get_pivot_legend(baseline_model: str, leakage_imputation_model: str) -> str:
-    return f"""
-Task definitions and raw results in CSV format are available on [GitHub](https://github.com/autogluon/fev/tree/main/benchmarks/fev_bench).
-
-Best results for each task are marked with
-<span style='background: {COLORS["gold"]}; color: {COLORS["text_default"]}; padding: 3px; border-radius: 5px;'>🥇 1st</span>
-<span style='background: {COLORS["silver"]}; color: {COLORS["text_default"]}; padding: 3px; border-radius: 5px;'>🥈 2nd</span>
-<span style='background: {COLORS["bronze"]}; color: {COLORS["text_default"]}; padding: 3px; border-radius: 5px;'>🥉 3rd</span>
-<br><br>
-**Imputation:**
-- <span style='color: {COLORS["failure_impute"]}; font-weight: bold;'>Failed tasks</span> imputed by {baseline_model}
-- <span style='color: {COLORS["leakage_impute"]}; font-weight: bold;'>Leaky tasks</span> imputed by {leakage_imputation_model}
-"""
-
-
-PAIRWISE_BENCHMARK_DETAILS = """
-The pairwise charts show head-to-head results between models:
-
-* **Win rate**: Percentage of tasks where Model 1 achieves lower error than Model 2 (ties count as half-wins).
-  A value above 50% means Model 1 is more accurate than Model 2 on average.
-
-* **Skill score**: Average relative error reduction of Model 1 with respect to Model 2.
-  A positive value means Model 1 reduces forecasting error compared to Model 2 on average.
-
-**Confidence Intervals**: 95% intervals are estimated using 1000 bootstrap samples over tasks.
-For each bootstrap sample, tasks are resampled with replacement and the pairwise win rate / skill score are recomputed.
-The intervals correspond to the 2.5th and 97.5th percentiles of these bootstrap distributions,
-capturing how model comparisons vary under alternative benchmark compositions.
-"""
-
-
-CITATION_CHRONOS = """
-```
-@article{ansari2024chronos,
-  title={Chronos: Learning the Language of Time Series},
-  author={Ansari, Abdul Fatir and Stella, Lorenzo and Turkmen, Caner and Zhang, Xiyuan, and Mercado, Pedro and Shen, Huibin and Shchur, Oleksandr and Rangapuram, Syama Syndar and Pineda Arango, Sebastian and Kapoor, Shubham and Zschiegner, Jasper and Maddix, Danielle C. and Wang, Hao and Mahoney, Michael W. and Torkkola, Kari and Gordon Wilson, Andrew and Bohlke-Schneider, Michael and Wang, Yuyang},
-  journal={Transactions on Machine Learning Research},
-  issn={2835-8856},
-  year={2024},
-  url={https://openreview.net/forum?id=gerNCVqqtR}
-}
-```
-"""

src/task_groups.py

@@ -1,209 +0,0 @@
-"""Task groupings for filtering the leaderboard by subsets."""
-
-# All tasks in the benchmark (100 tasks)
-ALL_TASKS = [
-    "ETT_15T", "ETT_1D", "ETT_1H", "ETT_1W",
-    "LOOP_SEATTLE_1D", "LOOP_SEATTLE_1H", "LOOP_SEATTLE_5T",
-    "M_DENSE_1D", "M_DENSE_1H",
-    "SZ_TAXI_15T", "SZ_TAXI_1H",
-    "australian_tourism",
-    "bizitobs_l2c_1H", "bizitobs_l2c_5T",
-    "boomlet_1062", "boomlet_1209", "boomlet_1225", "boomlet_1230", "boomlet_1282",
-    "boomlet_1487", "boomlet_1631", "boomlet_1676", "boomlet_1855", "boomlet_1975",
-    "boomlet_2187", "boomlet_285", "boomlet_619", "boomlet_772", "boomlet_963",
-    "ecdc_ili",
-    "entsoe_15T", "entsoe_1H", "entsoe_30T",
-    "epf_be", "epf_de", "epf_fr", "epf_np", "epf_pjm",
-    "ercot_1D", "ercot_1H", "ercot_1M", "ercot_1W",
-    "favorita_stores_1D", "favorita_stores_1M", "favorita_stores_1W",
-    "favorita_transactions_1D", "favorita_transactions_1M", "favorita_transactions_1W",
-    "fred_md_2025/cee", "fred_md_2025/macro",
-    "fred_qd_2025/cee", "fred_qd_2025/macro",
-    "gvar",
-    "hermes",
-    "hierarchical_sales_1D", "hierarchical_sales_1W",
-    "hospital", "hospital_admissions_1D", "hospital_admissions_1W",
-    "jena_weather_10T", "jena_weather_1D", "jena_weather_1H",
-    "kdd_cup_2022_10T", "kdd_cup_2022_1D", "kdd_cup_2022_30T",
-    "m5_1D", "m5_1M", "m5_1W",
-    "proenfo_gfc12", "proenfo_gfc14", "proenfo_gfc17",
-    "redset_15T", "redset_1H", "redset_5T",
-    "restaurant",
-    "rohlik_orders_1D", "rohlik_orders_1W", "rohlik_sales_1D", "rohlik_sales_1W",
-    "rossmann_1D", "rossmann_1W",
-    "solar_1D", "solar_1W", "solar_with_weather_15T", "solar_with_weather_1H",
-    "uci_air_quality_1D", "uci_air_quality_1H",
-    "uk_covid_nation_1D/cumulative", "uk_covid_nation_1D/new",
-    "uk_covid_nation_1W/cumulative", "uk_covid_nation_1W/new",
-    "uk_covid_utla_1D/new", "uk_covid_utla_1W/cumulative",
-    "us_consumption_1M", "us_consumption_1Q", "us_consumption_1Y",
-    "walmart",
-    "world_co2_emissions", "world_life_expectancy", "world_tourism",
-]
-
-# Mini benchmark - representative subset (20 tasks)
-MINI_TASKS = [
-    "jena_weather_1H",
-    "M_DENSE_1D",
-    "bizitobs_l2c_5T",
-    "rohlik_orders_1D",
-    "boomlet_1282",
-    "rossmann_1D",
-    "rossmann_1W",
-    "boomlet_1676",
-    "solar_with_weather_1H",
-    "boomlet_619",
-    "uci_air_quality_1H",
-    "uk_covid_nation_1D/cumulative",
-    "us_consumption_1Y",
-    "epf_np",
-    "world_co2_emissions",
-    "ETT_15T",
-    "ETT_1H",
-    "proenfo_gfc14",
-    "hospital_admissions_1D",
-    "hospital_admissions_1W",
-]
-
-# Frequency-based groupings
-FREQUENCY_GROUPS = {
-    "Sub-hourly": [
-        # T (1 minute)
-        "boomlet_1225", "boomlet_1282", "boomlet_285", "boomlet_619", "boomlet_772", "boomlet_963",
-        # 5T (5 minutes)
-        "LOOP_SEATTLE_5T", "bizitobs_l2c_5T", "redset_5T",
-        "boomlet_1062", "boomlet_1209", "boomlet_1230", "boomlet_1487",
-        # 10T (10 minutes)
-        "jena_weather_10T", "kdd_cup_2022_10T",
-        # 15T (15 minutes)
-        "ETT_15T", "SZ_TAXI_15T", "entsoe_15T", "redset_15T", "solar_with_weather_15T",
-        # 30T (30 minutes)
-        "entsoe_30T", "kdd_cup_2022_30T", "boomlet_1631", "boomlet_1676",
-    ],
-    "Hourly": [
-        "ETT_1H", "LOOP_SEATTLE_1H", "M_DENSE_1H", "SZ_TAXI_1H",
-        "bizitobs_l2c_1H", "entsoe_1H", "ercot_1H",
-        "epf_be", "epf_de", "epf_fr", "epf_np", "epf_pjm",
-        "jena_weather_1H",
-        "proenfo_gfc12", "proenfo_gfc14", "proenfo_gfc17",
-        "redset_1H", "solar_with_weather_1H", "uci_air_quality_1H",
-        "boomlet_1855", "boomlet_1975", "boomlet_2187",
-    ],
-    "Daily": [
-        "ETT_1D", "LOOP_SEATTLE_1D", "M_DENSE_1D",
-        "ercot_1D", "kdd_cup_2022_1D", "solar_1D",
-        "favorita_stores_1D", "favorita_transactions_1D",
-        "hierarchical_sales_1D", "m5_1D",
-        "restaurant",
-        "rohlik_orders_1D", "rohlik_sales_1D", "rossmann_1D",
-        "jena_weather_1D", "uci_air_quality_1D",
-        "hospital_admissions_1D",
-        "uk_covid_nation_1D/cumulative", "uk_covid_nation_1D/new", "uk_covid_utla_1D/new",
-    ],
-    "Weekly": [
-        "ETT_1W", "ercot_1W", "solar_1W",
-        "favorita_stores_1W", "favorita_transactions_1W",
-        "hierarchical_sales_1W", "m5_1W",
-        "hermes", "walmart",
-        "rohlik_orders_1W", "rohlik_sales_1W", "rossmann_1W",
-        "ecdc_ili",
-        "hospital_admissions_1W",
-        "uk_covid_nation_1W/cumulative", "uk_covid_nation_1W/new", "uk_covid_utla_1W/cumulative",
-    ],
-    "Monthly+": [
-        # Monthly
-        "ercot_1M",
-        "favorita_stores_1M", "favorita_transactions_1M", "m5_1M",
-        "fred_md_2025/cee", "fred_md_2025/macro",
-        "hospital",
-        "us_consumption_1M",
-        # Quarterly
-        "australian_tourism", "gvar",
-        "fred_qd_2025/cee", "fred_qd_2025/macro",
-        "us_consumption_1Q",
-        # Yearly
-        "us_consumption_1Y",
-        "world_co2_emissions", "world_life_expectancy", "world_tourism",
-    ],
-}
-
-# Domain-based groupings
-DOMAIN_GROUPS = {
-    "Energy": [
-        "ETT_15T", "ETT_1D", "ETT_1H", "ETT_1W",
-        "entsoe_15T", "entsoe_1H", "entsoe_30T",
-        "epf_be", "epf_de", "epf_fr", "epf_np", "epf_pjm",
-        "ercot_1D", "ercot_1H", "ercot_1M", "ercot_1W",
-        "kdd_cup_2022_10T", "kdd_cup_2022_1D", "kdd_cup_2022_30T",
-        "proenfo_gfc12", "proenfo_gfc14", "proenfo_gfc17",
-        "solar_1D", "solar_1W", "solar_with_weather_15T", "solar_with_weather_1H",
-    ],
-    "Retail": [
-        "favorita_stores_1D", "favorita_stores_1M", "favorita_stores_1W",
-        "favorita_transactions_1D", "favorita_transactions_1M", "favorita_transactions_1W",
-        "hermes",
-        "hierarchical_sales_1D", "hierarchical_sales_1W",
-        "m5_1D", "m5_1M", "m5_1W",
-        "restaurant",
-        "rohlik_orders_1D", "rohlik_orders_1W", "rohlik_sales_1D", "rohlik_sales_1W",
-        "rossmann_1D", "rossmann_1W",
-        "walmart",
-    ],
-    "Nature": [
-        "jena_weather_10T", "jena_weather_1D", "jena_weather_1H",
-        "uci_air_quality_1D", "uci_air_quality_1H",
-    ],
-    "Cloud": [
-        "bizitobs_l2c_1H", "bizitobs_l2c_5T",
-        "boomlet_1062", "boomlet_1209", "boomlet_1225", "boomlet_1230", "boomlet_1282",
-        "boomlet_1487", "boomlet_1631", "boomlet_1676", "boomlet_1855", "boomlet_1975",
-        "boomlet_2187", "boomlet_285", "boomlet_619", "boomlet_772", "boomlet_963",
-        "redset_15T", "redset_1H", "redset_5T",
-    ],
-    "Health": [
-        "ecdc_ili",
-        "hospital", "hospital_admissions_1D", "hospital_admissions_1W",
-        "uk_covid_nation_1D/cumulative", "uk_covid_nation_1D/new",
-        "uk_covid_nation_1W/cumulative", "uk_covid_nation_1W/new",
-        "uk_covid_utla_1D/new", "uk_covid_utla_1W/cumulative",
-    ],
-    "Econ": [
-        "australian_tourism",
-        "fred_md_2025/cee", "fred_md_2025/macro",
-        "fred_qd_2025/cee", "fred_qd_2025/macro",
-        "gvar",
-        "us_consumption_1M", "us_consumption_1Q", "us_consumption_1Y",
-        "world_co2_emissions", "world_life_expectancy", "world_tourism",
-    ],
-    "Mobility": [
-        "LOOP_SEATTLE_1D", "LOOP_SEATTLE_1H", "LOOP_SEATTLE_5T",
-        "M_DENSE_1D", "M_DENSE_1H",
-        "SZ_TAXI_15T", "SZ_TAXI_1H",
-    ],
-}
-
-
-def get_task_group(group_type: str, group_value: str | None = None) -> list[str]:
-    """Get the list of tasks for a given group type and value.
-
-    Args:
-        group_type: One of "full", "mini", "frequency", "domain"
-        group_value: Required for "frequency" and "domain" types
-
-    Returns:
-        List of task names belonging to the group
-    """
-    if group_type == "full":
-        return ALL_TASKS
-    elif group_type == "mini":
-        return MINI_TASKS
-    elif group_type == "frequency":
-        if group_value is None:
-            raise ValueError("group_value required for frequency grouping")
-        return FREQUENCY_GROUPS[group_value]
-    elif group_type == "domain":
-        if group_value is None:
-            raise ValueError("group_value required for domain grouping")
-        return DOMAIN_GROUPS[group_value]
-    else:
-        raise ValueError(f"Unknown group_type: {group_type}")

src/utils.py

@@ -1,374 +0,0 @@
-import altair as alt
-import fev
-import pandas as pd
-import pandas.io.formats.style
-
-# Color constants - all colors defined in one place
-
-COLORS = {
-    "dl_text": "#5A7FA5",
-    "st_text": "#A5795A",
-    # "st_text": "#666666",
-    "bar_fill": "#8d5eb7",
-    "error_bar": "#222222",
-    "point": "#111111",
-    "text_white": "white",
-    "text_black": "black",
-    "text_default": "#111",
-    "gold": "#F7D36B",
-    "silver": "#E5E7EB",
-    "bronze": "#E6B089",
-    "leakage_impute": "#3B82A0",
-    "failure_impute": "#E07B39",
-}
-HEATMAP_COLOR_SCHEME = "purplegreen"
-
-# Model configuration: (url, org, zero_shot, model_type)
-MODEL_CONFIG = {
-    # Chronos Models
-    "chronos_tiny": ("amazon/chronos-t5-tiny", "AWS", True, "DL"),
-    "chronos_mini": ("amazon/chronos-t5-mini", "AWS", True, "DL"),
-    "chronos_small": ("amazon/chronos-t5-small", "AWS", True, "DL"),
-    "chronos_base": ("amazon/chronos-t5-base", "AWS", True, "DL"),
-    "chronos_large": ("amazon/chronos-t5-large", "AWS", True, "DL"),
-    "chronos_bolt_tiny": ("amazon/chronos-bolt-tiny", "AWS", True, "DL"),
-    "chronos_bolt_mini": ("amazon/chronos-bolt-mini", "AWS", True, "DL"),
-    "chronos_bolt_small": ("amazon/chronos-bolt-small", "AWS", True, "DL"),
-    "chronos_bolt_base": ("amazon/chronos-bolt-base", "AWS", True, "DL"),
-    "chronos-bolt": ("amazon/chronos-bolt-base", "AWS", True, "DL"),
-    "chronos-2": ("amazon/chronos-2", "AWS", True, "DL"),
-    # Moirai Models
-    "moirai_large": ("Salesforce/moirai-1.1-R-large", "Salesforce", True, "DL"),
-    "moirai_base": ("Salesforce/moirai-1.1-R-base", "Salesforce", True, "DL"),
-    "moirai_small": ("Salesforce/moirai-1.1-R-small", "Salesforce", True, "DL"),
-    "moirai-2.0": ("Salesforce/moirai-2.0-R-small", "Salesforce", True, "DL"),
-    # TimesFM Models
-    "timesfm": ("google/timesfm-1.0-200m-pytorch", "Google", True, "DL"),
-    "timesfm-2.0": ("google/timesfm-2.0-500m-pytorch", "Google", True, "DL"),
-    "timesfm-2.5": ("google/timesfm-2.5-200m-pytorch", "Google", True, "DL"),
-    # Toto Models
-    "toto-1.0": ("Datadog/Toto-Open-Base-1.0", "Datadog", True, "DL"),
-    # Other Models
-    "tirex": ("NX-AI/TiRex", "NX-AI", True, "DL"),
-    "tabpfn-ts": ("Prior-Labs/TabPFN-v2-reg", "Prior Labs", True, "DL"),
-    "sundial-base": ("thuml/sundial-base-128m", "Tsinghua University", True, "DL"),
-    "ttm-r2": ("ibm-granite/granite-timeseries-ttm-r2", "IBM", True, "DL"),
-    # Task-specific models
-    "stat. ensemble": (
-        "https://nixtlaverse.nixtla.io/statsforecast/",
-        "—",
-        False,
-        "ST",
-    ),
-    "autoarima": ("https://nixtlaverse.nixtla.io/statsforecast/", "—", False, "ST"),
-    "autotheta": ("https://nixtlaverse.nixtla.io/statsforecast/", "—", False, "ST"),
-    "autoets": ("https://nixtlaverse.nixtla.io/statsforecast/", "—", False, "ST"),
-    "seasonalnaive": ("https://nixtlaverse.nixtla.io/statsforecast/", "—", False, "ST"),
-    "seasonal naive": (
-        "https://nixtlaverse.nixtla.io/statsforecast/",
-        "—",
-        False,
-        "ST",
-    ),
-    "drift": ("https://nixtlaverse.nixtla.io/statsforecast/", "—", False, "ST"),
-    "naive": ("https://nixtlaverse.nixtla.io/statsforecast/", "—", False, "ST"),
-}
-
-
-ALL_METRICS = {
-    "SQL": (
-        "SQL: Scaled Quantile Loss",
-        "The [Scaled Quantile Loss (SQL)](https://auto.gluon.ai/dev/tutorials/timeseries/forecasting-metrics.html#autogluon.timeseries.metrics.SQL) is a **scale-invariant** metric for evaluating **probabilistic** forecasts.",
-    ),
-    "MASE": (
-        "MASE: Mean Absolute Scaled Error",
-        "The [Mean Absolute Scaled Error (MASE)](https://auto.gluon.ai/dev/tutorials/timeseries/forecasting-metrics.html#autogluon.timeseries.metrics.MASE) is a **scale-invariant** metric for evaluating **point** forecasts.",
-    ),
-    "WQL": (
-        "WQL: Weighted Quantile Loss",
-        "The [Weighted Quantile Loss (WQL)](https://auto.gluon.ai/dev/tutorials/timeseries/forecasting-metrics.html#autogluon.timeseries.metrics.WQL), is a **scale-dependent** metric for evaluating **probabilistic** forecasts.",
-    ),
-    "WAPE": (
-        "WAPE: Weighted Absolute Percentage Error",
-        "The [Weighted Absolute Percentage Error (WAPE)](https://auto.gluon.ai/dev/tutorials/timeseries/forecasting-metrics.html#autogluon.timeseries.metrics.WAPE) is a **scale-dependent** metric for evaluating **point** forecasts.",
-    ),
-}
-
-
-def format_metric_name(metric_name: str):
-    return ALL_METRICS[metric_name][0]
-
-
-def get_metric_description(metric_name: str):
-    return ALL_METRICS[metric_name][1]
-
-
-def get_model_link(model_name):
-    config = MODEL_CONFIG.get(model_name.lower())
-    if not config or not config[0]:
-        return ""
-    url = config[0]
-    return url if url.startswith("https:") else f"https://huggingface.co/{url}"
-
-
-def get_model_organization(model_name):
-    config = MODEL_CONFIG.get(model_name.lower())
-    return config[1] if config else "—"
-
-
-def get_zero_shot_status(model_name):
-    config = MODEL_CONFIG.get(model_name.lower())
-    return "✓" if config and config[2] else "×"
-
-
-def get_model_type(model_name):
-    config = MODEL_CONFIG.get(model_name.lower())
-    return config[3] if config else "—"
-
-
-def highlight_model_type_color(cell):
-    config = MODEL_CONFIG.get(cell.lower())
-    if config:
-        color = COLORS["dl_text"] if config[3] == "DL" else COLORS["st_text"]
-        return f"font-weight: bold; color: {color}"
-    return "font-weight: bold"
-
-
-def format_leaderboard(df: pd.DataFrame):
-    df = df.copy()
-    df["skill_score"] = df["skill_score"].round(1)
-    df["win_rate"] = df["win_rate"].round(1)
-    df["zero_shot"] = df["model_name"].apply(get_zero_shot_status)
-    # Format leakage column: convert to int for all models, 0 for non-zero-shot
-    df["training_corpus_overlap"] = df.apply(
-        lambda row: int(round(row["training_corpus_overlap"] * 100)) if row["zero_shot"] == "✓" else 0,
-        axis=1,
-    )
-    df["link"] = df["model_name"].apply(get_model_link)
-    df["org"] = df["model_name"].apply(get_model_organization)
-    df = df[
-        [
-            "model_name",
-            "win_rate",
-            "skill_score",
-            "median_inference_time_s_per100",
-            "training_corpus_overlap",
-            "num_failures",
-            "zero_shot",
-            "org",
-            "link",
-        ]
-    ]
-    return (
-        df.style.map(highlight_model_type_color, subset=["model_name"])
-        .map(lambda x: "font-weight: bold", subset=["zero_shot"])
-        .apply(
-            lambda x: ["background-color: #f8f9fa" if i % 2 == 1 else "" for i in range(len(x))],
-            axis=0,
-        )
-    )
-
-
-def construct_bar_chart(df: pd.DataFrame, col: str, metric_name: str):
-    label = "Skill Score" if col == "skill_score" else "Win Rate"
-
-    tooltip = [
-        alt.Tooltip("model_name:N"),
-        alt.Tooltip(f"{col}:Q", format=".2f"),
-        alt.Tooltip(f"{col}_lower:Q", title="95% CI Lower", format=".2f"),
-        alt.Tooltip(f"{col}_upper:Q", title="95% CI Upper", format=".2f"),
-    ]
-
-    base_encode = {
-        "y": alt.Y("model_name:N", title="Forecasting Model", sort=None),
-        "tooltip": tooltip,
-    }
-
-    bars = (
-        alt.Chart(df)
-        .mark_bar(color=COLORS["bar_fill"], cornerRadius=4)
-        .encode(
-            x=alt.X(f"{col}:Q", title=f"{label} (%)", scale=alt.Scale(zero=False)),
-            **base_encode,
-        )
-    )
-
-    error_bars = (
-        alt.Chart(df)
-        .mark_errorbar(ticks={"height": 5}, color=COLORS["error_bar"])
-        .encode(
-            y=alt.Y("model_name:N", title=None, sort=None),
-            x=alt.X(f"{col}_lower:Q", title=f"{label} (%)"),
-            x2=alt.X2(f"{col}_upper:Q"),
-            tooltip=tooltip,
-        )
-    )
-
-    points = (
-        alt.Chart(df)
-        .mark_point(filled=True, color=COLORS["point"])
-        .encode(x=alt.X(f"{col}:Q", title=f"{label} (%)"), **base_encode)
-    )
-
-    return (
-        (bars + error_bars + points)
-        .properties(height=500, title=f"{label} ({metric_name}) with 95% CIs")
-        .configure_title(fontSize=16)
-    )
-
-
-def construct_pairwise_chart(df: pd.DataFrame, col: str, metric_name: str):
-    config = {
-        "win_rate": ("Win Rate", [0, 100], 50, f"abs(datum.{col} - 50) > 30"),
-        "skill_score": ("Skill Score", [-15, 15], 0, f"abs(datum.{col}) > 10"),
-    }
-    cbar_label, domain, domain_mid, text_condition = config[col]
-
-    df = df.copy()
-    for c in [col, f"{col}_lower", f"{col}_upper"]:
-        df[c] *= 100
-
-    model_order = df.groupby("model_1")[col].mean().sort_values(ascending=False).index.tolist()
-
-    tooltip = [
-        alt.Tooltip("model_1:N", title="Model 1"),
-        alt.Tooltip("model_2:N", title="Model 2"),
-        alt.Tooltip(f"{col}:Q", title=cbar_label.split(" ")[0], format=".1f"),
-        alt.Tooltip(f"{col}_lower:Q", title="95% CI Lower", format=".1f"),
-        alt.Tooltip(f"{col}_upper:Q", title="95% CI Upper", format=".1f"),
-    ]
-
-    base = alt.Chart(df).encode(
-        x=alt.X(
-            "model_2:N",
-            sort=model_order,
-            title="Model 2",
-            axis=alt.Axis(orient="top", labelAngle=-90),
-        ),
-        y=alt.Y("model_1:N", sort=model_order, title="Model 1"),
-    )
-
-    heatmap = base.mark_rect().encode(
-        color=alt.Color(
-            f"{col}:Q",
-            legend=None,
-            scale=alt.Scale(
-                scheme=HEATMAP_COLOR_SCHEME,
-                domain=domain,
-                domainMid=domain_mid,
-                clamp=True,
-            ),
-        ),
-        tooltip=tooltip,
-    )
-
-    text_main = base.mark_text(dy=-8, fontSize=8, baseline="top", yOffset=5).encode(
-        text=alt.Text(f"{col}:Q", format=".1f"),
-        color=alt.condition(
-            text_condition,
-            alt.value(COLORS["text_white"]),
-            alt.value(COLORS["text_black"]),
-        ),
-        tooltip=tooltip,
-    )
-
-    return (
-        (heatmap + text_main)
-        .properties(
-            height=550,
-            title={
-                "text": f"Pairwise {cbar_label} ({metric_name}) with 95% CIs",
-                "fontSize": 16,
-            },
-        )
-        .configure_axis(labelFontSize=11, titleFontSize=13, titleFontWeight="bold")
-        .resolve_scale(color="independent")
-    )
-
-
-def construct_pivot_table_from_df(errors: pd.DataFrame, metric_name: str) -> pd.io.formats.style.Styler:
-    """Construct styled pivot table from precomputed DataFrame."""
-
-    def highlight_by_position(styler):
-        rank_colors = {1: COLORS["gold"], 2: COLORS["silver"], 3: COLORS["bronze"]}
-
-        for row_idx in errors.index:
-            row_ranks = errors.loc[row_idx].rank(method="min")
-            for col_idx in errors.columns:
-                rank = row_ranks[col_idx]
-                style_parts = []
-
-                # Rank background colors
-                if rank <= 3:
-                    style_parts.append(f"background-color: {rank_colors[rank]}")
-                else:
-                    style_parts.append(f"color: {COLORS['text_default']}")
-
-                if style_parts:
-                    styler = styler.map(
-                        lambda x, s="; ".join(style_parts): s,
-                        subset=pd.IndexSlice[row_idx:row_idx, col_idx:col_idx],
-                    )
-        return styler
-
-    return highlight_by_position(errors.style).format(precision=3)
-
-
-def construct_pivot_table(
-    summaries: pd.DataFrame,
-    metric_name: str,
-    baseline_model: str,
-    leakage_imputation_model: str,
-) -> pd.io.formats.style.Styler:
-    errors = fev.pivot_table(summaries=summaries, metric_column=metric_name, task_columns=["task_name"])
-    train_overlap = (
-        fev.pivot_table(
-            summaries=summaries,
-            metric_column="trained_on_this_dataset",
-            task_columns=["task_name"],
-        )
-        .fillna(False)
-        .astype(bool)
-    )
-
-    is_imputed_baseline = errors.isna()
-    is_leakage_imputed = train_overlap
-
-    # Handle imputations
-    errors = errors.mask(train_overlap, errors[leakage_imputation_model], axis=0)
-    for col in errors.columns:
-        if col != baseline_model:
-            errors[col] = errors[col].fillna(errors[baseline_model])
-
-    errors = errors[errors.rank(axis=1).mean().sort_values().index]
-    errors.index.rename("Task name", inplace=True)
-
-    def highlight_by_position(styler):
-        rank_colors = {1: COLORS["gold"], 2: COLORS["silver"], 3: COLORS["bronze"]}
-
-        for row_idx in errors.index:
-            row_ranks = errors.loc[row_idx].rank(method="min")
-            for col_idx in errors.columns:
-                rank = row_ranks[col_idx]
-                style_parts = []
-
-                # Rank background colors
-                if rank <= 3:
-                    style_parts.append(f"background-color: {rank_colors[rank]}")
-
-                # Imputation text colors
-                if is_leakage_imputed.loc[row_idx, col_idx]:
-                    style_parts.append(f"color: {COLORS['leakage_impute']}")
-                elif is_imputed_baseline.loc[row_idx, col_idx]:
-                    style_parts.append(f"color: {COLORS['failure_impute']}")
-                elif not style_parts or (len(style_parts) == 1 and "font-weight" in style_parts[0]):
-                    style_parts.append(f"color: {COLORS['text_default']}")
-
-                if style_parts:
-                    styler = styler.map(
-                        lambda x, s="; ".join(style_parts): s,
-                        subset=pd.IndexSlice[row_idx:row_idx, col_idx:col_idx],
-                    )
-        return styler
-
-    return highlight_by_position(errors.style).format(precision=3)

tables/domain_cloud/leaderboard_MASE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Toto-1.0,91.42857142857143,41.8883400084654,0.0,45.868307151375,0.0,0.0
-Chronos-2,89.64285714285714,42.5157996441094,0.0,1.18621670825,0.0,0.0
-TimesFM-2.5,82.85714285714286,40.26605685743614,0.0,6.4447616408988475,0.0,0.0
-TiRex,77.14285714285714,38.09940067233815,0.0,0.20031914146825397,0.0,0.0
-Moirai-2.0,65.00000000000001,35.579899495667256,0.0,0.36445902017857146,0.1,0.0
-Sundial-Base,61.07142857142858,34.44493255866505,0.0,7.874095355196429,0.0,0.0
-Chronos-Bolt,59.285714285714285,30.890375999745544,0.0,0.21195593629285714,0.0,0.0
-TabPFN-TS,57.857142857142854,31.63477113942893,0.0,187.1612375475248,0.0,0.0
-Stat. Ensemble,33.75,10.28649321258931,0.0,117.23392411285715,0.0,15.0
-AutoARIMA,32.32142857142858,11.154754181413495,0.0,17.96621433673913,0.0,15.0
-AutoTheta,27.500000000000004,6.832233342669413,0.0,3.978930596261481,0.0,0.0
-AutoETS,24.107142857142858,9.00813293024486,0.0,2.879931537857143,0.0,15.0
-Naive,23.214285714285715,-25.50199780067157,0.0,0.40027213541999274,0.0,0.0
-Seasonal Naive,15.892857142857144,0.0,0.0,0.43848143432692305,0.0,0.0
-Drift,8.928571428571429,-32.85429096552939,0.0,0.4187055400961539,0.0,0.0

tables/domain_cloud/leaderboard_SQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Toto-1.0,91.78571428571428,63.05871546601058,0.0,45.868307151375,0.0,0.0
-Chronos-2,91.42857142857143,63.92789671191335,0.0,1.18621670825,0.0,0.0
-TimesFM-2.5,83.57142857142857,61.98663044796851,0.0,6.4447616408988475,0.0,0.0
-TiRex,80.35714285714285,60.87156591539495,0.0,0.20031914146825397,0.0,0.0
-Moirai-2.0,68.21428571428572,58.41610987284718,0.0,0.36445902017857146,0.1,0.0
-Sundial-Base,59.64285714285715,56.647735223836015,0.0,7.874095355196429,0.0,0.0
-Chronos-Bolt,58.57142857142858,54.85929055283013,0.0,0.21195593629285714,0.0,0.0
-TabPFN-TS,57.50000000000001,53.05789136035014,0.0,187.1612375475248,0.0,0.0
-AutoARIMA,38.03571428571428,32.58752067086065,0.0,17.96621433673913,0.0,15.0
-Stat. Ensemble,34.82142857142856,20.1052436267542,0.0,117.23392411285715,0.0,15.0
-AutoETS,28.035714285714285,-26.853793211441058,0.0,2.879931537857143,0.0,15.0
-AutoTheta,22.142857142857146,-2.1145171067117996,0.0,3.978930596261481,0.0,0.0
-Seasonal Naive,19.107142857142854,0.0,0.0,0.43848143432692305,0.0,0.0
-Naive,13.571428571428573,-102.14853969808834,0.0,0.40027213541999274,0.0,0.0
-Drift,3.214285714285715,-110.2431874466197,0.0,0.4187055400961539,0.0,0.0

tables/domain_cloud/leaderboard_WAPE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,91.78571428571428,50.572763239812836,0.0,1.18621670825,0.0,0.0
-Toto-1.0,91.07142857142858,47.121114882947566,0.0,45.868307151375,0.0,0.0
-TimesFM-2.5,81.42857142857142,46.247719637805126,0.0,6.4447616408988475,0.0,0.0
-TiRex,77.49999999999999,44.52063533547439,0.0,0.20031914146825397,0.0,0.0
-Moirai-2.0,67.5,42.792548583451826,0.0,0.36445902017857146,0.1,0.0
-TabPFN-TS,66.78571428571429,43.22422155533027,0.0,187.1612375475248,0.0,0.0
-Chronos-Bolt,60.71428571428571,39.893668788246494,0.0,0.21195593629285714,0.0,0.0
-Sundial-Base,60.71428571428571,41.96635843416393,0.0,7.874095355196429,0.0,0.0
-Stat. Ensemble,32.32142857142857,15.58065226385924,0.0,117.23392411285715,0.0,15.0
-AutoARIMA,28.392857142857142,14.643401272886948,0.0,17.96621433673913,0.0,15.0
-AutoETS,23.75,5.6618608592632125,0.0,2.879931537857143,0.0,15.0
-AutoTheta,23.57142857142857,12.660865629460227,0.0,3.978930596261481,0.0,0.0
-Naive,23.21428571428571,4.49837332329478,0.0,0.40027213541999274,0.0,0.0
-Seasonal Naive,12.321428571428573,0.0,0.0,0.43848143432692305,0.0,0.0
-Drift,8.928571428571429,-1.7537851144645122,0.0,0.4187055400961539,0.0,0.0

tables/domain_cloud/leaderboard_WQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,92.5,66.94397676110452,0.0,1.18621670825,0.0,0.0
-Toto-1.0,90.35714285714286,64.45690397421868,0.0,45.868307151375,0.0,0.0
-TimesFM-2.5,82.14285714285712,63.753748722726,0.0,6.4447616408988475,0.0,0.0
-TiRex,80.71428571428571,62.74382048051918,0.0,0.20031914146825397,0.0,0.0
-Moirai-2.0,68.92857142857143,61.08750955081813,0.0,0.36445902017857146,0.1,0.0
-TabPFN-TS,65.00000000000001,59.171399643395425,0.0,187.1612375475248,0.0,0.0
-Chronos-Bolt,60.357142857142854,58.158162956775406,0.0,0.21195593629285714,0.0,0.0
-Sundial-Base,58.928571428571445,59.213728716592804,0.0,7.874095355196429,0.0,0.0
-AutoARIMA,36.25,33.35224855510223,0.0,17.96621433673913,0.0,15.0
-Stat. Ensemble,32.67857142857142,20.842955409190168,0.0,117.23392411285715,0.0,15.0
-AutoETS,24.82142857142857,-39.99504769759261,0.0,2.879931537857143,0.0,15.0
-AutoTheta,22.142857142857146,7.361982670066736,0.0,3.978930596261481,0.0,0.0
-Seasonal Naive,18.035714285714285,0.0,0.0,0.43848143432692305,0.0,0.0
-Naive,13.214285714285715,-64.97889014234586,0.0,0.40027213541999274,0.0,0.0
-Drift,3.9285714285714293,-71.7471743876533,0.0,0.4187055400961539,0.0,0.0

tables/domain_cloud/pairwise_MASE.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,Chronos-2,0.65,0.45,0.85,-0.011,-0.063,0.027
-Toto-1.0,TimesFM-2.5,0.7,0.5,0.9,0.027,-0.02,0.072
-Toto-1.0,TiRex,0.85,0.7,1.0,0.061,0.027,0.114
-Toto-1.0,Moirai-2.0,0.95,0.85,1.0,0.098,0.053,0.164
-Toto-1.0,Sundial-Base,0.9,0.75,1.0,0.114,0.037,0.182
-Toto-1.0,Chronos-Bolt,0.9,0.75,1.0,0.159,0.094,0.224
-Toto-1.0,TabPFN-TS,0.85,0.65,1.0,0.15,0.075,0.231
-Toto-1.0,Stat. Ensemble,1.0,1.0,1.0,0.352,0.266,0.45
-Toto-1.0,AutoARIMA,1.0,1.0,1.0,0.346,0.264,0.448
-Toto-1.0,AutoTheta,1.0,1.0,1.0,0.376,0.295,0.468
-Toto-1.0,AutoETS,1.0,1.0,1.0,0.361,0.267,0.469
-Toto-1.0,Naive,1.0,1.0,1.0,0.537,0.377,0.689
-Toto-1.0,Seasonal Naive,1.0,1.0,1.0,0.419,0.323,0.529
-Toto-1.0,Drift,1.0,1.0,1.0,0.563,0.414,0.702
-Chronos-2,Toto-1.0,0.35,0.15,0.55,0.011,-0.028,0.059
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,TimesFM-2.5,0.7,0.5,0.9,0.038,0.009,0.071
-Chronos-2,TiRex,0.75,0.55,0.95,0.071,0.022,0.128
-Chronos-2,Moirai-2.0,1.0,1.0,1.0,0.108,0.053,0.168
-Chronos-2,Sundial-Base,0.9,0.75,1.0,0.123,0.07,0.182
-Chronos-2,Chronos-Bolt,0.95,0.85,1.0,0.168,0.101,0.231
-Chronos-2,TabPFN-TS,0.9,0.75,1.0,0.159,0.092,0.234
-Chronos-2,Stat. Ensemble,1.0,1.0,1.0,0.359,0.272,0.454
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.353,0.266,0.453
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.383,0.296,0.47
-Chronos-2,AutoETS,1.0,1.0,1.0,0.368,0.281,0.468
-Chronos-2,Naive,1.0,1.0,1.0,0.542,0.386,0.689
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.425,0.331,0.534
-Chronos-2,Drift,1.0,1.0,1.0,0.567,0.421,0.704
-TimesFM-2.5,Toto-1.0,0.3,0.1,0.5,-0.028,-0.078,0.02
-TimesFM-2.5,Chronos-2,0.3,0.1,0.5,-0.039,-0.077,-0.009
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TiRex,0.65,0.45,0.85,0.035,-0.0,0.077
-TimesFM-2.5,Moirai-2.0,0.9,0.75,1.0,0.073,0.035,0.115
-TimesFM-2.5,Sundial-Base,0.85,0.7,1.0,0.089,0.041,0.137
-TimesFM-2.5,Chronos-Bolt,0.85,0.65,1.0,0.136,0.068,0.199
-TimesFM-2.5,TabPFN-TS,0.75,0.55,0.9,0.126,0.059,0.207
-TimesFM-2.5,Stat. Ensemble,1.0,1.0,1.0,0.334,0.261,0.415
-TimesFM-2.5,AutoARIMA,1.0,1.0,1.0,0.328,0.254,0.415
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.359,0.283,0.436
-TimesFM-2.5,AutoETS,1.0,1.0,1.0,0.344,0.268,0.433
-TimesFM-2.5,Naive,1.0,1.0,1.0,0.524,0.366,0.676
-TimesFM-2.5,Seasonal Naive,1.0,1.0,1.0,0.403,0.318,0.501
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.55,0.402,0.69
-TiRex,Toto-1.0,0.15,0.0,0.3,-0.065,-0.129,-0.028
-TiRex,Chronos-2,0.25,0.05,0.45,-0.077,-0.146,-0.022
-TiRex,TimesFM-2.5,0.35,0.15,0.55,-0.036,-0.083,0.0
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Moirai-2.0,0.8,0.6,0.95,0.039,0.014,0.065
-TiRex,Sundial-Base,0.85,0.7,1.0,0.056,-0.026,0.115
-TiRex,Chronos-Bolt,0.75,0.55,0.9,0.104,0.038,0.166
-TiRex,TabPFN-TS,0.7,0.5,0.9,0.095,0.018,0.172
-TiRex,Stat. Ensemble,1.0,1.0,1.0,0.31,0.233,0.392
-TiRex,AutoARIMA,1.0,1.0,1.0,0.303,0.231,0.382
-TiRex,AutoTheta,1.0,1.0,1.0,0.336,0.263,0.412
-TiRex,AutoETS,1.0,1.0,1.0,0.32,0.235,0.41
-TiRex,Naive,0.95,0.85,1.0,0.507,0.342,0.658
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.381,0.295,0.476
-TiRex,Drift,1.0,1.0,1.0,0.534,0.385,0.676
-Moirai-2.0,Toto-1.0,0.05,0.0,0.15,-0.109,-0.196,-0.056
-Moirai-2.0,Chronos-2,0.0,0.0,0.0,-0.121,-0.202,-0.056
-Moirai-2.0,TimesFM-2.5,0.1,0.0,0.25,-0.078,-0.13,-0.036
-Moirai-2.0,TiRex,0.2,0.05,0.4,-0.041,-0.07,-0.015
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Sundial-Base,0.8,0.6,0.95,0.017,-0.056,0.065
-Moirai-2.0,Chronos-Bolt,0.65,0.425,0.825,0.068,-0.002,0.135
-Moirai-2.0,TabPFN-TS,0.55,0.35,0.75,0.058,-0.02,0.138
-Moirai-2.0,Stat. Ensemble,0.95,0.85,1.0,0.282,0.201,0.366
-Moirai-2.0,AutoARIMA,0.95,0.85,1.0,0.275,0.203,0.354
-Moirai-2.0,AutoTheta,1.0,1.0,1.0,0.309,0.237,0.386
-Moirai-2.0,AutoETS,0.95,0.85,1.0,0.292,0.208,0.382
-Moirai-2.0,Naive,0.95,0.85,1.0,0.487,0.322,0.638
-Moirai-2.0,Seasonal Naive,0.95,0.85,1.0,0.356,0.268,0.451
-Moirai-2.0,Drift,1.0,1.0,1.0,0.515,0.369,0.657
-Sundial-Base,Toto-1.0,0.1,0.0,0.25,-0.128,-0.223,-0.038
-Sundial-Base,Chronos-2,0.1,0.0,0.25,-0.14,-0.222,-0.075
-Sundial-Base,TimesFM-2.5,0.15,0.0,0.3,-0.097,-0.159,-0.043
-Sundial-Base,TiRex,0.15,0.0,0.3,-0.059,-0.129,0.025
-Sundial-Base,Moirai-2.0,0.2,0.05,0.4,-0.018,-0.069,0.053
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Chronos-Bolt,0.5,0.25,0.7,0.051,-0.023,0.126
-Sundial-Base,TabPFN-TS,0.55,0.35,0.75,0.041,-0.034,0.11
-Sundial-Base,Stat. Ensemble,0.95,0.85,1.0,0.269,0.182,0.361
-Sundial-Base,AutoARIMA,0.95,0.85,1.0,0.262,0.175,0.353
-Sundial-Base,AutoTheta,1.0,1.0,1.0,0.296,0.22,0.377
-Sundial-Base,AutoETS,0.95,0.85,1.0,0.28,0.184,0.379
-Sundial-Base,Naive,1.0,1.0,1.0,0.478,0.322,0.626
-Sundial-Base,Seasonal Naive,0.95,0.85,1.0,0.344,0.246,0.453
-Sundial-Base,Drift,1.0,1.0,1.0,0.507,0.358,0.646
-Chronos-Bolt,Toto-1.0,0.1,0.0,0.25,-0.189,-0.288,-0.104
-Chronos-Bolt,Chronos-2,0.05,0.0,0.15,-0.202,-0.3,-0.112
-Chronos-Bolt,TimesFM-2.5,0.15,0.0,0.35,-0.157,-0.248,-0.073
-Chronos-Bolt,TiRex,0.25,0.1,0.45,-0.116,-0.198,-0.039
-Chronos-Bolt,Moirai-2.0,0.35,0.175,0.575,-0.073,-0.156,0.002
-Chronos-Bolt,Sundial-Base,0.5,0.3,0.75,-0.054,-0.144,0.022
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,TabPFN-TS,0.5,0.3,0.7,-0.011,-0.128,0.086
-Chronos-Bolt,Stat. Ensemble,0.9,0.75,1.0,0.23,0.124,0.336
-Chronos-Bolt,AutoARIMA,0.9,0.75,1.0,0.222,0.121,0.337
-Chronos-Bolt,AutoTheta,0.95,0.85,1.0,0.258,0.171,0.353
-Chronos-Bolt,AutoETS,0.85,0.7,1.0,0.24,0.131,0.358
-Chronos-Bolt,Naive,0.95,0.85,1.0,0.449,0.297,0.601
-Chronos-Bolt,Seasonal Naive,0.85,0.7,1.0,0.309,0.185,0.434
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.48,0.341,0.624
-TabPFN-TS,Toto-1.0,0.15,0.0,0.35,-0.176,-0.301,-0.081
-TabPFN-TS,Chronos-2,0.1,0.0,0.25,-0.189,-0.305,-0.102
-TabPFN-TS,TimesFM-2.5,0.25,0.1,0.45,-0.144,-0.261,-0.063
-TabPFN-TS,TiRex,0.3,0.1,0.5,-0.104,-0.208,-0.019
-TabPFN-TS,Moirai-2.0,0.45,0.25,0.65,-0.061,-0.161,0.019
-TabPFN-TS,Sundial-Base,0.45,0.25,0.65,-0.043,-0.123,0.033
-TabPFN-TS,Chronos-Bolt,0.5,0.3,0.7,0.011,-0.095,0.114
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Stat. Ensemble,0.8,0.6,0.95,0.238,0.095,0.36
-TabPFN-TS,AutoARIMA,0.8,0.6,0.95,0.231,0.096,0.351
-TabPFN-TS,AutoTheta,0.75,0.55,0.95,0.266,0.154,0.375
-TabPFN-TS,AutoETS,0.85,0.7,1.0,0.249,0.114,0.367
-TabPFN-TS,Naive,0.85,0.65,1.0,0.455,0.285,0.6
-TabPFN-TS,Seasonal Naive,0.9,0.75,1.0,0.316,0.178,0.443
-TabPFN-TS,Drift,0.95,0.85,1.0,0.485,0.326,0.621
-Stat. Ensemble,Toto-1.0,0.0,0.0,0.0,-0.544,-0.817,-0.362
-Stat. Ensemble,Chronos-2,0.0,0.0,0.0,-0.561,-0.833,-0.374
-Stat. Ensemble,TimesFM-2.5,0.0,0.0,0.0,-0.502,-0.711,-0.353
-Stat. Ensemble,TiRex,0.0,0.0,0.0,-0.449,-0.646,-0.304
-Stat. Ensemble,Moirai-2.0,0.05,0.0,0.15,-0.393,-0.578,-0.251
-Stat. Ensemble,Sundial-Base,0.05,0.0,0.15,-0.369,-0.564,-0.223
-Stat. Ensemble,Chronos-Bolt,0.1,0.0,0.25,-0.298,-0.505,-0.141
-Stat. Ensemble,TabPFN-TS,0.2,0.05,0.4,-0.312,-0.562,-0.106
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoARIMA,0.475,0.274,0.675,-0.01,-0.058,0.025
-Stat. Ensemble,AutoTheta,0.75,0.55,0.95,0.037,-0.013,0.087
-Stat. Ensemble,AutoETS,0.775,0.6,0.95,0.014,-0.091,0.087
-Stat. Ensemble,Naive,0.75,0.55,0.95,0.285,0.06,0.51
-Stat. Ensemble,Seasonal Naive,0.775,0.624,0.925,0.103,-0.009,0.19
-Stat. Ensemble,Drift,0.8,0.6,0.95,0.325,0.11,0.536
-AutoARIMA,Toto-1.0,0.0,0.0,0.0,-0.529,-0.811,-0.358
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-0.546,-0.83,-0.363
-AutoARIMA,TimesFM-2.5,0.0,0.0,0.0,-0.487,-0.708,-0.341
-AutoARIMA,TiRex,0.0,0.0,0.0,-0.435,-0.619,-0.301
-AutoARIMA,Moirai-2.0,0.05,0.0,0.15,-0.379,-0.548,-0.255
-AutoARIMA,Sundial-Base,0.05,0.0,0.15,-0.355,-0.545,-0.212
-AutoARIMA,Chronos-Bolt,0.1,0.0,0.25,-0.286,-0.508,-0.137
-AutoARIMA,TabPFN-TS,0.2,0.05,0.4,-0.3,-0.541,-0.106
-AutoARIMA,Stat. Ensemble,0.525,0.325,0.726,0.01,-0.026,0.055
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,AutoTheta,0.6,0.4,0.8,0.046,-0.017,0.107
-AutoARIMA,AutoETS,0.725,0.549,0.9,0.024,-0.086,0.103
-AutoARIMA,Naive,0.65,0.45,0.85,0.292,0.063,0.513
-AutoARIMA,Seasonal Naive,0.825,0.675,0.95,0.112,0.008,0.197
-AutoARIMA,Drift,0.8,0.6,0.95,0.331,0.115,0.536
-AutoTheta,Toto-1.0,0.0,0.0,0.0,-0.603,-0.881,-0.419
-AutoTheta,Chronos-2,0.0,0.0,0.0,-0.621,-0.887,-0.421
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-0.56,-0.774,-0.394
-AutoTheta,TiRex,0.0,0.0,0.0,-0.505,-0.701,-0.356
-AutoTheta,Moirai-2.0,0.0,0.0,0.0,-0.446,-0.628,-0.31
-AutoTheta,Sundial-Base,0.0,0.0,0.0,-0.421,-0.606,-0.282
-AutoTheta,Chronos-Bolt,0.05,0.0,0.15,-0.348,-0.546,-0.206
-AutoTheta,TabPFN-TS,0.25,0.05,0.45,-0.363,-0.6,-0.182
-AutoTheta,Stat. Ensemble,0.25,0.05,0.45,-0.039,-0.096,0.013
-AutoTheta,AutoARIMA,0.4,0.2,0.6,-0.049,-0.12,0.017
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,AutoETS,0.55,0.3,0.75,-0.024,-0.139,0.055
-AutoTheta,Naive,0.7,0.5,0.9,0.258,0.05,0.468
-AutoTheta,Seasonal Naive,0.8,0.65,0.95,0.068,-0.06,0.171
-AutoTheta,Drift,0.85,0.7,1.0,0.299,0.104,0.491
-AutoETS,Toto-1.0,0.0,0.0,0.0,-0.566,-0.883,-0.364
-AutoETS,Chronos-2,0.0,0.0,0.0,-0.583,-0.881,-0.391
-AutoETS,TimesFM-2.5,0.0,0.0,0.0,-0.523,-0.762,-0.366
-AutoETS,TiRex,0.0,0.0,0.0,-0.47,-0.695,-0.307
-AutoETS,Moirai-2.0,0.05,0.0,0.15,-0.412,-0.619,-0.262
-AutoETS,Sundial-Base,0.05,0.0,0.15,-0.388,-0.611,-0.225
-AutoETS,Chronos-Bolt,0.15,0.0,0.3,-0.317,-0.559,-0.151
-AutoETS,TabPFN-TS,0.15,0.0,0.3,-0.331,-0.581,-0.129
-AutoETS,Stat. Ensemble,0.225,0.05,0.4,-0.014,-0.096,0.083
-AutoETS,AutoARIMA,0.275,0.1,0.451,-0.024,-0.115,0.079
-AutoETS,AutoTheta,0.45,0.25,0.7,0.023,-0.058,0.122
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,Naive,0.55,0.3,0.75,0.275,0.032,0.5
-AutoETS,Seasonal Naive,0.725,0.55,0.9,0.09,0.026,0.154
-AutoETS,Drift,0.75,0.55,0.9,0.315,0.081,0.528
-Naive,Toto-1.0,0.0,0.0,0.0,-1.16,-2.211,-0.604
-Naive,Chronos-2,0.0,0.0,0.0,-1.183,-2.214,-0.628
-Naive,TimesFM-2.5,0.0,0.0,0.0,-1.101,-2.085,-0.578
-Naive,TiRex,0.05,0.0,0.15,-1.027,-1.923,-0.52
-Naive,Moirai-2.0,0.05,0.0,0.15,-0.948,-1.762,-0.475
-Naive,Sundial-Base,0.0,0.0,0.0,-0.914,-1.673,-0.476
-Naive,Chronos-Bolt,0.05,0.0,0.15,-0.816,-1.505,-0.423
-Naive,TabPFN-TS,0.15,0.0,0.35,-0.836,-1.501,-0.399
-Naive,Stat. Ensemble,0.25,0.05,0.45,-0.399,-1.041,-0.063
-Naive,AutoARIMA,0.35,0.15,0.55,-0.413,-1.052,-0.067
-Naive,AutoTheta,0.3,0.1,0.5,-0.347,-0.881,-0.053
-Naive,AutoETS,0.45,0.25,0.7,-0.379,-1.0,-0.033
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Seasonal Naive,0.6,0.425,0.775,-0.255,-0.874,0.087
-Naive,Drift,1.0,1.0,1.0,0.055,0.031,0.085
-Seasonal Naive,Toto-1.0,0.0,0.0,0.0,-0.721,-1.122,-0.477
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-0.74,-1.148,-0.496
-Seasonal Naive,TimesFM-2.5,0.0,0.0,0.0,-0.674,-1.003,-0.466
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.615,-0.909,-0.418
-Seasonal Naive,Moirai-2.0,0.05,0.0,0.15,-0.552,-0.822,-0.365
-Seasonal Naive,Sundial-Base,0.05,0.0,0.15,-0.525,-0.83,-0.326
-Seasonal Naive,Chronos-Bolt,0.15,0.0,0.3,-0.447,-0.766,-0.228
-Seasonal Naive,TabPFN-TS,0.1,0.0,0.25,-0.463,-0.796,-0.216
-Seasonal Naive,Stat. Ensemble,0.225,0.075,0.376,-0.115,-0.235,0.009
-Seasonal Naive,AutoARIMA,0.175,0.05,0.325,-0.126,-0.245,-0.008
-Seasonal Naive,AutoTheta,0.2,0.05,0.35,-0.073,-0.206,0.057
-Seasonal Naive,AutoETS,0.275,0.1,0.45,-0.099,-0.181,-0.027
-Seasonal Naive,Naive,0.4,0.225,0.575,0.203,-0.096,0.466
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,Drift,0.6,0.35,0.8,0.247,-0.04,0.497
-Drift,Toto-1.0,0.0,0.0,0.0,-1.286,-2.359,-0.705
-Drift,Chronos-2,0.0,0.0,0.0,-1.311,-2.381,-0.728
-Drift,TimesFM-2.5,0.0,0.0,0.0,-1.224,-2.223,-0.673
-Drift,TiRex,0.0,0.0,0.0,-1.146,-2.086,-0.627
-Drift,Moirai-2.0,0.0,0.0,0.0,-1.062,-1.913,-0.584
-Drift,Sundial-Base,0.0,0.0,0.0,-1.027,-1.828,-0.559
-Drift,Chronos-Bolt,0.0,0.0,0.0,-0.922,-1.659,-0.517
-Drift,TabPFN-TS,0.05,0.0,0.15,-0.943,-1.639,-0.484
-Drift,Stat. Ensemble,0.2,0.05,0.4,-0.481,-1.157,-0.124
-Drift,AutoARIMA,0.2,0.05,0.4,-0.495,-1.156,-0.13
-Drift,AutoTheta,0.15,0.0,0.3,-0.426,-0.966,-0.116
-Drift,AutoETS,0.25,0.1,0.45,-0.46,-1.12,-0.088
-Drift,Naive,0.0,0.0,0.0,-0.059,-0.093,-0.032
-Drift,Seasonal Naive,0.4,0.2,0.65,-0.329,-0.989,0.038
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_cloud/pairwise_SQL.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,Chronos-2,0.6,0.4,0.8,-0.024,-0.08,0.018
-Toto-1.0,TimesFM-2.5,0.75,0.55,0.95,0.028,-0.024,0.076
-Toto-1.0,TiRex,0.8,0.6,0.95,0.056,0.022,0.104
-Toto-1.0,Moirai-2.0,0.95,0.85,1.0,0.112,0.064,0.169
-Toto-1.0,Sundial-Base,0.95,0.85,1.0,0.148,0.071,0.211
-Toto-1.0,Chronos-Bolt,0.95,0.85,1.0,0.182,0.124,0.236
-Toto-1.0,TabPFN-TS,0.85,0.65,1.0,0.213,0.131,0.291
-Toto-1.0,AutoARIMA,1.0,1.0,1.0,0.452,0.369,0.537
-Toto-1.0,Stat. Ensemble,1.0,1.0,1.0,0.538,0.448,0.619
-Toto-1.0,AutoETS,1.0,1.0,1.0,0.666,0.482,0.815
-Toto-1.0,AutoTheta,1.0,1.0,1.0,0.638,0.54,0.72
-Toto-1.0,Seasonal Naive,1.0,1.0,1.0,0.631,0.521,0.73
-Toto-1.0,Naive,1.0,1.0,1.0,0.817,0.735,0.877
-Toto-1.0,Drift,1.0,1.0,1.0,0.824,0.744,0.882
-Chronos-2,Toto-1.0,0.4,0.2,0.6,0.024,-0.018,0.074
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,TimesFM-2.5,0.75,0.55,0.9,0.051,0.017,0.09
-Chronos-2,TiRex,0.75,0.55,0.9,0.078,0.024,0.139
-Chronos-2,Moirai-2.0,1.0,1.0,1.0,0.133,0.069,0.2
-Chronos-2,Sundial-Base,0.95,0.85,1.0,0.168,0.118,0.222
-Chronos-2,Chronos-Bolt,1.0,1.0,1.0,0.201,0.134,0.261
-Chronos-2,TabPFN-TS,0.95,0.85,1.0,0.232,0.162,0.3
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.465,0.381,0.55
-Chronos-2,Stat. Ensemble,1.0,1.0,1.0,0.549,0.47,0.623
-Chronos-2,AutoETS,1.0,1.0,1.0,0.674,0.501,0.819
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.647,0.555,0.727
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.639,0.533,0.735
-Chronos-2,Naive,1.0,1.0,1.0,0.822,0.746,0.879
-Chronos-2,Drift,1.0,1.0,1.0,0.828,0.756,0.883
-TimesFM-2.5,Toto-1.0,0.25,0.05,0.45,-0.029,-0.082,0.024
-TimesFM-2.5,Chronos-2,0.25,0.1,0.45,-0.054,-0.099,-0.017
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TiRex,0.6,0.4,0.8,0.028,-0.014,0.08
-TimesFM-2.5,Moirai-2.0,0.9,0.75,1.0,0.086,0.038,0.143
-TimesFM-2.5,Sundial-Base,0.95,0.85,1.0,0.123,0.079,0.165
-TimesFM-2.5,Chronos-Bolt,0.9,0.75,1.0,0.158,0.095,0.217
-TimesFM-2.5,TabPFN-TS,0.85,0.7,1.0,0.19,0.124,0.257
-TimesFM-2.5,AutoARIMA,1.0,1.0,1.0,0.436,0.367,0.511
-TimesFM-2.5,Stat. Ensemble,1.0,1.0,1.0,0.524,0.451,0.594
-TimesFM-2.5,AutoETS,1.0,1.0,1.0,0.657,0.476,0.811
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.628,0.534,0.71
-TimesFM-2.5,Seasonal Naive,1.0,1.0,1.0,0.62,0.519,0.713
-TimesFM-2.5,Naive,1.0,1.0,1.0,0.812,0.731,0.872
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.819,0.742,0.877
-TiRex,Toto-1.0,0.2,0.05,0.4,-0.059,-0.116,-0.023
-TiRex,Chronos-2,0.25,0.1,0.45,-0.085,-0.162,-0.025
-TiRex,TimesFM-2.5,0.4,0.2,0.6,-0.029,-0.086,0.013
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Moirai-2.0,0.85,0.7,1.0,0.059,0.03,0.09
-TiRex,Sundial-Base,0.95,0.85,1.0,0.097,0.013,0.151
-TiRex,Chronos-Bolt,0.9,0.75,1.0,0.133,0.073,0.186
-TiRex,TabPFN-TS,0.75,0.55,0.95,0.166,0.079,0.247
-TiRex,AutoARIMA,1.0,1.0,1.0,0.42,0.343,0.497
-TiRex,Stat. Ensemble,1.0,1.0,1.0,0.51,0.425,0.585
-TiRex,AutoETS,1.0,1.0,1.0,0.647,0.453,0.808
-TiRex,AutoTheta,1.0,1.0,1.0,0.617,0.513,0.698
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.609,0.499,0.709
-TiRex,Naive,0.95,0.85,1.0,0.806,0.72,0.869
-TiRex,Drift,1.0,1.0,1.0,0.814,0.729,0.874
-Moirai-2.0,Toto-1.0,0.05,0.0,0.15,-0.126,-0.203,-0.068
-Moirai-2.0,Chronos-2,0.0,0.0,0.0,-0.153,-0.249,-0.074
-Moirai-2.0,TimesFM-2.5,0.1,0.0,0.25,-0.094,-0.167,-0.039
-Moirai-2.0,TiRex,0.15,0.0,0.3,-0.063,-0.099,-0.03
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Sundial-Base,0.9,0.75,1.0,0.041,-0.045,0.091
-Moirai-2.0,Chronos-Bolt,0.8,0.6,0.95,0.079,0.017,0.14
-Moirai-2.0,TabPFN-TS,0.75,0.55,0.901,0.114,0.03,0.194
-Moirai-2.0,AutoARIMA,1.0,1.0,1.0,0.383,0.305,0.459
-Moirai-2.0,Stat. Ensemble,0.95,0.85,1.0,0.48,0.391,0.559
-Moirai-2.0,AutoETS,0.95,0.85,1.0,0.628,0.423,0.798
-Moirai-2.0,AutoTheta,0.95,0.85,1.0,0.593,0.49,0.673
-Moirai-2.0,Seasonal Naive,1.0,1.0,1.0,0.584,0.469,0.689
-Moirai-2.0,Naive,0.95,0.85,1.0,0.794,0.704,0.859
-Moirai-2.0,Drift,1.0,1.0,1.0,0.802,0.716,0.864
-Sundial-Base,Toto-1.0,0.05,0.0,0.15,-0.174,-0.268,-0.077
-Sundial-Base,Chronos-2,0.05,0.0,0.15,-0.202,-0.285,-0.134
-Sundial-Base,TimesFM-2.5,0.05,0.0,0.15,-0.14,-0.198,-0.085
-Sundial-Base,TiRex,0.05,0.0,0.15,-0.108,-0.178,-0.013
-Sundial-Base,Moirai-2.0,0.1,0.0,0.25,-0.043,-0.1,0.043
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Chronos-Bolt,0.4,0.2,0.6,0.04,-0.037,0.117
-Sundial-Base,TabPFN-TS,0.65,0.45,0.85,0.076,0.006,0.138
-Sundial-Base,AutoARIMA,1.0,1.0,1.0,0.357,0.281,0.436
-Sundial-Base,Stat. Ensemble,1.0,1.0,1.0,0.457,0.378,0.534
-Sundial-Base,AutoETS,1.0,1.0,1.0,0.615,0.404,0.793
-Sundial-Base,AutoTheta,1.0,1.0,1.0,0.575,0.487,0.657
-Sundial-Base,Seasonal Naive,1.0,1.0,1.0,0.566,0.454,0.668
-Sundial-Base,Naive,1.0,1.0,1.0,0.786,0.704,0.848
-Sundial-Base,Drift,1.0,1.0,1.0,0.794,0.716,0.854
-Chronos-Bolt,Toto-1.0,0.05,0.0,0.15,-0.222,-0.31,-0.142
-Chronos-Bolt,Chronos-2,0.0,0.0,0.0,-0.251,-0.352,-0.155
-Chronos-Bolt,TimesFM-2.5,0.1,0.0,0.25,-0.187,-0.277,-0.105
-Chronos-Bolt,TiRex,0.1,0.0,0.25,-0.154,-0.229,-0.079
-Chronos-Bolt,Moirai-2.0,0.2,0.05,0.4,-0.086,-0.162,-0.017
-Chronos-Bolt,Sundial-Base,0.6,0.4,0.8,-0.041,-0.133,0.036
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,TabPFN-TS,0.7,0.5,0.851,0.038,-0.082,0.139
-Chronos-Bolt,AutoARIMA,0.9,0.75,1.0,0.33,0.236,0.434
-Chronos-Bolt,Stat. Ensemble,0.85,0.7,1.0,0.435,0.328,0.531
-Chronos-Bolt,AutoETS,0.9,0.75,1.0,0.597,0.372,0.786
-Chronos-Bolt,AutoTheta,0.95,0.85,1.0,0.558,0.453,0.645
-Chronos-Bolt,Seasonal Naive,0.9,0.75,1.0,0.549,0.413,0.67
-Chronos-Bolt,Naive,0.95,0.85,1.0,0.777,0.686,0.844
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.785,0.7,0.851
-TabPFN-TS,Toto-1.0,0.15,0.0,0.35,-0.271,-0.41,-0.151
-TabPFN-TS,Chronos-2,0.05,0.0,0.15,-0.301,-0.429,-0.193
-TabPFN-TS,TimesFM-2.5,0.15,0.0,0.3,-0.235,-0.346,-0.141
-TabPFN-TS,TiRex,0.25,0.05,0.45,-0.2,-0.328,-0.086
-TabPFN-TS,Moirai-2.0,0.25,0.099,0.45,-0.129,-0.24,-0.03
-TabPFN-TS,Sundial-Base,0.35,0.15,0.55,-0.083,-0.161,-0.006
-TabPFN-TS,Chronos-Bolt,0.3,0.149,0.5,-0.04,-0.162,0.076
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,AutoARIMA,0.9,0.75,1.0,0.304,0.205,0.404
-TabPFN-TS,Stat. Ensemble,0.9,0.75,1.0,0.412,0.312,0.505
-TabPFN-TS,AutoETS,0.85,0.7,1.0,0.586,0.354,0.779
-TabPFN-TS,AutoTheta,0.95,0.85,1.0,0.54,0.433,0.639
-TabPFN-TS,Seasonal Naive,0.95,0.85,1.0,0.531,0.407,0.643
-TabPFN-TS,Naive,1.0,1.0,1.0,0.768,0.681,0.835
-TabPFN-TS,Drift,1.0,1.0,1.0,0.777,0.696,0.841
-AutoARIMA,Toto-1.0,0.0,0.0,0.0,-0.825,-1.161,-0.584
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-0.869,-1.222,-0.615
-AutoARIMA,TimesFM-2.5,0.0,0.0,0.0,-0.773,-1.043,-0.58
-AutoARIMA,TiRex,0.0,0.0,0.0,-0.723,-0.989,-0.521
-AutoARIMA,Moirai-2.0,0.0,0.0,0.0,-0.621,-0.848,-0.438
-AutoARIMA,Sundial-Base,0.0,0.0,0.0,-0.555,-0.773,-0.392
-AutoARIMA,Chronos-Bolt,0.1,0.0,0.25,-0.493,-0.768,-0.309
-AutoARIMA,TabPFN-TS,0.1,0.0,0.25,-0.436,-0.677,-0.258
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Stat. Ensemble,0.725,0.525,0.875,0.156,0.073,0.244
-AutoARIMA,AutoETS,0.775,0.6,0.95,0.434,0.08,0.713
-AutoARIMA,AutoTheta,0.85,0.7,1.0,0.34,0.203,0.476
-AutoARIMA,Seasonal Naive,0.875,0.75,0.975,0.326,0.18,0.457
-AutoARIMA,Naive,0.95,0.85,1.0,0.667,0.521,0.764
-AutoARIMA,Drift,0.95,0.85,1.0,0.679,0.545,0.773
-Stat. Ensemble,Toto-1.0,0.0,0.0,0.0,-1.163,-1.625,-0.811
-Stat. Ensemble,Chronos-2,0.0,0.0,0.0,-1.215,-1.654,-0.887
-Stat. Ensemble,TimesFM-2.5,0.0,0.0,0.0,-1.102,-1.466,-0.82
-Stat. Ensemble,TiRex,0.0,0.0,0.0,-1.042,-1.411,-0.739
-Stat. Ensemble,Moirai-2.0,0.05,0.0,0.15,-0.921,-1.267,-0.642
-Stat. Ensemble,Sundial-Base,0.0,0.0,0.0,-0.843,-1.145,-0.609
-Stat. Ensemble,Chronos-Bolt,0.15,0.0,0.3,-0.77,-1.134,-0.488
-Stat. Ensemble,TabPFN-TS,0.1,0.0,0.25,-0.702,-1.019,-0.454
-Stat. Ensemble,AutoARIMA,0.275,0.125,0.475,-0.185,-0.323,-0.078
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoETS,0.725,0.55,0.9,0.363,-0.063,0.684
-Stat. Ensemble,AutoTheta,0.9,0.75,1.0,0.218,0.089,0.375
-Stat. Ensemble,Seasonal Naive,0.725,0.525,0.9,0.201,-0.013,0.369
-Stat. Ensemble,Naive,0.95,0.85,1.0,0.605,0.443,0.725
-Stat. Ensemble,Drift,1.0,1.0,1.0,0.62,0.465,0.735
-AutoETS,Toto-1.0,0.0,0.0,0.0,-1.991,-4.394,-0.932
-AutoETS,Chronos-2,0.0,0.0,0.0,-2.072,-4.534,-1.004
-AutoETS,TimesFM-2.5,0.0,0.0,0.0,-1.92,-4.291,-0.907
-AutoETS,TiRex,0.0,0.0,0.0,-1.837,-4.196,-0.83
-AutoETS,Moirai-2.0,0.05,0.0,0.15,-1.688,-3.95,-0.734
-AutoETS,Sundial-Base,0.0,0.0,0.0,-1.595,-3.825,-0.677
-AutoETS,Chronos-Bolt,0.1,0.0,0.25,-1.484,-3.678,-0.591
-AutoETS,TabPFN-TS,0.15,0.0,0.3,-1.414,-3.528,-0.549
-AutoETS,AutoARIMA,0.225,0.05,0.4,-0.767,-2.48,-0.087
-AutoETS,Stat. Ensemble,0.275,0.1,0.45,-0.569,-2.162,0.059
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,AutoTheta,0.75,0.55,0.9,-0.25,-1.708,0.289
-AutoETS,Seasonal Naive,0.725,0.525,0.875,-0.269,-1.702,0.285
-AutoETS,Naive,0.8,0.6,0.95,0.34,-0.612,0.679
-AutoETS,Drift,0.85,0.65,1.0,0.363,-0.555,0.692
-AutoTheta,Toto-1.0,0.0,0.0,0.0,-1.764,-2.575,-1.173
-AutoTheta,Chronos-2,0.0,0.0,0.0,-1.831,-2.661,-1.245
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-1.686,-2.451,-1.145
-AutoTheta,TiRex,0.0,0.0,0.0,-1.61,-2.316,-1.054
-AutoTheta,Moirai-2.0,0.05,0.0,0.15,-1.456,-2.054,-0.962
-AutoTheta,Sundial-Base,0.0,0.0,0.0,-1.355,-1.914,-0.951
-AutoTheta,Chronos-Bolt,0.05,0.0,0.15,-1.262,-1.821,-0.828
-AutoTheta,TabPFN-TS,0.05,0.0,0.15,-1.175,-1.772,-0.765
-AutoTheta,AutoARIMA,0.15,0.0,0.3,-0.515,-0.908,-0.255
-AutoTheta,Stat. Ensemble,0.1,0.0,0.25,-0.278,-0.6,-0.097
-AutoTheta,AutoETS,0.25,0.1,0.45,0.2,-0.407,0.631
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Seasonal Naive,0.7,0.5,0.9,-0.021,-0.413,0.245
-AutoTheta,Naive,0.85,0.65,1.0,0.495,0.359,0.605
-AutoTheta,Drift,0.9,0.75,1.0,0.514,0.382,0.62
-Seasonal Naive,Toto-1.0,0.0,0.0,0.0,-1.707,-2.709,-1.086
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-1.772,-2.77,-1.139
-Seasonal Naive,TimesFM-2.5,0.0,0.0,0.0,-1.631,-2.486,-1.077
-Seasonal Naive,TiRex,0.0,0.0,0.0,-1.556,-2.437,-0.997
-Seasonal Naive,Moirai-2.0,0.0,0.0,0.0,-1.405,-2.214,-0.883
-Seasonal Naive,Sundial-Base,0.0,0.0,0.0,-1.307,-2.016,-0.832
-Seasonal Naive,Chronos-Bolt,0.1,0.0,0.25,-1.215,-2.027,-0.704
-Seasonal Naive,TabPFN-TS,0.05,0.0,0.15,-1.13,-1.801,-0.687
-Seasonal Naive,AutoARIMA,0.125,0.025,0.25,-0.483,-0.841,-0.219
-Seasonal Naive,Stat. Ensemble,0.275,0.1,0.475,-0.252,-0.585,0.013
-Seasonal Naive,AutoETS,0.275,0.125,0.475,0.212,-0.398,0.63
-Seasonal Naive,AutoTheta,0.3,0.1,0.5,0.021,-0.325,0.292
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,Naive,0.7,0.525,0.85,0.505,0.274,0.671
-Seasonal Naive,Drift,0.85,0.65,1.0,0.524,0.302,0.688
-Naive,Toto-1.0,0.0,0.0,0.0,-4.472,-7.119,-2.772
-Naive,Chronos-2,0.0,0.0,0.0,-4.604,-7.24,-2.94
-Naive,TimesFM-2.5,0.0,0.0,0.0,-4.318,-6.839,-2.721
-Naive,TiRex,0.05,0.0,0.15,-4.166,-6.661,-2.569
-Naive,Moirai-2.0,0.05,0.0,0.15,-3.861,-6.072,-2.376
-Naive,Sundial-Base,0.0,0.0,0.0,-3.663,-5.597,-2.374
-Naive,Chronos-Bolt,0.05,0.0,0.15,-3.478,-5.427,-2.188
-Naive,TabPFN-TS,0.0,0.0,0.0,-3.306,-5.057,-2.139
-Naive,AutoARIMA,0.05,0.0,0.15,-1.999,-3.24,-1.086
-Naive,Stat. Ensemble,0.05,0.0,0.15,-1.53,-2.642,-0.795
-Naive,AutoETS,0.2,0.05,0.4,-0.516,-2.117,0.38
-Naive,AutoTheta,0.15,0.0,0.35,-0.98,-1.531,-0.56
-Naive,Seasonal Naive,0.3,0.15,0.475,-1.021,-2.041,-0.378
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Drift,1.0,1.0,1.0,0.039,0.023,0.057
-Drift,Toto-1.0,0.0,0.0,0.0,-4.691,-7.48,-2.9
-Drift,Chronos-2,0.0,0.0,0.0,-4.828,-7.54,-3.102
-Drift,TimesFM-2.5,0.0,0.0,0.0,-4.531,-7.137,-2.871
-Drift,TiRex,0.0,0.0,0.0,-4.373,-6.957,-2.696
-Drift,Moirai-2.0,0.0,0.0,0.0,-4.056,-6.361,-2.518
-Drift,Sundial-Base,0.0,0.0,0.0,-3.85,-5.831,-2.522
-Drift,Chronos-Bolt,0.0,0.0,0.0,-3.658,-5.689,-2.332
-Drift,TabPFN-TS,0.0,0.0,0.0,-3.479,-5.272,-2.287
-Drift,AutoARIMA,0.05,0.0,0.15,-2.119,-3.399,-1.198
-Drift,Stat. Ensemble,0.0,0.0,0.0,-1.632,-2.779,-0.87
-Drift,AutoETS,0.15,0.0,0.35,-0.569,-2.25,0.357
-Drift,AutoTheta,0.1,0.0,0.25,-1.059,-1.634,-0.617
-Drift,Seasonal Naive,0.15,0.0,0.35,-1.102,-2.206,-0.433
-Drift,Naive,0.0,0.0,0.0,-0.04,-0.061,-0.024
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_cloud/pairwise_WAPE.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,Toto-1.0,0.35,0.15,0.55,0.065,-0.014,0.173
-Chronos-2,TimesFM-2.5,0.75,0.55,0.9,0.08,0.03,0.14
-Chronos-2,TiRex,0.8,0.6,0.95,0.109,0.032,0.209
-Chronos-2,Moirai-2.0,1.0,1.0,1.0,0.136,0.059,0.226
-Chronos-2,TabPFN-TS,0.95,0.85,1.0,0.129,0.084,0.186
-Chronos-2,Chronos-Bolt,1.0,1.0,1.0,0.178,0.103,0.259
-Chronos-2,Sundial-Base,1.0,1.0,1.0,0.148,0.099,0.198
-Chronos-2,Stat. Ensemble,1.0,1.0,1.0,0.415,0.319,0.51
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.421,0.312,0.531
-Chronos-2,AutoETS,1.0,1.0,1.0,0.476,0.341,0.599
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.434,0.333,0.532
-Chronos-2,Naive,1.0,1.0,1.0,0.482,0.382,0.572
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.506,0.388,0.617
-Chronos-2,Drift,1.0,1.0,1.0,0.514,0.42,0.599
-Toto-1.0,Chronos-2,0.65,0.45,0.85,-0.07,-0.21,0.014
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,TimesFM-2.5,0.8,0.6,0.95,0.016,-0.048,0.069
-Toto-1.0,TiRex,0.75,0.55,0.9,0.047,-0.0,0.1
-Toto-1.0,Moirai-2.0,0.9,0.75,1.0,0.076,0.017,0.14
-Toto-1.0,TabPFN-TS,0.8,0.6,0.95,0.069,-0.015,0.137
-Toto-1.0,Chronos-Bolt,0.95,0.85,1.0,0.12,0.053,0.182
-Toto-1.0,Sundial-Base,0.9,0.75,1.0,0.089,-0.011,0.164
-Toto-1.0,Stat. Ensemble,1.0,1.0,1.0,0.374,0.288,0.473
-Toto-1.0,AutoARIMA,1.0,1.0,1.0,0.38,0.29,0.482
-Toto-1.0,AutoETS,1.0,1.0,1.0,0.439,0.315,0.574
-Toto-1.0,AutoTheta,1.0,1.0,1.0,0.395,0.314,0.488
-Toto-1.0,Naive,1.0,1.0,1.0,0.446,0.353,0.545
-Toto-1.0,Seasonal Naive,1.0,1.0,1.0,0.471,0.37,0.581
-Toto-1.0,Drift,1.0,1.0,1.0,0.48,0.397,0.565
-TimesFM-2.5,Chronos-2,0.25,0.1,0.45,-0.088,-0.162,-0.031
-TimesFM-2.5,Toto-1.0,0.2,0.05,0.4,-0.017,-0.074,0.046
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TiRex,0.6,0.4,0.8,0.031,-0.009,0.083
-TimesFM-2.5,Moirai-2.0,0.85,0.65,1.0,0.06,0.024,0.101
-TimesFM-2.5,TabPFN-TS,0.7,0.5,0.9,0.053,0.014,0.094
-TimesFM-2.5,Chronos-Bolt,0.9,0.75,1.0,0.106,0.065,0.148
-TimesFM-2.5,Sundial-Base,0.9,0.75,1.0,0.074,0.024,0.112
-TimesFM-2.5,Stat. Ensemble,1.0,1.0,1.0,0.363,0.285,0.453
-TimesFM-2.5,AutoARIMA,1.0,1.0,1.0,0.37,0.28,0.472
-TimesFM-2.5,AutoETS,1.0,1.0,1.0,0.43,0.307,0.558
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.385,0.303,0.473
-TimesFM-2.5,Naive,1.0,1.0,1.0,0.437,0.35,0.527
-TimesFM-2.5,Seasonal Naive,1.0,1.0,1.0,0.462,0.364,0.568
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.472,0.391,0.548
-TiRex,Chronos-2,0.2,0.05,0.4,-0.122,-0.264,-0.033
-TiRex,Toto-1.0,0.25,0.1,0.45,-0.049,-0.111,0.0
-TiRex,TimesFM-2.5,0.4,0.2,0.6,-0.032,-0.091,0.009
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Moirai-2.0,0.75,0.55,0.9,0.03,0.004,0.056
-TiRex,TabPFN-TS,0.7,0.5,0.9,0.023,-0.05,0.081
-TiRex,Chronos-Bolt,0.85,0.65,1.0,0.077,0.03,0.124
-TiRex,Sundial-Base,0.85,0.7,0.95,0.044,-0.063,0.112
-TiRex,Stat. Ensemble,0.95,0.85,1.0,0.343,0.258,0.427
-TiRex,AutoARIMA,1.0,1.0,1.0,0.35,0.265,0.443
-TiRex,AutoETS,0.95,0.85,1.0,0.412,0.283,0.547
-TiRex,AutoTheta,1.0,1.0,1.0,0.365,0.286,0.45
-TiRex,Naive,0.95,0.85,1.0,0.419,0.321,0.511
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.445,0.35,0.545
-TiRex,Drift,1.0,1.0,1.0,0.455,0.377,0.533
-Moirai-2.0,Chronos-2,0.0,0.0,0.0,-0.157,-0.292,-0.062
-Moirai-2.0,Toto-1.0,0.1,0.0,0.25,-0.082,-0.162,-0.017
-Moirai-2.0,TimesFM-2.5,0.15,0.0,0.35,-0.064,-0.112,-0.025
-Moirai-2.0,TiRex,0.25,0.1,0.45,-0.031,-0.06,-0.004
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,TabPFN-TS,0.5,0.3,0.75,-0.008,-0.075,0.05
-Moirai-2.0,Chronos-Bolt,0.75,0.55,0.9,0.048,0.003,0.096
-Moirai-2.0,Sundial-Base,0.75,0.55,0.9,0.014,-0.074,0.068
-Moirai-2.0,Stat. Ensemble,1.0,1.0,1.0,0.322,0.239,0.411
-Moirai-2.0,AutoARIMA,1.0,1.0,1.0,0.33,0.244,0.426
-Moirai-2.0,AutoETS,1.0,1.0,1.0,0.394,0.265,0.533
-Moirai-2.0,AutoTheta,1.0,1.0,1.0,0.345,0.268,0.435
-Moirai-2.0,Naive,0.95,0.85,1.0,0.401,0.311,0.491
-Moirai-2.0,Seasonal Naive,1.0,1.0,1.0,0.428,0.333,0.532
-Moirai-2.0,Drift,1.0,1.0,1.0,0.438,0.36,0.513
-TabPFN-TS,Chronos-2,0.05,0.0,0.15,-0.149,-0.229,-0.091
-TabPFN-TS,Toto-1.0,0.2,0.05,0.4,-0.074,-0.159,0.015
-TabPFN-TS,TimesFM-2.5,0.3,0.1,0.5,-0.056,-0.103,-0.015
-TabPFN-TS,TiRex,0.3,0.1,0.5,-0.023,-0.089,0.048
-TabPFN-TS,Moirai-2.0,0.5,0.25,0.7,0.008,-0.053,0.07
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Chronos-Bolt,0.6,0.4,0.8,0.055,-0.022,0.125
-TabPFN-TS,Sundial-Base,0.5,0.3,0.7,0.022,-0.044,0.083
-TabPFN-TS,Stat. Ensemble,1.0,1.0,1.0,0.327,0.226,0.427
-TabPFN-TS,AutoARIMA,1.0,1.0,1.0,0.335,0.224,0.447
-TabPFN-TS,AutoETS,0.95,0.85,1.0,0.398,0.258,0.535
-TabPFN-TS,AutoTheta,0.95,0.85,1.0,0.35,0.246,0.448
-TabPFN-TS,Naive,1.0,1.0,1.0,0.405,0.295,0.51
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.432,0.311,0.55
-TabPFN-TS,Drift,1.0,1.0,1.0,0.442,0.337,0.534
-Chronos-Bolt,Chronos-2,0.0,0.0,0.0,-0.216,-0.349,-0.115
-Chronos-Bolt,Toto-1.0,0.05,0.0,0.15,-0.137,-0.223,-0.056
-Chronos-Bolt,TimesFM-2.5,0.1,0.0,0.25,-0.118,-0.174,-0.069
-Chronos-Bolt,TiRex,0.15,0.0,0.35,-0.083,-0.141,-0.031
-Chronos-Bolt,Moirai-2.0,0.25,0.1,0.45,-0.051,-0.106,-0.003
-Chronos-Bolt,TabPFN-TS,0.4,0.2,0.6,-0.059,-0.143,0.022
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,Sundial-Base,0.6,0.4,0.8,-0.036,-0.131,0.028
-Chronos-Bolt,Stat. Ensemble,1.0,1.0,1.0,0.288,0.207,0.384
-Chronos-Bolt,AutoARIMA,1.0,1.0,1.0,0.296,0.204,0.41
-Chronos-Bolt,AutoETS,1.0,1.0,1.0,0.363,0.23,0.509
-Chronos-Bolt,AutoTheta,1.0,1.0,1.0,0.312,0.234,0.407
-Chronos-Bolt,Naive,0.95,0.85,1.0,0.371,0.282,0.464
-Chronos-Bolt,Seasonal Naive,1.0,1.0,1.0,0.399,0.295,0.512
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.409,0.333,0.49
-Sundial-Base,Chronos-2,0.0,0.0,0.0,-0.174,-0.247,-0.11
-Sundial-Base,Toto-1.0,0.1,0.0,0.25,-0.097,-0.196,0.011
-Sundial-Base,TimesFM-2.5,0.1,0.0,0.25,-0.08,-0.126,-0.024
-Sundial-Base,TiRex,0.15,0.05,0.3,-0.046,-0.126,0.059
-Sundial-Base,Moirai-2.0,0.25,0.1,0.45,-0.014,-0.073,0.069
-Sundial-Base,TabPFN-TS,0.5,0.3,0.7,-0.022,-0.09,0.042
-Sundial-Base,Chronos-Bolt,0.4,0.2,0.6,0.034,-0.029,0.116
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Stat. Ensemble,1.0,1.0,1.0,0.313,0.227,0.408
-Sundial-Base,AutoARIMA,1.0,1.0,1.0,0.32,0.224,0.423
-Sundial-Base,AutoETS,1.0,1.0,1.0,0.385,0.251,0.518
-Sundial-Base,AutoTheta,1.0,1.0,1.0,0.336,0.245,0.426
-Sundial-Base,Naive,1.0,1.0,1.0,0.392,0.306,0.478
-Sundial-Base,Seasonal Naive,1.0,1.0,1.0,0.42,0.312,0.53
-Sundial-Base,Drift,1.0,1.0,1.0,0.43,0.347,0.507
-Stat. Ensemble,Chronos-2,0.0,0.0,0.0,-0.708,-1.042,-0.469
-Stat. Ensemble,Toto-1.0,0.0,0.0,0.0,-0.596,-0.898,-0.404
-Stat. Ensemble,TimesFM-2.5,0.0,0.0,0.0,-0.571,-0.828,-0.398
-Stat. Ensemble,TiRex,0.05,0.0,0.15,-0.522,-0.746,-0.348
-Stat. Ensemble,Moirai-2.0,0.0,0.0,0.0,-0.476,-0.699,-0.314
-Stat. Ensemble,TabPFN-TS,0.0,0.0,0.0,-0.487,-0.746,-0.292
-Stat. Ensemble,Chronos-Bolt,0.0,0.0,0.0,-0.405,-0.624,-0.261
-Stat. Ensemble,Sundial-Base,0.0,0.0,0.0,-0.455,-0.688,-0.293
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoARIMA,0.525,0.325,0.725,0.011,-0.045,0.063
-Stat. Ensemble,AutoETS,0.725,0.525,0.9,0.105,-0.033,0.285
-Stat. Ensemble,AutoTheta,0.85,0.7,1.0,0.033,-0.007,0.072
-Stat. Ensemble,Naive,0.7,0.5,0.9,0.116,0.037,0.193
-Stat. Ensemble,Seasonal Naive,0.875,0.75,0.975,0.156,0.072,0.236
-Stat. Ensemble,Drift,0.8,0.6,0.95,0.17,0.096,0.237
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-0.727,-1.133,-0.453
-AutoARIMA,Toto-1.0,0.0,0.0,0.0,-0.614,-0.93,-0.408
-AutoARIMA,TimesFM-2.5,0.0,0.0,0.0,-0.588,-0.895,-0.389
-AutoARIMA,TiRex,0.0,0.0,0.0,-0.539,-0.797,-0.361
-AutoARIMA,Moirai-2.0,0.0,0.0,0.0,-0.492,-0.744,-0.323
-AutoARIMA,TabPFN-TS,0.0,0.0,0.0,-0.503,-0.808,-0.289
-AutoARIMA,Chronos-Bolt,0.0,0.0,0.0,-0.42,-0.694,-0.256
-AutoARIMA,Sundial-Base,0.0,0.0,0.0,-0.471,-0.734,-0.288
-AutoARIMA,Stat. Ensemble,0.475,0.275,0.675,-0.011,-0.067,0.043
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,AutoETS,0.625,0.425,0.825,0.095,-0.068,0.276
-AutoARIMA,AutoTheta,0.65,0.4,0.85,0.023,-0.039,0.083
-AutoARIMA,Naive,0.6,0.399,0.8,0.106,-0.005,0.204
-AutoARIMA,Seasonal Naive,0.875,0.75,0.975,0.146,0.061,0.226
-AutoARIMA,Drift,0.75,0.55,0.95,0.161,0.062,0.25
-AutoETS,Chronos-2,0.0,0.0,0.0,-0.909,-1.495,-0.517
-AutoETS,Toto-1.0,0.0,0.0,0.0,-0.784,-1.345,-0.461
-AutoETS,TimesFM-2.5,0.0,0.0,0.0,-0.755,-1.264,-0.444
-AutoETS,TiRex,0.05,0.0,0.15,-0.7,-1.21,-0.394
-AutoETS,Moirai-2.0,0.0,0.0,0.0,-0.649,-1.141,-0.36
-AutoETS,TabPFN-TS,0.05,0.0,0.15,-0.662,-1.151,-0.347
-AutoETS,Chronos-Bolt,0.0,0.0,0.0,-0.57,-1.035,-0.299
-AutoETS,Sundial-Base,0.0,0.0,0.0,-0.626,-1.074,-0.334
-AutoETS,Stat. Ensemble,0.275,0.1,0.475,-0.117,-0.399,0.032
-AutoETS,AutoARIMA,0.375,0.175,0.575,-0.105,-0.381,0.063
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,AutoTheta,0.6,0.4,0.8,-0.08,-0.32,0.051
-AutoETS,Naive,0.5,0.3,0.7,0.012,-0.2,0.142
-AutoETS,Seasonal Naive,0.725,0.55,0.9,0.057,-0.196,0.202
-AutoETS,Drift,0.75,0.55,0.9,0.073,-0.137,0.197
-AutoTheta,Chronos-2,0.0,0.0,0.0,-0.767,-1.135,-0.5
-AutoTheta,Toto-1.0,0.0,0.0,0.0,-0.652,-0.955,-0.458
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-0.625,-0.898,-0.434
-AutoTheta,TiRex,0.0,0.0,0.0,-0.574,-0.817,-0.401
-AutoTheta,Moirai-2.0,0.0,0.0,0.0,-0.527,-0.77,-0.366
-AutoTheta,TabPFN-TS,0.05,0.0,0.15,-0.538,-0.812,-0.326
-AutoTheta,Chronos-Bolt,0.0,0.0,0.0,-0.453,-0.687,-0.305
-AutoTheta,Sundial-Base,0.0,0.0,0.0,-0.505,-0.741,-0.324
-AutoTheta,Stat. Ensemble,0.15,0.0,0.3,-0.035,-0.077,0.007
-AutoTheta,AutoARIMA,0.35,0.15,0.6,-0.023,-0.09,0.037
-AutoTheta,AutoETS,0.4,0.2,0.6,0.074,-0.054,0.242
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Naive,0.7,0.5,0.9,0.085,0.008,0.16
-AutoTheta,Seasonal Naive,0.8,0.65,0.95,0.127,0.043,0.209
-AutoTheta,Drift,0.85,0.7,1.0,0.142,0.077,0.202
-Naive,Chronos-2,0.0,0.0,0.0,-0.932,-1.338,-0.617
-Naive,Toto-1.0,0.0,0.0,0.0,-0.806,-1.198,-0.545
-Naive,TimesFM-2.5,0.0,0.0,0.0,-0.777,-1.112,-0.538
-Naive,TiRex,0.05,0.0,0.15,-0.721,-1.045,-0.472
-Naive,Moirai-2.0,0.05,0.0,0.15,-0.669,-0.964,-0.452
-Naive,TabPFN-TS,0.0,0.0,0.0,-0.682,-1.039,-0.417
-Naive,Chronos-Bolt,0.05,0.0,0.15,-0.589,-0.866,-0.392
-Naive,Sundial-Base,0.0,0.0,0.0,-0.646,-0.915,-0.441
-Naive,Stat. Ensemble,0.3,0.1,0.5,-0.131,-0.239,-0.038
-Naive,AutoARIMA,0.4,0.2,0.601,-0.119,-0.256,0.005
-Naive,AutoETS,0.5,0.3,0.7,-0.012,-0.166,0.167
-Naive,AutoTheta,0.3,0.1,0.5,-0.093,-0.19,-0.008
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Seasonal Naive,0.6,0.425,0.775,0.045,-0.064,0.152
-Naive,Drift,1.0,1.0,1.0,0.061,0.032,0.1
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-1.023,-1.609,-0.634
-Seasonal Naive,Toto-1.0,0.0,0.0,0.0,-0.891,-1.385,-0.588
-Seasonal Naive,TimesFM-2.5,0.0,0.0,0.0,-0.86,-1.313,-0.572
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.802,-1.198,-0.539
-Seasonal Naive,Moirai-2.0,0.0,0.0,0.0,-0.748,-1.138,-0.499
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.761,-1.22,-0.452
-Seasonal Naive,Chronos-Bolt,0.0,0.0,0.0,-0.664,-1.049,-0.418
-Seasonal Naive,Sundial-Base,0.0,0.0,0.0,-0.723,-1.13,-0.453
-Seasonal Naive,Stat. Ensemble,0.125,0.025,0.25,-0.185,-0.309,-0.078
-Seasonal Naive,AutoARIMA,0.125,0.025,0.25,-0.172,-0.293,-0.065
-Seasonal Naive,AutoETS,0.275,0.1,0.45,-0.06,-0.252,0.164
-Seasonal Naive,AutoTheta,0.2,0.05,0.35,-0.145,-0.264,-0.045
-Seasonal Naive,Naive,0.4,0.225,0.575,-0.047,-0.179,0.06
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,Drift,0.6,0.35,0.8,0.017,-0.096,0.115
-Drift,Chronos-2,0.0,0.0,0.0,-1.059,-1.492,-0.723
-Drift,Toto-1.0,0.0,0.0,0.0,-0.924,-1.3,-0.659
-Drift,TimesFM-2.5,0.0,0.0,0.0,-0.893,-1.214,-0.641
-Drift,TiRex,0.0,0.0,0.0,-0.834,-1.143,-0.606
-Drift,Moirai-2.0,0.0,0.0,0.0,-0.779,-1.054,-0.562
-Drift,TabPFN-TS,0.0,0.0,0.0,-0.792,-1.147,-0.508
-Drift,Chronos-Bolt,0.0,0.0,0.0,-0.693,-0.96,-0.5
-Drift,Sundial-Base,0.0,0.0,0.0,-0.753,-1.028,-0.532
-Drift,Stat. Ensemble,0.2,0.05,0.4,-0.205,-0.311,-0.106
-Drift,AutoARIMA,0.25,0.05,0.45,-0.192,-0.333,-0.066
-Drift,AutoETS,0.25,0.1,0.45,-0.079,-0.245,0.12
-Drift,AutoTheta,0.15,0.0,0.3,-0.165,-0.253,-0.083
-Drift,Naive,0.0,0.0,0.0,-0.065,-0.111,-0.033
-Drift,Seasonal Naive,0.4,0.2,0.65,-0.018,-0.129,0.088
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_cloud/pairwise_WQL.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,Toto-1.0,0.35,0.15,0.55,0.07,-0.014,0.185
-Chronos-2,TimesFM-2.5,0.75,0.55,0.9,0.088,0.033,0.155
-Chronos-2,TiRex,0.85,0.7,1.0,0.113,0.032,0.222
-Chronos-2,Moirai-2.0,1.0,1.0,1.0,0.151,0.066,0.248
-Chronos-2,TabPFN-TS,1.0,1.0,1.0,0.19,0.134,0.247
-Chronos-2,Chronos-Bolt,1.0,1.0,1.0,0.21,0.134,0.3
-Chronos-2,Sundial-Base,1.0,1.0,1.0,0.19,0.145,0.236
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.504,0.412,0.597
-Chronos-2,Stat. Ensemble,1.0,1.0,1.0,0.582,0.505,0.655
-Chronos-2,AutoETS,1.0,1.0,1.0,0.73,0.573,0.855
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.643,0.566,0.707
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.669,0.582,0.746
-Chronos-2,Naive,1.0,1.0,1.0,0.8,0.757,0.837
-Chronos-2,Drift,1.0,1.0,1.0,0.808,0.765,0.845
-Toto-1.0,Chronos-2,0.65,0.45,0.85,-0.075,-0.227,0.013
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,TimesFM-2.5,0.75,0.55,0.95,0.019,-0.047,0.073
-Toto-1.0,TiRex,0.75,0.55,0.9,0.046,0.0,0.095
-Toto-1.0,Moirai-2.0,0.85,0.7,1.0,0.087,0.03,0.145
-Toto-1.0,TabPFN-TS,0.8,0.6,0.95,0.129,0.024,0.218
-Toto-1.0,Chronos-Bolt,0.95,0.85,1.0,0.151,0.089,0.21
-Toto-1.0,Sundial-Base,0.9,0.75,1.0,0.129,0.024,0.201
-Toto-1.0,AutoARIMA,1.0,1.0,1.0,0.467,0.398,0.544
-Toto-1.0,Stat. Ensemble,1.0,1.0,1.0,0.551,0.473,0.628
-Toto-1.0,AutoETS,1.0,1.0,1.0,0.709,0.535,0.843
-Toto-1.0,AutoTheta,1.0,1.0,1.0,0.616,0.541,0.683
-Toto-1.0,Seasonal Naive,1.0,1.0,1.0,0.645,0.562,0.725
-Toto-1.0,Naive,1.0,1.0,1.0,0.785,0.726,0.829
-Toto-1.0,Drift,1.0,1.0,1.0,0.793,0.736,0.838
-TimesFM-2.5,Chronos-2,0.25,0.1,0.45,-0.097,-0.183,-0.034
-TimesFM-2.5,Toto-1.0,0.25,0.05,0.45,-0.02,-0.079,0.045
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TiRex,0.6,0.4,0.8,0.027,-0.018,0.086
-TimesFM-2.5,Moirai-2.0,0.85,0.65,1.0,0.069,0.028,0.116
-TimesFM-2.5,TabPFN-TS,0.7,0.5,0.9,0.112,0.044,0.188
-TimesFM-2.5,Chronos-Bolt,0.9,0.75,1.0,0.134,0.092,0.179
-TimesFM-2.5,Sundial-Base,0.95,0.85,1.0,0.111,0.06,0.148
-TimesFM-2.5,AutoARIMA,1.0,1.0,1.0,0.456,0.384,0.539
-TimesFM-2.5,Stat. Ensemble,1.0,1.0,1.0,0.542,0.468,0.619
-TimesFM-2.5,AutoETS,1.0,1.0,1.0,0.704,0.532,0.841
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.609,0.535,0.678
-TimesFM-2.5,Seasonal Naive,1.0,1.0,1.0,0.638,0.557,0.714
-TimesFM-2.5,Naive,1.0,1.0,1.0,0.78,0.727,0.825
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.789,0.739,0.832
-TiRex,Chronos-2,0.15,0.0,0.3,-0.127,-0.285,-0.033
-TiRex,Toto-1.0,0.25,0.1,0.45,-0.048,-0.105,-0.0
-TiRex,TimesFM-2.5,0.4,0.2,0.6,-0.028,-0.095,0.018
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Moirai-2.0,0.9,0.75,1.0,0.043,0.019,0.07
-TiRex,TabPFN-TS,0.75,0.55,0.95,0.087,-0.014,0.174
-TiRex,Chronos-Bolt,0.9,0.75,1.0,0.11,0.064,0.155
-TiRex,Sundial-Base,0.95,0.85,1.0,0.087,-0.025,0.153
-TiRex,AutoARIMA,1.0,1.0,1.0,0.441,0.376,0.512
-TiRex,Stat. Ensemble,1.0,1.0,1.0,0.529,0.452,0.602
-TiRex,AutoETS,1.0,1.0,1.0,0.697,0.514,0.839
-TiRex,AutoTheta,1.0,1.0,1.0,0.598,0.521,0.663
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.627,0.54,0.71
-TiRex,Naive,1.0,1.0,1.0,0.774,0.712,0.821
-TiRex,Drift,1.0,1.0,1.0,0.783,0.724,0.829
-Moirai-2.0,Chronos-2,0.0,0.0,0.0,-0.177,-0.33,-0.071
-Moirai-2.0,Toto-1.0,0.15,0.0,0.3,-0.095,-0.169,-0.031
-Moirai-2.0,TimesFM-2.5,0.15,0.0,0.35,-0.074,-0.131,-0.029
-Moirai-2.0,TiRex,0.1,0.0,0.25,-0.044,-0.075,-0.02
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,TabPFN-TS,0.5,0.3,0.7,0.047,-0.055,0.137
-Moirai-2.0,Chronos-Bolt,0.85,0.699,0.975,0.07,0.023,0.118
-Moirai-2.0,Sundial-Base,0.9,0.75,1.0,0.046,-0.056,0.104
-Moirai-2.0,AutoARIMA,1.0,1.0,1.0,0.416,0.347,0.496
-Moirai-2.0,Stat. Ensemble,1.0,1.0,1.0,0.508,0.423,0.589
-Moirai-2.0,AutoETS,1.0,1.0,1.0,0.685,0.494,0.835
-Moirai-2.0,AutoTheta,1.0,1.0,1.0,0.58,0.498,0.651
-Moirai-2.0,Seasonal Naive,1.0,1.0,1.0,0.611,0.523,0.693
-Moirai-2.0,Naive,1.0,1.0,1.0,0.764,0.7,0.813
-Moirai-2.0,Drift,1.0,1.0,1.0,0.773,0.715,0.821
-TabPFN-TS,Chronos-2,0.0,0.0,0.0,-0.235,-0.329,-0.154
-TabPFN-TS,Toto-1.0,0.2,0.05,0.4,-0.149,-0.279,-0.025
-TabPFN-TS,TimesFM-2.5,0.3,0.1,0.5,-0.126,-0.232,-0.046
-TabPFN-TS,TiRex,0.25,0.05,0.45,-0.096,-0.211,0.014
-TabPFN-TS,Moirai-2.0,0.5,0.3,0.7,-0.049,-0.159,0.053
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Chronos-Bolt,0.5,0.3,0.7,0.024,-0.094,0.121
-TabPFN-TS,Sundial-Base,0.5,0.3,0.7,-0.001,-0.093,0.08
-TabPFN-TS,AutoARIMA,0.95,0.85,1.0,0.387,0.274,0.491
-TabPFN-TS,Stat. Ensemble,0.95,0.85,1.0,0.484,0.39,0.568
-TabPFN-TS,AutoETS,0.95,0.85,1.0,0.676,0.465,0.829
-TabPFN-TS,AutoTheta,1.0,1.0,1.0,0.559,0.468,0.641
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.592,0.483,0.685
-TabPFN-TS,Naive,1.0,1.0,1.0,0.753,0.698,0.798
-TabPFN-TS,Drift,1.0,1.0,1.0,0.762,0.711,0.806
-Chronos-Bolt,Chronos-2,0.0,0.0,0.0,-0.266,-0.429,-0.154
-Chronos-Bolt,Toto-1.0,0.05,0.0,0.15,-0.177,-0.266,-0.097
-Chronos-Bolt,TimesFM-2.5,0.1,0.0,0.25,-0.154,-0.218,-0.101
-Chronos-Bolt,TiRex,0.1,0.0,0.25,-0.123,-0.183,-0.069
-Chronos-Bolt,Moirai-2.0,0.15,0.025,0.301,-0.075,-0.134,-0.024
-Chronos-Bolt,TabPFN-TS,0.5,0.3,0.7,-0.025,-0.137,0.086
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,Sundial-Base,0.55,0.35,0.75,-0.026,-0.135,0.046
-Chronos-Bolt,AutoARIMA,1.0,1.0,1.0,0.372,0.287,0.466
-Chronos-Bolt,Stat. Ensemble,1.0,1.0,1.0,0.471,0.385,0.561
-Chronos-Bolt,AutoETS,1.0,1.0,1.0,0.664,0.456,0.823
-Chronos-Bolt,AutoTheta,1.0,1.0,1.0,0.548,0.464,0.625
-Chronos-Bolt,Seasonal Naive,1.0,1.0,1.0,0.582,0.486,0.675
-Chronos-Bolt,Naive,1.0,1.0,1.0,0.746,0.682,0.796
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.756,0.698,0.805
-Sundial-Base,Chronos-2,0.0,0.0,0.0,-0.234,-0.309,-0.169
-Sundial-Base,Toto-1.0,0.1,0.0,0.25,-0.148,-0.251,-0.025
-Sundial-Base,TimesFM-2.5,0.05,0.0,0.15,-0.125,-0.173,-0.063
-Sundial-Base,TiRex,0.05,0.0,0.15,-0.095,-0.18,0.025
-Sundial-Base,Moirai-2.0,0.1,0.0,0.25,-0.048,-0.116,0.053
-Sundial-Base,TabPFN-TS,0.5,0.3,0.7,0.001,-0.086,0.085
-Sundial-Base,Chronos-Bolt,0.45,0.25,0.65,0.025,-0.048,0.119
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,AutoARIMA,1.0,1.0,1.0,0.388,0.302,0.48
-Sundial-Base,Stat. Ensemble,1.0,1.0,1.0,0.485,0.404,0.566
-Sundial-Base,AutoETS,1.0,1.0,1.0,0.672,0.475,0.826
-Sundial-Base,AutoTheta,1.0,1.0,1.0,0.56,0.48,0.633
-Sundial-Base,Seasonal Naive,1.0,1.0,1.0,0.592,0.504,0.674
-Sundial-Base,Naive,1.0,1.0,1.0,0.753,0.702,0.798
-Sundial-Base,Drift,1.0,1.0,1.0,0.763,0.715,0.808
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-1.016,-1.48,-0.701
-AutoARIMA,Toto-1.0,0.0,0.0,0.0,-0.875,-1.195,-0.66
-AutoARIMA,TimesFM-2.5,0.0,0.0,0.0,-0.839,-1.168,-0.623
-AutoARIMA,TiRex,0.0,0.0,0.0,-0.789,-1.05,-0.601
-AutoARIMA,Moirai-2.0,0.0,0.0,0.0,-0.713,-0.984,-0.532
-AutoARIMA,TabPFN-TS,0.05,0.0,0.15,-0.632,-0.965,-0.378
-AutoARIMA,Chronos-Bolt,0.0,0.0,0.0,-0.593,-0.873,-0.403
-AutoARIMA,Sundial-Base,0.0,0.0,0.0,-0.634,-0.922,-0.434
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Stat. Ensemble,0.775,0.6,0.925,0.158,0.066,0.258
-AutoARIMA,AutoETS,0.775,0.6,0.95,0.493,0.136,0.747
-AutoARIMA,AutoTheta,0.8,0.6,0.95,0.281,0.178,0.381
-AutoARIMA,Seasonal Naive,0.875,0.75,0.975,0.334,0.206,0.456
-AutoARIMA,Naive,0.9,0.75,1.0,0.596,0.479,0.687
-AutoARIMA,Drift,0.9,0.75,1.0,0.612,0.496,0.7
-Stat. Ensemble,Chronos-2,0.0,0.0,0.0,-1.395,-1.901,-1.02
-Stat. Ensemble,Toto-1.0,0.0,0.0,0.0,-1.227,-1.689,-0.897
-Stat. Ensemble,TimesFM-2.5,0.0,0.0,0.0,-1.184,-1.624,-0.88
-Stat. Ensemble,TiRex,0.0,0.0,0.0,-1.125,-1.511,-0.826
-Stat. Ensemble,Moirai-2.0,0.0,0.0,0.0,-1.034,-1.432,-0.734
-Stat. Ensemble,TabPFN-TS,0.05,0.0,0.15,-0.939,-1.317,-0.639
-Stat. Ensemble,Chronos-Bolt,0.0,0.0,0.0,-0.892,-1.278,-0.626
-Stat. Ensemble,Sundial-Base,0.0,0.0,0.0,-0.941,-1.304,-0.678
-Stat. Ensemble,AutoARIMA,0.225,0.075,0.4,-0.188,-0.348,-0.07
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoETS,0.725,0.55,0.9,0.432,0.004,0.725
-Stat. Ensemble,AutoTheta,0.9,0.75,1.0,0.146,0.088,0.213
-Stat. Ensemble,Seasonal Naive,0.725,0.525,0.9,0.208,-0.0,0.366
-Stat. Ensemble,Naive,0.95,0.85,1.0,0.52,0.395,0.624
-Stat. Ensemble,Drift,1.0,1.0,1.0,0.539,0.421,0.64
-AutoETS,Chronos-2,0.0,0.0,0.0,-2.71,-5.874,-1.34
-AutoETS,Toto-1.0,0.0,0.0,0.0,-2.437,-5.367,-1.149
-AutoETS,TimesFM-2.5,0.0,0.0,0.0,-2.382,-5.287,-1.137
-AutoETS,TiRex,0.0,0.0,0.0,-2.298,-5.196,-1.056
-AutoETS,Moirai-2.0,0.0,0.0,0.0,-2.171,-5.048,-0.977
-AutoETS,TabPFN-TS,0.05,0.0,0.15,-2.09,-4.847,-0.87
-AutoETS,Chronos-Bolt,0.0,0.0,0.0,-1.974,-4.644,-0.837
-AutoETS,Sundial-Base,0.0,0.0,0.0,-2.046,-4.752,-0.906
-AutoETS,AutoARIMA,0.225,0.05,0.4,-0.971,-2.951,-0.157
-AutoETS,Stat. Ensemble,0.275,0.1,0.45,-0.759,-2.642,-0.004
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,AutoTheta,0.65,0.45,0.85,-0.532,-2.194,0.136
-AutoETS,Seasonal Naive,0.725,0.525,0.875,-0.4,-1.897,0.22
-AutoETS,Naive,0.75,0.55,0.95,0.097,-1.154,0.533
-AutoETS,Drift,0.8,0.6,0.95,0.128,-1.084,0.55
-AutoTheta,Chronos-2,0.0,0.0,0.0,-1.802,-2.419,-1.302
-AutoTheta,Toto-1.0,0.0,0.0,0.0,-1.606,-2.154,-1.18
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-1.556,-2.107,-1.153
-AutoTheta,TiRex,0.0,0.0,0.0,-1.487,-1.971,-1.087
-AutoTheta,Moirai-2.0,0.0,0.0,0.0,-1.381,-1.863,-0.992
-AutoTheta,TabPFN-TS,0.0,0.0,0.0,-1.269,-1.786,-0.878
-AutoTheta,Chronos-Bolt,0.0,0.0,0.0,-1.214,-1.668,-0.867
-AutoTheta,Sundial-Base,0.0,0.0,0.0,-1.271,-1.728,-0.922
-AutoTheta,AutoARIMA,0.2,0.05,0.4,-0.39,-0.615,-0.216
-AutoTheta,Stat. Ensemble,0.1,0.0,0.25,-0.17,-0.271,-0.097
-AutoTheta,AutoETS,0.35,0.15,0.55,0.347,-0.157,0.687
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Seasonal Naive,0.7,0.5,0.9,0.074,-0.208,0.273
-AutoTheta,Naive,0.85,0.65,1.0,0.438,0.308,0.555
-AutoTheta,Drift,0.9,0.75,1.0,0.461,0.336,0.574
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-2.025,-2.936,-1.394
-Seasonal Naive,Toto-1.0,0.0,0.0,0.0,-1.813,-2.641,-1.285
-Seasonal Naive,TimesFM-2.5,0.0,0.0,0.0,-1.759,-2.495,-1.259
-Seasonal Naive,TiRex,0.0,0.0,0.0,-1.684,-2.448,-1.175
-Seasonal Naive,Moirai-2.0,0.0,0.0,0.0,-1.57,-2.259,-1.095
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-1.449,-2.176,-0.936
-Seasonal Naive,Chronos-Bolt,0.0,0.0,0.0,-1.39,-2.073,-0.945
-Seasonal Naive,Sundial-Base,0.0,0.0,0.0,-1.452,-2.065,-1.017
-Seasonal Naive,AutoARIMA,0.125,0.025,0.25,-0.5,-0.839,-0.259
-Seasonal Naive,Stat. Ensemble,0.275,0.1,0.475,-0.263,-0.577,0.0
-Seasonal Naive,AutoETS,0.275,0.125,0.475,0.286,-0.282,0.655
-Seasonal Naive,AutoTheta,0.3,0.1,0.5,-0.079,-0.376,0.172
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,Naive,0.7,0.525,0.85,0.394,0.19,0.555
-Seasonal Naive,Drift,0.85,0.65,1.0,0.418,0.219,0.575
-Naive,Chronos-2,0.0,0.0,0.0,-3.991,-5.118,-3.118
-Naive,Toto-1.0,0.0,0.0,0.0,-3.642,-4.863,-2.647
-Naive,TimesFM-2.5,0.0,0.0,0.0,-3.552,-4.721,-2.668
-Naive,TiRex,0.0,0.0,0.0,-3.428,-4.591,-2.469
-Naive,Moirai-2.0,0.0,0.0,0.0,-3.24,-4.339,-2.329
-Naive,TabPFN-TS,0.0,0.0,0.0,-3.041,-3.961,-2.314
-Naive,Chronos-Bolt,0.0,0.0,0.0,-2.943,-3.911,-2.141
-Naive,Sundial-Base,0.0,0.0,0.0,-3.045,-3.943,-2.361
-Naive,AutoARIMA,0.1,0.0,0.25,-1.475,-2.196,-0.918
-Naive,Stat. Ensemble,0.05,0.0,0.15,-1.084,-1.66,-0.654
-Naive,AutoETS,0.25,0.05,0.45,-0.107,-1.141,0.536
-Naive,AutoTheta,0.15,0.0,0.35,-0.781,-1.246,-0.445
-Naive,Seasonal Naive,0.3,0.15,0.475,-0.65,-1.249,-0.234
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Drift,1.0,1.0,1.0,0.039,0.023,0.06
-Drift,Chronos-2,0.0,0.0,0.0,-4.196,-5.431,-3.251
-Drift,Toto-1.0,0.0,0.0,0.0,-3.832,-5.185,-2.788
-Drift,TimesFM-2.5,0.0,0.0,0.0,-3.738,-4.957,-2.832
-Drift,TiRex,0.0,0.0,0.0,-3.61,-4.86,-2.621
-Drift,Moirai-2.0,0.0,0.0,0.0,-3.414,-4.579,-2.506
-Drift,TabPFN-TS,0.0,0.0,0.0,-3.207,-4.157,-2.457
-Drift,Chronos-Bolt,0.0,0.0,0.0,-3.105,-4.13,-2.31
-Drift,Sundial-Base,0.0,0.0,0.0,-3.211,-4.217,-2.506
-Drift,AutoARIMA,0.1,0.0,0.25,-1.577,-2.334,-0.985
-Drift,Stat. Ensemble,0.0,0.0,0.0,-1.17,-1.781,-0.726
-Drift,AutoETS,0.2,0.05,0.4,-0.147,-1.224,0.52
-Drift,AutoTheta,0.1,0.0,0.25,-0.854,-1.347,-0.507
-Drift,Seasonal Naive,0.15,0.0,0.35,-0.717,-1.355,-0.28
-Drift,Naive,0.0,0.0,0.0,-0.041,-0.064,-0.024
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_econ/leaderboard_MASE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Stat. Ensemble,86.30952380952381,38.477174892111854,0.0,74.63583592838711,0.0,0.0
-TiRex,75.5952380952381,36.87592043229662,0.0,0.1577310358200435,0.0,0.0
-Chronos-2,72.61904761904762,36.307128150625054,0.0,0.14296296275774828,0.0,0.0
-AutoETS,70.23809523809524,35.90887063284529,0.0,1.6644653908191018,0.0,0.0
-Toto-1.0,62.5,32.38399314015735,0.0,4.9976272313807435,0.16666666666666666,0.0
-AutoTheta,55.952380952380956,32.52926187728524,0.0,0.9511661320240354,0.0,0.0
-AutoARIMA,55.35714285714286,33.893658153503736,0.0,6.514016621697861,0.0,0.0
-Drift,54.76190476190476,29.639454010750132,0.0,0.41642842294117643,0.0,0.0
-TimesFM-2.5,50.5952380952381,32.83344752168508,0.0,0.3182864435592606,0.16666666666666666,0.0
-TabPFN-TS,44.04761904761905,28.784283550853363,0.0,19.382406450060035,0.0,0.0
-Chronos-Bolt,40.476190476190474,30.133730037212626,0.0,0.1589436945079739,0.0,0.0
-Moirai-2.0,39.285714285714285,28.236498726371508,0.0,0.2231920597172889,0.16666666666666666,0.0
-Sundial-Base,21.428571428571434,19.709514878708077,0.0,8.00438149490214,0.0,0.0
-Naive,13.69047619047619,15.593612409564518,0.0,0.43518170598039213,0.0,0.0
-Seasonal Naive,7.142857142857142,0.0,0.0,0.4101321087745098,0.0,0.0

tables/domain_econ/leaderboard_SQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-TiRex,86.30952380952381,38.167872268859085,0.0,0.1577310358200435,0.0,0.0
-Chronos-2,85.11904761904762,37.73601239308477,0.0,0.14296296275774828,0.0,0.0
-Stat. Ensemble,78.57142857142858,37.11985318962454,0.0,74.63583592838711,0.0,0.0
-Toto-1.0,65.47619047619048,33.46942523050611,0.0,4.9976272313807435,0.16666666666666666,0.0
-AutoETS,62.5,29.185864234223647,0.0,1.6644653908191018,0.0,0.0
-TimesFM-2.5,54.761904761904766,32.77207032704751,0.0,0.3182864435592606,0.16666666666666666,0.0
-AutoARIMA,50.5952380952381,31.721452512923896,0.0,6.514016621697861,0.0,0.0
-TabPFN-TS,50.000000000000014,31.399100116246935,0.0,19.382406450060035,0.0,0.0
-Chronos-Bolt,50.0,31.818155351289157,0.0,0.1589436945079739,0.0,0.0
-Drift,43.45238095238095,25.58270358587049,0.0,0.41642842294117643,0.0,0.0
-Moirai-2.0,43.45238095238095,29.079628772649933,0.0,0.2231920597172889,0.16666666666666666,0.0
-AutoTheta,42.26190476190476,29.618968091908858,0.0,0.9511661320240354,0.0,0.0
-Sundial-Base,16.666666666666668,14.51026077809361,0.0,8.00438149490214,0.0,0.0
-Naive,13.69047619047619,11.510639986489046,0.0,0.43518170598039213,0.0,0.0
-Seasonal Naive,7.142857142857142,0.0,0.0,0.4101321087745098,0.0,0.0

tables/domain_econ/leaderboard_WAPE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-TiRex,75.5952380952381,37.758720420324885,0.0,0.1577310358200435,0.0,0.0
-Stat. Ensemble,72.02380952380952,38.17771635240618,0.0,74.63583592838711,0.0,0.0
-Chronos-2,67.26190476190477,36.55873563764085,0.0,0.14296296275774828,0.0,0.0
-Toto-1.0,66.66666666666667,33.97027771884091,0.0,4.9976272313807435,0.16666666666666666,0.0
-TimesFM-2.5,63.09523809523808,35.892561173334556,0.0,0.3182864435592606,0.16666666666666666,0.0
-AutoETS,58.92857142857143,34.84116701303258,0.0,1.6644653908191018,0.0,0.0
-TabPFN-TS,58.333333333333336,34.28532719607597,0.0,19.382406450060035,0.0,0.0
-AutoARIMA,50.5952380952381,35.73217052389407,0.0,6.514016621697861,0.0,0.0
-Chronos-Bolt,49.404761904761905,30.47814442259772,0.0,0.1589436945079739,0.0,0.0
-Drift,48.214285714285715,28.435825322795704,0.0,0.41642842294117643,0.0,0.0
-Moirai-2.0,47.023809523809526,28.27751413794253,0.0,0.2231920597172889,0.16666666666666666,0.0
-AutoTheta,45.83333333333333,31.81292365948549,0.0,0.9511661320240354,0.0,0.0
-Naive,20.833333333333332,15.872311480991785,0.0,0.43518170598039213,0.0,0.0
-Sundial-Base,17.261904761904763,17.737236553561875,0.0,8.00438149490214,0.0,0.0
-Seasonal Naive,8.928571428571427,0.0,0.0,0.4101321087745098,0.0,0.0

tables/domain_econ/leaderboard_WQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-TiRex,83.33333333333334,42.09146585989647,0.0,0.1577310358200435,0.0,0.0
-Chronos-2,76.78571428571426,40.960757226189024,0.0,0.14296296275774828,0.0,0.0
-Toto-1.0,70.83333333333334,38.07263005492061,0.0,4.9976272313807435,0.16666666666666666,0.0
-TimesFM-2.5,66.07142857142857,39.26649292250398,0.0,0.3182864435592606,0.16666666666666666,0.0
-TabPFN-TS,62.5,38.29459805551885,0.0,19.382406450060035,0.0,0.0
-Stat. Ensemble,61.9047619047619,37.99224244127147,0.0,74.63583592838711,0.0,0.0
-Chronos-Bolt,58.33333333333333,35.53495230927829,0.0,0.1589436945079739,0.0,0.0
-AutoETS,55.35714285714286,36.76571975679717,0.0,1.6644653908191018,0.0,0.0
-Moirai-2.0,52.976190476190474,32.793225134905136,0.0,0.2231920597172889,0.16666666666666666,0.0
-AutoARIMA,46.42857142857143,34.4461529963365,0.0,6.514016621697861,0.0,0.0
-AutoTheta,36.90476190476191,29.146207716766202,0.0,0.9511661320240354,0.0,0.0
-Drift,33.92857142857143,23.82864899904751,0.0,0.41642842294117643,0.0,0.0
-Sundial-Base,19.642857142857142,17.532849713917752,0.0,8.00438149490214,0.0,0.0
-Naive,15.47619047619048,11.37195841028552,0.0,0.43518170598039213,0.0,0.0
-Seasonal Naive,9.523809523809524,0.0,0.0,0.4101321087745098,0.0,0.0

tables/domain_econ/pairwise_MASE.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,TiRex,0.75,0.5,1.0,0.025,-0.023,0.072
-Stat. Ensemble,Chronos-2,0.75,0.5,1.0,0.034,-0.014,0.087
-Stat. Ensemble,AutoETS,0.75,0.5,1.0,0.04,0.016,0.067
-Stat. Ensemble,Toto-1.0,0.75,0.417,1.0,0.09,0.025,0.15
-Stat. Ensemble,AutoTheta,0.917,0.75,1.0,0.088,0.053,0.126
-Stat. Ensemble,AutoARIMA,0.833,0.583,1.0,0.069,0.021,0.135
-Stat. Ensemble,Drift,0.917,0.75,1.0,0.126,0.051,0.221
-Stat. Ensemble,TimesFM-2.5,0.833,0.583,1.0,0.084,0.03,0.143
-Stat. Ensemble,TabPFN-TS,0.833,0.583,1.0,0.136,0.054,0.213
-Stat. Ensemble,Chronos-Bolt,0.917,0.75,1.0,0.119,0.068,0.168
-Stat. Ensemble,Moirai-2.0,0.917,0.75,1.0,0.143,0.081,0.197
-Stat. Ensemble,Sundial-Base,0.917,0.75,1.0,0.234,0.156,0.302
-Stat. Ensemble,Naive,1.0,1.0,1.0,0.271,0.195,0.346
-Stat. Ensemble,Seasonal Naive,1.0,1.0,1.0,0.385,0.289,0.476
-TiRex,Stat. Ensemble,0.25,0.0,0.5,-0.026,-0.078,0.023
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Chronos-2,0.583,0.333,0.833,0.009,-0.026,0.043
-TiRex,AutoETS,0.5,0.25,0.75,0.015,-0.013,0.045
-TiRex,Toto-1.0,0.583,0.333,0.833,0.066,-0.018,0.15
-TiRex,AutoTheta,0.833,0.583,1.0,0.064,-0.009,0.126
-TiRex,AutoARIMA,0.667,0.417,0.917,0.045,-0.033,0.133
-TiRex,Drift,0.75,0.5,1.0,0.103,-0.002,0.214
-TiRex,TimesFM-2.5,0.833,0.583,1.0,0.06,0.02,0.109
-TiRex,TabPFN-TS,0.833,0.583,1.0,0.114,0.033,0.194
-TiRex,Chronos-Bolt,0.917,0.75,1.0,0.097,0.05,0.149
-TiRex,Moirai-2.0,0.917,0.75,1.0,0.12,0.05,0.194
-TiRex,Sundial-Base,0.917,0.75,1.0,0.214,0.149,0.271
-TiRex,Naive,1.0,1.0,1.0,0.252,0.163,0.335
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.369,0.261,0.466
-Chronos-2,Stat. Ensemble,0.25,0.0,0.5,-0.035,-0.095,0.014
-Chronos-2,TiRex,0.417,0.167,0.667,-0.009,-0.045,0.026
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,AutoETS,0.5,0.25,0.75,0.006,-0.027,0.041
-Chronos-2,Toto-1.0,0.583,0.333,0.833,0.058,-0.037,0.145
-Chronos-2,AutoTheta,0.583,0.333,0.833,0.056,-0.023,0.124
-Chronos-2,AutoARIMA,0.667,0.417,0.917,0.037,-0.046,0.122
-Chronos-2,Drift,0.583,0.333,0.833,0.095,-0.024,0.22
-Chronos-2,TimesFM-2.5,0.75,0.5,1.0,0.052,0.004,0.099
-Chronos-2,TabPFN-TS,0.833,0.583,1.0,0.106,0.025,0.194
-Chronos-2,Chronos-Bolt,1.0,1.0,1.0,0.088,0.044,0.143
-Chronos-2,Moirai-2.0,1.0,1.0,1.0,0.112,0.045,0.186
-Chronos-2,Sundial-Base,1.0,1.0,1.0,0.207,0.156,0.255
-Chronos-2,Naive,1.0,1.0,1.0,0.245,0.159,0.328
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.363,0.264,0.451
-AutoETS,Stat. Ensemble,0.25,0.0,0.5,-0.042,-0.072,-0.016
-AutoETS,TiRex,0.5,0.25,0.75,-0.015,-0.047,0.013
-AutoETS,Chronos-2,0.5,0.25,0.75,-0.006,-0.043,0.026
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,Toto-1.0,0.583,0.333,0.833,0.052,-0.025,0.122
-AutoETS,AutoTheta,0.667,0.417,0.917,0.05,-0.003,0.1
-AutoETS,AutoARIMA,0.667,0.417,0.917,0.03,-0.036,0.112
-AutoETS,Drift,0.667,0.417,0.917,0.089,-0.005,0.196
-AutoETS,TimesFM-2.5,0.75,0.5,1.0,0.046,-0.003,0.098
-AutoETS,TabPFN-TS,0.667,0.417,0.917,0.1,0.02,0.183
-AutoETS,Chronos-Bolt,0.833,0.583,1.0,0.083,0.041,0.127
-AutoETS,Moirai-2.0,0.833,0.583,1.0,0.107,0.05,0.164
-AutoETS,Sundial-Base,0.917,0.75,1.0,0.202,0.132,0.262
-AutoETS,Naive,1.0,1.0,1.0,0.241,0.162,0.316
-AutoETS,Seasonal Naive,1.0,1.0,1.0,0.359,0.256,0.456
-Toto-1.0,Stat. Ensemble,0.25,0.0,0.583,-0.099,-0.177,-0.025
-Toto-1.0,TiRex,0.417,0.167,0.667,-0.071,-0.176,0.018
-Toto-1.0,Chronos-2,0.417,0.167,0.667,-0.062,-0.17,0.035
-Toto-1.0,AutoETS,0.417,0.167,0.667,-0.055,-0.139,0.024
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,AutoTheta,0.417,0.167,0.75,-0.002,-0.089,0.077
-Toto-1.0,AutoARIMA,0.583,0.333,0.833,-0.023,-0.108,0.048
-Toto-1.0,Drift,0.5,0.25,0.75,0.039,-0.065,0.133
-Toto-1.0,TimesFM-2.5,0.75,0.5,0.958,-0.007,-0.103,0.063
-Toto-1.0,TabPFN-TS,0.667,0.417,0.917,0.051,-0.107,0.186
-Toto-1.0,Chronos-Bolt,0.75,0.5,0.958,0.032,-0.034,0.096
-Toto-1.0,Moirai-2.0,0.833,0.625,0.958,0.058,0.018,0.106
-Toto-1.0,Sundial-Base,0.833,0.583,1.0,0.158,0.042,0.265
-Toto-1.0,Naive,1.0,1.0,1.0,0.199,0.116,0.285
-Toto-1.0,Seasonal Naive,0.917,0.75,1.0,0.324,0.199,0.433
-AutoTheta,Stat. Ensemble,0.083,0.0,0.25,-0.097,-0.144,-0.056
-AutoTheta,TiRex,0.167,0.0,0.417,-0.069,-0.144,0.009
-AutoTheta,Chronos-2,0.417,0.167,0.667,-0.059,-0.142,0.022
-AutoTheta,AutoETS,0.333,0.083,0.583,-0.053,-0.111,0.003
-AutoTheta,Toto-1.0,0.583,0.25,0.833,0.002,-0.083,0.082
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,AutoARIMA,0.5,0.25,0.75,-0.021,-0.102,0.059
-AutoTheta,Drift,0.333,0.083,0.583,0.041,-0.014,0.136
-AutoTheta,TimesFM-2.5,0.5,0.25,0.75,-0.005,-0.086,0.079
-AutoTheta,TabPFN-TS,0.583,0.25,0.833,0.053,-0.043,0.132
-AutoTheta,Chronos-Bolt,0.667,0.417,0.917,0.034,-0.034,0.1
-AutoTheta,Moirai-2.0,0.75,0.5,1.0,0.06,-0.011,0.123
-AutoTheta,Sundial-Base,0.917,0.75,1.0,0.16,0.076,0.236
-AutoTheta,Naive,1.0,1.0,1.0,0.201,0.136,0.266
-AutoTheta,Seasonal Naive,1.0,1.0,1.0,0.325,0.231,0.412
-AutoARIMA,Stat. Ensemble,0.167,0.0,0.417,-0.075,-0.156,-0.022
-AutoARIMA,TiRex,0.333,0.083,0.583,-0.047,-0.154,0.032
-AutoARIMA,Chronos-2,0.333,0.083,0.583,-0.038,-0.139,0.044
-AutoARIMA,AutoETS,0.333,0.083,0.583,-0.031,-0.126,0.035
-AutoARIMA,Toto-1.0,0.417,0.167,0.667,0.022,-0.051,0.097
-AutoARIMA,AutoTheta,0.5,0.25,0.75,0.02,-0.063,0.093
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Drift,0.5,0.25,0.75,0.06,-0.052,0.172
-AutoARIMA,TimesFM-2.5,0.583,0.25,0.833,0.016,-0.047,0.078
-AutoARIMA,TabPFN-TS,0.667,0.333,0.917,0.072,-0.036,0.171
-AutoARIMA,Chronos-Bolt,0.667,0.417,0.917,0.054,-0.009,0.113
-AutoARIMA,Moirai-2.0,0.583,0.333,0.833,0.079,-0.006,0.156
-AutoARIMA,Sundial-Base,0.75,0.5,1.0,0.177,0.068,0.264
-AutoARIMA,Naive,0.917,0.75,1.0,0.217,0.128,0.295
-AutoARIMA,Seasonal Naive,1.0,1.0,1.0,0.339,0.256,0.422
-Drift,Stat. Ensemble,0.083,0.0,0.25,-0.144,-0.284,-0.054
-Drift,TiRex,0.25,0.0,0.5,-0.115,-0.273,0.002
-Drift,Chronos-2,0.417,0.167,0.667,-0.105,-0.282,0.024
-Drift,AutoETS,0.333,0.083,0.583,-0.098,-0.244,0.005
-Drift,Toto-1.0,0.5,0.25,0.75,-0.041,-0.153,0.061
-Drift,AutoTheta,0.667,0.417,0.917,-0.043,-0.158,0.014
-Drift,AutoARIMA,0.5,0.25,0.75,-0.064,-0.208,0.05
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0
-Drift,TimesFM-2.5,0.417,0.167,0.667,-0.048,-0.227,0.071
-Drift,TabPFN-TS,0.583,0.25,0.833,0.012,-0.193,0.142
-Drift,Chronos-Bolt,0.583,0.333,0.833,-0.007,-0.119,0.086
-Drift,Moirai-2.0,0.667,0.417,0.917,0.02,-0.09,0.108
-Drift,Sundial-Base,0.833,0.583,1.0,0.124,-0.064,0.234
-Drift,Naive,0.917,0.75,1.0,0.166,0.089,0.241
-Drift,Seasonal Naive,0.917,0.75,1.0,0.296,0.127,0.41
-TimesFM-2.5,Stat. Ensemble,0.167,0.0,0.417,-0.092,-0.167,-0.031
-TimesFM-2.5,TiRex,0.167,0.0,0.417,-0.064,-0.122,-0.02
-TimesFM-2.5,Chronos-2,0.25,0.0,0.5,-0.055,-0.11,-0.004
-TimesFM-2.5,AutoETS,0.25,0.0,0.5,-0.048,-0.109,0.003
-TimesFM-2.5,Toto-1.0,0.25,0.042,0.5,0.007,-0.067,0.093
-TimesFM-2.5,AutoTheta,0.5,0.25,0.75,0.005,-0.086,0.079
-TimesFM-2.5,AutoARIMA,0.417,0.167,0.75,-0.016,-0.085,0.045
-TimesFM-2.5,Drift,0.583,0.333,0.833,0.045,-0.076,0.185
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TabPFN-TS,0.5,0.25,0.75,0.057,-0.034,0.15
-TimesFM-2.5,Chronos-Bolt,0.583,0.333,0.833,0.039,-0.007,0.091
-TimesFM-2.5,Moirai-2.0,0.667,0.417,0.917,0.064,-0.008,0.143
-TimesFM-2.5,Sundial-Base,0.917,0.75,1.0,0.163,0.095,0.228
-TimesFM-2.5,Naive,0.917,0.75,1.0,0.204,0.109,0.293
-TimesFM-2.5,Seasonal Naive,0.917,0.75,1.0,0.328,0.226,0.415
-TabPFN-TS,Stat. Ensemble,0.167,0.0,0.417,-0.158,-0.271,-0.057
-TabPFN-TS,TiRex,0.167,0.0,0.417,-0.128,-0.241,-0.034
-TabPFN-TS,Chronos-2,0.167,0.0,0.417,-0.118,-0.241,-0.026
-TabPFN-TS,AutoETS,0.333,0.083,0.583,-0.111,-0.224,-0.02
-TabPFN-TS,Toto-1.0,0.333,0.083,0.583,-0.053,-0.228,0.097
-TabPFN-TS,AutoTheta,0.417,0.167,0.75,-0.056,-0.153,0.042
-TabPFN-TS,AutoARIMA,0.333,0.083,0.667,-0.077,-0.207,0.035
-TabPFN-TS,Drift,0.417,0.167,0.75,-0.012,-0.165,0.162
-TabPFN-TS,TimesFM-2.5,0.5,0.25,0.75,-0.06,-0.177,0.033
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Chronos-Bolt,0.5,0.25,0.75,-0.019,-0.156,0.089
-TabPFN-TS,Moirai-2.0,0.5,0.25,0.75,0.008,-0.146,0.14
-TabPFN-TS,Sundial-Base,0.667,0.417,0.917,0.113,0.02,0.191
-TabPFN-TS,Naive,0.667,0.417,0.917,0.156,0.023,0.274
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.288,0.181,0.389
-Chronos-Bolt,Stat. Ensemble,0.083,0.0,0.25,-0.136,-0.202,-0.073
-Chronos-Bolt,TiRex,0.083,0.0,0.25,-0.107,-0.175,-0.053
-Chronos-Bolt,Chronos-2,0.0,0.0,0.0,-0.097,-0.167,-0.046
-Chronos-Bolt,AutoETS,0.167,0.0,0.417,-0.09,-0.145,-0.043
-Chronos-Bolt,Toto-1.0,0.25,0.042,0.5,-0.033,-0.107,0.033
-Chronos-Bolt,AutoTheta,0.333,0.083,0.583,-0.036,-0.112,0.033
-Chronos-Bolt,AutoARIMA,0.333,0.083,0.583,-0.057,-0.128,0.009
-Chronos-Bolt,Drift,0.417,0.167,0.667,0.007,-0.094,0.106
-Chronos-Bolt,TimesFM-2.5,0.417,0.167,0.667,-0.04,-0.1,0.007
-Chronos-Bolt,TabPFN-TS,0.5,0.25,0.75,0.019,-0.097,0.135
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,Moirai-2.0,0.417,0.167,0.667,0.026,-0.016,0.079
-Chronos-Bolt,Sundial-Base,0.917,0.75,1.0,0.13,0.036,0.202
-Chronos-Bolt,Naive,0.917,0.75,1.0,0.172,0.107,0.235
-Chronos-Bolt,Seasonal Naive,0.833,0.583,1.0,0.301,0.198,0.389
-Moirai-2.0,Stat. Ensemble,0.083,0.0,0.25,-0.166,-0.245,-0.088
-Moirai-2.0,TiRex,0.083,0.0,0.25,-0.137,-0.24,-0.053
-Moirai-2.0,Chronos-2,0.0,0.0,0.0,-0.127,-0.229,-0.048
-Moirai-2.0,AutoETS,0.167,0.0,0.417,-0.12,-0.196,-0.053
-Moirai-2.0,Toto-1.0,0.167,0.042,0.375,-0.061,-0.118,-0.018
-Moirai-2.0,AutoTheta,0.25,0.0,0.5,-0.064,-0.141,0.011
-Moirai-2.0,AutoARIMA,0.417,0.167,0.667,-0.086,-0.185,0.006
-Moirai-2.0,Drift,0.333,0.083,0.583,-0.02,-0.121,0.083
-Moirai-2.0,TimesFM-2.5,0.333,0.083,0.583,-0.068,-0.166,0.008
-Moirai-2.0,TabPFN-TS,0.5,0.25,0.75,-0.008,-0.163,0.127
-Moirai-2.0,Chronos-Bolt,0.583,0.333,0.833,-0.027,-0.086,0.015
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Sundial-Base,0.833,0.583,1.0,0.106,-0.0,0.194
-Moirai-2.0,Naive,0.917,0.75,1.0,0.15,0.086,0.21
-Moirai-2.0,Seasonal Naive,0.833,0.583,1.0,0.282,0.162,0.388
-Sundial-Base,Stat. Ensemble,0.083,0.0,0.25,-0.305,-0.434,-0.185
-Sundial-Base,TiRex,0.083,0.0,0.25,-0.272,-0.372,-0.176
-Sundial-Base,Chronos-2,0.0,0.0,0.0,-0.261,-0.342,-0.184
-Sundial-Base,AutoETS,0.083,0.0,0.25,-0.253,-0.356,-0.153
-Sundial-Base,Toto-1.0,0.167,0.0,0.417,-0.187,-0.361,-0.044
-Sundial-Base,AutoTheta,0.083,0.0,0.25,-0.19,-0.309,-0.083
-Sundial-Base,AutoARIMA,0.25,0.0,0.5,-0.215,-0.358,-0.073
-Sundial-Base,Drift,0.167,0.0,0.417,-0.141,-0.305,0.06
-Sundial-Base,TimesFM-2.5,0.083,0.0,0.25,-0.195,-0.296,-0.104
-Sundial-Base,TabPFN-TS,0.333,0.083,0.583,-0.127,-0.237,-0.021
-Sundial-Base,Chronos-Bolt,0.083,0.0,0.25,-0.149,-0.254,-0.037
-Sundial-Base,Moirai-2.0,0.167,0.0,0.417,-0.119,-0.24,0.0
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Naive,0.583,0.333,0.833,0.049,-0.051,0.164
-Sundial-Base,Seasonal Naive,0.833,0.583,1.0,0.197,0.082,0.299
-Naive,Stat. Ensemble,0.0,0.0,0.0,-0.372,-0.529,-0.242
-Naive,TiRex,0.0,0.0,0.0,-0.337,-0.504,-0.194
-Naive,Chronos-2,0.0,0.0,0.0,-0.325,-0.488,-0.189
-Naive,AutoETS,0.0,0.0,0.0,-0.317,-0.462,-0.193
-Naive,Toto-1.0,0.0,0.0,0.0,-0.248,-0.398,-0.131
-Naive,AutoTheta,0.0,0.0,0.0,-0.251,-0.362,-0.157
-Naive,AutoARIMA,0.083,0.0,0.25,-0.277,-0.419,-0.147
-Naive,Drift,0.083,0.0,0.25,-0.2,-0.318,-0.098
-Naive,TimesFM-2.5,0.083,0.0,0.25,-0.257,-0.415,-0.122
-Naive,TabPFN-TS,0.333,0.083,0.583,-0.185,-0.377,-0.024
-Naive,Chronos-Bolt,0.083,0.0,0.25,-0.208,-0.307,-0.119
-Naive,Moirai-2.0,0.083,0.0,0.25,-0.176,-0.265,-0.094
-Naive,Sundial-Base,0.417,0.167,0.667,-0.051,-0.197,0.049
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Seasonal Naive,0.75,0.542,0.917,0.156,0.021,0.261
-Seasonal Naive,Stat. Ensemble,0.0,0.0,0.0,-0.625,-0.907,-0.407
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.584,-0.872,-0.353
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-0.57,-0.821,-0.358
-Seasonal Naive,AutoETS,0.0,0.0,0.0,-0.56,-0.837,-0.345
-Seasonal Naive,Toto-1.0,0.083,0.0,0.25,-0.479,-0.763,-0.248
-Seasonal Naive,AutoTheta,0.0,0.0,0.0,-0.482,-0.7,-0.301
-Seasonal Naive,AutoARIMA,0.0,0.0,0.0,-0.513,-0.73,-0.344
-Seasonal Naive,Drift,0.083,0.0,0.25,-0.421,-0.694,-0.146
-Seasonal Naive,TimesFM-2.5,0.083,0.0,0.25,-0.489,-0.709,-0.292
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.404,-0.637,-0.222
-Seasonal Naive,Chronos-Bolt,0.167,0.0,0.417,-0.431,-0.638,-0.246
-Seasonal Naive,Moirai-2.0,0.167,0.0,0.417,-0.393,-0.634,-0.193
-Seasonal Naive,Sundial-Base,0.167,0.0,0.417,-0.245,-0.427,-0.09
-Seasonal Naive,Naive,0.25,0.083,0.458,-0.185,-0.353,-0.022
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_econ/pairwise_SQL.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Chronos-2,0.667,0.417,0.917,0.007,-0.028,0.036
-TiRex,Stat. Ensemble,0.75,0.5,1.0,0.017,-0.034,0.06
-TiRex,Toto-1.0,0.667,0.333,0.917,0.071,-0.005,0.143
-TiRex,AutoETS,0.75,0.5,1.0,0.127,0.025,0.268
-TiRex,TimesFM-2.5,0.917,0.75,1.0,0.08,0.038,0.122
-TiRex,AutoARIMA,0.833,0.583,1.0,0.094,0.019,0.181
-TiRex,Chronos-Bolt,0.917,0.75,1.0,0.093,0.05,0.141
-TiRex,TabPFN-TS,0.833,0.583,1.0,0.099,0.031,0.166
-TiRex,Moirai-2.0,0.917,0.75,1.0,0.128,0.063,0.197
-TiRex,Drift,0.917,0.75,1.0,0.169,0.055,0.293
-TiRex,AutoTheta,0.917,0.75,1.0,0.121,0.053,0.185
-TiRex,Sundial-Base,1.0,1.0,1.0,0.277,0.209,0.334
-TiRex,Naive,1.0,1.0,1.0,0.301,0.213,0.382
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.382,0.282,0.467
-Chronos-2,TiRex,0.333,0.083,0.583,-0.007,-0.037,0.027
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,Stat. Ensemble,0.833,0.583,1.0,0.01,-0.042,0.051
-Chronos-2,Toto-1.0,0.667,0.333,0.917,0.064,-0.02,0.141
-Chronos-2,AutoETS,0.75,0.5,1.0,0.121,0.02,0.261
-Chronos-2,TimesFM-2.5,0.917,0.75,1.0,0.074,0.039,0.111
-Chronos-2,AutoARIMA,0.75,0.5,1.0,0.088,0.009,0.174
-Chronos-2,Chronos-Bolt,1.0,1.0,1.0,0.087,0.043,0.142
-Chronos-2,TabPFN-TS,0.833,0.583,1.0,0.092,0.031,0.161
-Chronos-2,Moirai-2.0,1.0,1.0,1.0,0.122,0.059,0.192
-Chronos-2,Drift,0.917,0.75,1.0,0.163,0.036,0.303
-Chronos-2,AutoTheta,0.917,0.75,1.0,0.115,0.046,0.178
-Chronos-2,Sundial-Base,1.0,1.0,1.0,0.272,0.215,0.324
-Chronos-2,Naive,1.0,1.0,1.0,0.296,0.202,0.388
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.377,0.286,0.457
-Stat. Ensemble,TiRex,0.25,0.0,0.5,-0.017,-0.064,0.033
-Stat. Ensemble,Chronos-2,0.167,0.0,0.417,-0.01,-0.053,0.04
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,Toto-1.0,0.667,0.333,0.917,0.055,-0.011,0.117
-Stat. Ensemble,AutoETS,0.667,0.417,0.917,0.112,0.011,0.257
-Stat. Ensemble,TimesFM-2.5,0.833,0.583,1.0,0.065,0.013,0.121
-Stat. Ensemble,AutoARIMA,0.833,0.583,1.0,0.079,0.03,0.146
-Stat. Ensemble,Chronos-Bolt,0.917,0.75,1.0,0.078,0.024,0.131
-Stat. Ensemble,TabPFN-TS,0.833,0.583,1.0,0.083,0.023,0.143
-Stat. Ensemble,Moirai-2.0,0.917,0.75,1.0,0.113,0.048,0.171
-Stat. Ensemble,Drift,0.917,0.75,1.0,0.155,0.061,0.279
-Stat. Ensemble,AutoTheta,1.0,1.0,1.0,0.107,0.07,0.148
-Stat. Ensemble,Sundial-Base,1.0,1.0,1.0,0.264,0.191,0.329
-Stat. Ensemble,Naive,1.0,1.0,1.0,0.289,0.206,0.375
-Stat. Ensemble,Seasonal Naive,1.0,1.0,1.0,0.371,0.288,0.452
-Toto-1.0,TiRex,0.333,0.083,0.667,-0.076,-0.167,0.005
-Toto-1.0,Chronos-2,0.333,0.083,0.667,-0.069,-0.164,0.019
-Toto-1.0,Stat. Ensemble,0.333,0.083,0.667,-0.058,-0.133,0.011
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,AutoETS,0.5,0.25,0.752,0.06,-0.086,0.229
-Toto-1.0,TimesFM-2.5,0.75,0.5,0.958,0.01,-0.075,0.075
-Toto-1.0,AutoARIMA,0.667,0.417,0.917,0.026,-0.06,0.101
-Toto-1.0,Chronos-Bolt,0.667,0.417,0.875,0.024,-0.031,0.081
-Toto-1.0,TabPFN-TS,0.667,0.417,0.917,0.03,-0.103,0.146
-Toto-1.0,Moirai-2.0,0.833,0.625,0.958,0.062,0.022,0.109
-Toto-1.0,Drift,0.75,0.5,1.0,0.106,-0.014,0.216
-Toto-1.0,AutoTheta,0.667,0.417,0.917,0.055,-0.028,0.135
-Toto-1.0,Sundial-Base,0.833,0.583,1.0,0.222,0.115,0.321
-Toto-1.0,Naive,0.917,0.75,1.0,0.248,0.157,0.335
-Toto-1.0,Seasonal Naive,0.917,0.75,1.0,0.335,0.216,0.435
-AutoETS,TiRex,0.25,0.0,0.5,-0.145,-0.367,-0.025
-AutoETS,Chronos-2,0.25,0.0,0.5,-0.137,-0.354,-0.02
-AutoETS,Stat. Ensemble,0.333,0.083,0.583,-0.126,-0.345,-0.012
-AutoETS,Toto-1.0,0.5,0.248,0.75,-0.064,-0.297,0.08
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,TimesFM-2.5,0.5,0.25,0.75,-0.053,-0.259,0.072
-AutoETS,AutoARIMA,0.5,0.167,0.75,-0.037,-0.271,0.112
-AutoETS,Chronos-Bolt,0.75,0.5,1.0,-0.039,-0.251,0.086
-AutoETS,TabPFN-TS,0.667,0.417,0.917,-0.032,-0.263,0.102
-AutoETS,Moirai-2.0,0.75,0.5,1.0,0.001,-0.211,0.131
-AutoETS,Drift,0.75,0.5,1.0,0.048,-0.2,0.237
-AutoETS,AutoTheta,0.75,0.5,1.0,-0.006,-0.228,0.119
-AutoETS,Sundial-Base,0.917,0.75,1.0,0.172,0.002,0.286
-AutoETS,Naive,0.917,0.75,1.0,0.2,0.001,0.337
-AutoETS,Seasonal Naive,0.917,0.75,1.0,0.292,0.084,0.43
-TimesFM-2.5,TiRex,0.083,0.0,0.25,-0.087,-0.139,-0.039
-TimesFM-2.5,Chronos-2,0.083,0.0,0.25,-0.08,-0.124,-0.041
-TimesFM-2.5,Stat. Ensemble,0.167,0.0,0.417,-0.069,-0.137,-0.014
-TimesFM-2.5,Toto-1.0,0.25,0.042,0.5,-0.01,-0.081,0.07
-TimesFM-2.5,AutoETS,0.5,0.25,0.75,0.051,-0.077,0.205
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,AutoARIMA,0.583,0.333,0.833,0.015,-0.066,0.089
-TimesFM-2.5,Chronos-Bolt,0.583,0.333,0.833,0.014,-0.034,0.072
-TimesFM-2.5,TabPFN-TS,0.5,0.25,0.75,0.02,-0.059,0.098
-TimesFM-2.5,Moirai-2.0,0.583,0.333,0.833,0.052,-0.014,0.127
-TimesFM-2.5,Drift,0.75,0.5,1.0,0.097,-0.046,0.255
-TimesFM-2.5,AutoTheta,0.75,0.5,1.0,0.045,-0.043,0.116
-TimesFM-2.5,Sundial-Base,1.0,1.0,1.0,0.214,0.15,0.272
-TimesFM-2.5,Naive,0.917,0.75,1.0,0.24,0.136,0.34
-TimesFM-2.5,Seasonal Naive,0.917,0.75,1.0,0.328,0.224,0.407
-AutoARIMA,TiRex,0.167,0.0,0.417,-0.104,-0.221,-0.019
-AutoARIMA,Chronos-2,0.25,0.0,0.5,-0.097,-0.21,-0.009
-AutoARIMA,Stat. Ensemble,0.167,0.0,0.417,-0.086,-0.171,-0.031
-AutoARIMA,Toto-1.0,0.333,0.083,0.583,-0.026,-0.112,0.056
-AutoARIMA,AutoETS,0.5,0.25,0.833,0.036,-0.126,0.213
-AutoARIMA,TimesFM-2.5,0.417,0.167,0.667,-0.016,-0.097,0.062
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Chronos-Bolt,0.417,0.167,0.667,-0.001,-0.08,0.069
-AutoARIMA,TabPFN-TS,0.5,0.167,0.75,0.005,-0.101,0.09
-AutoARIMA,Moirai-2.0,0.5,0.25,0.75,0.037,-0.064,0.125
-AutoARIMA,Drift,0.583,0.333,0.833,0.082,-0.049,0.224
-AutoARIMA,AutoTheta,0.583,0.25,0.833,0.03,-0.058,0.107
-AutoARIMA,Sundial-Base,0.75,0.5,1.0,0.201,0.09,0.294
-AutoARIMA,Naive,0.917,0.75,1.0,0.228,0.128,0.322
-AutoARIMA,Seasonal Naive,1.0,1.0,1.0,0.317,0.242,0.396
-Chronos-Bolt,TiRex,0.083,0.0,0.25,-0.103,-0.164,-0.052
-Chronos-Bolt,Chronos-2,0.0,0.0,0.0,-0.095,-0.165,-0.044
-Chronos-Bolt,Stat. Ensemble,0.083,0.0,0.25,-0.084,-0.151,-0.024
-Chronos-Bolt,Toto-1.0,0.333,0.125,0.583,-0.025,-0.088,0.03
-Chronos-Bolt,AutoETS,0.25,0.0,0.5,0.037,-0.094,0.201
-Chronos-Bolt,TimesFM-2.5,0.417,0.167,0.667,-0.014,-0.077,0.032
-Chronos-Bolt,AutoARIMA,0.583,0.333,0.833,0.001,-0.074,0.074
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,TabPFN-TS,0.5,0.25,0.75,0.006,-0.093,0.108
-Chronos-Bolt,Moirai-2.0,0.5,0.25,0.75,0.039,-0.001,0.09
-Chronos-Bolt,Drift,0.75,0.5,1.0,0.084,-0.028,0.199
-Chronos-Bolt,AutoTheta,0.667,0.417,0.917,0.031,-0.044,0.102
-Chronos-Bolt,Sundial-Base,0.917,0.75,1.0,0.202,0.109,0.273
-Chronos-Bolt,Naive,1.0,1.0,1.0,0.229,0.158,0.296
-Chronos-Bolt,Seasonal Naive,0.917,0.75,1.0,0.318,0.219,0.402
-TabPFN-TS,TiRex,0.167,0.0,0.417,-0.109,-0.2,-0.031
-TabPFN-TS,Chronos-2,0.167,0.0,0.417,-0.102,-0.192,-0.032
-TabPFN-TS,Stat. Ensemble,0.167,0.0,0.417,-0.091,-0.167,-0.023
-TabPFN-TS,Toto-1.0,0.333,0.083,0.583,-0.031,-0.171,0.093
-TabPFN-TS,AutoETS,0.333,0.083,0.583,0.031,-0.114,0.208
-TabPFN-TS,TimesFM-2.5,0.5,0.25,0.75,-0.02,-0.109,0.056
-TabPFN-TS,AutoARIMA,0.5,0.25,0.833,-0.005,-0.099,0.092
-TabPFN-TS,Chronos-Bolt,0.5,0.25,0.75,-0.006,-0.121,0.085
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Moirai-2.0,0.583,0.331,0.833,0.033,-0.089,0.147
-TabPFN-TS,Drift,0.5,0.25,0.833,0.078,-0.057,0.251
-TabPFN-TS,AutoTheta,0.5,0.25,0.833,0.025,-0.038,0.091
-TabPFN-TS,Sundial-Base,0.917,0.75,1.0,0.198,0.122,0.263
-TabPFN-TS,Naive,0.833,0.583,1.0,0.225,0.109,0.337
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.314,0.229,0.401
-Moirai-2.0,TiRex,0.083,0.0,0.25,-0.147,-0.245,-0.067
-Moirai-2.0,Chronos-2,0.0,0.0,0.0,-0.139,-0.237,-0.063
-Moirai-2.0,Stat. Ensemble,0.083,0.0,0.25,-0.128,-0.206,-0.05
-Moirai-2.0,Toto-1.0,0.167,0.042,0.375,-0.066,-0.123,-0.022
-Moirai-2.0,AutoETS,0.25,0.0,0.5,-0.002,-0.151,0.175
-Moirai-2.0,TimesFM-2.5,0.417,0.167,0.667,-0.055,-0.145,0.014
-Moirai-2.0,AutoARIMA,0.5,0.25,0.75,-0.039,-0.142,0.06
-Moirai-2.0,Chronos-Bolt,0.5,0.25,0.75,-0.04,-0.099,0.001
-Moirai-2.0,TabPFN-TS,0.417,0.167,0.669,-0.034,-0.173,0.082
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Drift,0.583,0.333,0.833,0.047,-0.076,0.17
-Moirai-2.0,AutoTheta,0.5,0.25,0.75,-0.008,-0.087,0.069
-Moirai-2.0,Sundial-Base,0.833,0.583,1.0,0.17,0.072,0.253
-Moirai-2.0,Naive,0.917,0.75,1.0,0.199,0.12,0.272
-Moirai-2.0,Seasonal Naive,0.833,0.583,1.0,0.291,0.176,0.391
-Drift,TiRex,0.083,0.0,0.25,-0.204,-0.414,-0.058
-Drift,Chronos-2,0.083,0.0,0.25,-0.195,-0.435,-0.037
-Drift,Stat. Ensemble,0.083,0.0,0.25,-0.183,-0.386,-0.065
-Drift,Toto-1.0,0.25,0.0,0.5,-0.119,-0.276,0.014
-Drift,AutoETS,0.25,0.0,0.5,-0.051,-0.31,0.166
-Drift,TimesFM-2.5,0.25,0.0,0.5,-0.107,-0.342,0.044
-Drift,AutoARIMA,0.417,0.167,0.667,-0.09,-0.289,0.047
-Drift,Chronos-Bolt,0.25,0.0,0.5,-0.091,-0.248,0.027
-Drift,TabPFN-TS,0.5,0.167,0.75,-0.085,-0.334,0.054
-Drift,Moirai-2.0,0.417,0.167,0.667,-0.049,-0.204,0.07
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0
-Drift,AutoTheta,0.75,0.5,0.917,-0.057,-0.232,0.028
-Drift,Sundial-Base,0.917,0.75,1.0,0.13,-0.097,0.26
-Drift,Naive,0.917,0.75,1.0,0.159,0.082,0.234
-Drift,Seasonal Naive,0.917,0.75,1.0,0.256,0.054,0.378
-AutoTheta,TiRex,0.083,0.0,0.25,-0.138,-0.227,-0.056
-AutoTheta,Chronos-2,0.083,0.0,0.25,-0.13,-0.217,-0.048
-AutoTheta,Stat. Ensemble,0.0,0.0,0.0,-0.119,-0.173,-0.075
-AutoTheta,Toto-1.0,0.333,0.083,0.583,-0.058,-0.156,0.027
-AutoTheta,AutoETS,0.25,0.0,0.5,0.006,-0.136,0.186
-AutoTheta,TimesFM-2.5,0.25,0.0,0.5,-0.047,-0.131,0.041
-AutoTheta,AutoARIMA,0.417,0.167,0.75,-0.031,-0.12,0.055
-AutoTheta,Chronos-Bolt,0.333,0.083,0.583,-0.032,-0.114,0.042
-AutoTheta,TabPFN-TS,0.5,0.167,0.75,-0.026,-0.101,0.037
-AutoTheta,Moirai-2.0,0.5,0.25,0.75,0.008,-0.075,0.08
-AutoTheta,Drift,0.25,0.083,0.5,0.054,-0.029,0.188
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Sundial-Base,0.917,0.75,1.0,0.177,0.099,0.254
-AutoTheta,Naive,1.0,1.0,1.0,0.205,0.128,0.291
-AutoTheta,Seasonal Naive,1.0,1.0,1.0,0.296,0.216,0.375
-Sundial-Base,TiRex,0.0,0.0,0.0,-0.383,-0.501,-0.264
-Sundial-Base,Chronos-2,0.0,0.0,0.0,-0.373,-0.48,-0.274
-Sundial-Base,Stat. Ensemble,0.0,0.0,0.0,-0.36,-0.49,-0.236
-Sundial-Base,Toto-1.0,0.167,0.0,0.417,-0.285,-0.472,-0.13
-Sundial-Base,AutoETS,0.083,0.0,0.25,-0.207,-0.401,-0.002
-Sundial-Base,TimesFM-2.5,0.0,0.0,0.0,-0.272,-0.374,-0.176
-Sundial-Base,AutoARIMA,0.25,0.0,0.5,-0.252,-0.417,-0.099
-Sundial-Base,Chronos-Bolt,0.083,0.0,0.25,-0.254,-0.376,-0.123
-Sundial-Base,TabPFN-TS,0.083,0.0,0.25,-0.246,-0.357,-0.139
-Sundial-Base,Moirai-2.0,0.167,0.0,0.417,-0.205,-0.339,-0.077
-Sundial-Base,Drift,0.083,0.0,0.25,-0.149,-0.351,0.088
-Sundial-Base,AutoTheta,0.083,0.0,0.25,-0.215,-0.341,-0.11
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Naive,0.5,0.25,0.75,0.034,-0.089,0.177
-Sundial-Base,Seasonal Naive,0.833,0.583,1.0,0.145,0.018,0.25
-Naive,TiRex,0.0,0.0,0.0,-0.431,-0.619,-0.27
-Naive,Chronos-2,0.0,0.0,0.0,-0.421,-0.633,-0.253
-Naive,Stat. Ensemble,0.0,0.0,0.0,-0.407,-0.6,-0.259
-Naive,Toto-1.0,0.083,0.0,0.25,-0.33,-0.503,-0.186
-Naive,AutoETS,0.083,0.0,0.25,-0.25,-0.508,-0.001
-Naive,TimesFM-2.5,0.083,0.0,0.25,-0.316,-0.515,-0.157
-Naive,AutoARIMA,0.083,0.0,0.25,-0.296,-0.475,-0.146
-Naive,Chronos-Bolt,0.0,0.0,0.0,-0.298,-0.42,-0.188
-Naive,TabPFN-TS,0.167,0.0,0.417,-0.29,-0.508,-0.122
-Naive,Moirai-2.0,0.083,0.0,0.25,-0.248,-0.374,-0.137
-Naive,Drift,0.083,0.0,0.25,-0.189,-0.305,-0.089
-Naive,AutoTheta,0.0,0.0,0.0,-0.257,-0.41,-0.146
-Naive,Sundial-Base,0.5,0.25,0.75,-0.035,-0.215,0.082
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Seasonal Naive,0.75,0.542,0.917,0.115,-0.055,0.229
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.617,-0.877,-0.392
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-0.606,-0.842,-0.401
-Seasonal Naive,Stat. Ensemble,0.0,0.0,0.0,-0.59,-0.824,-0.404
-Seasonal Naive,Toto-1.0,0.083,0.0,0.25,-0.503,-0.771,-0.275
-Seasonal Naive,AutoETS,0.083,0.0,0.25,-0.412,-0.755,-0.091
-Seasonal Naive,TimesFM-2.5,0.083,0.0,0.25,-0.487,-0.686,-0.288
-Seasonal Naive,AutoARIMA,0.0,0.0,0.0,-0.465,-0.656,-0.319
-Seasonal Naive,Chronos-Bolt,0.083,0.0,0.25,-0.467,-0.673,-0.28
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.458,-0.668,-0.296
-Seasonal Naive,Moirai-2.0,0.167,0.0,0.417,-0.41,-0.643,-0.214
-Seasonal Naive,Drift,0.083,0.0,0.25,-0.344,-0.607,-0.057
-Seasonal Naive,AutoTheta,0.0,0.0,0.0,-0.421,-0.599,-0.275
-Seasonal Naive,Sundial-Base,0.167,0.0,0.417,-0.17,-0.333,-0.019
-Seasonal Naive,Naive,0.25,0.083,0.458,-0.13,-0.297,0.052
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_econ/pairwise_WAPE.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Stat. Ensemble,0.583,0.333,0.833,-0.007,-0.118,0.073
-TiRex,Chronos-2,0.667,0.333,0.917,0.019,-0.026,0.055
-TiRex,Toto-1.0,0.5,0.25,0.833,0.057,-0.013,0.132
-TiRex,TimesFM-2.5,0.833,0.667,1.0,0.029,-0.01,0.066
-TiRex,AutoETS,0.667,0.417,0.917,0.045,-0.022,0.113
-TiRex,TabPFN-TS,0.75,0.5,1.0,0.053,-0.079,0.157
-TiRex,AutoARIMA,0.583,0.333,0.833,0.032,-0.075,0.116
-TiRex,Chronos-Bolt,0.917,0.75,1.0,0.105,0.043,0.176
-TiRex,Drift,0.75,0.5,1.0,0.13,-0.069,0.279
-TiRex,Moirai-2.0,0.917,0.75,1.0,0.132,0.04,0.235
-TiRex,AutoTheta,0.667,0.417,0.917,0.087,-0.092,0.23
-TiRex,Naive,0.917,0.75,1.0,0.26,0.135,0.394
-TiRex,Sundial-Base,0.917,0.75,1.0,0.243,0.154,0.322
-TiRex,Seasonal Naive,0.917,0.75,1.0,0.378,0.252,0.503
-Stat. Ensemble,TiRex,0.417,0.167,0.667,0.007,-0.079,0.105
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,Chronos-2,0.333,0.083,0.583,0.026,-0.052,0.124
-Stat. Ensemble,Toto-1.0,0.583,0.25,0.833,0.064,-0.026,0.153
-Stat. Ensemble,TimesFM-2.5,0.5,0.25,0.75,0.036,-0.044,0.138
-Stat. Ensemble,AutoETS,0.75,0.5,1.0,0.051,0.006,0.104
-Stat. Ensemble,TabPFN-TS,0.5,0.25,0.75,0.059,-0.02,0.135
-Stat. Ensemble,AutoARIMA,0.833,0.583,1.0,0.038,-0.015,0.077
-Stat. Ensemble,Chronos-Bolt,0.75,0.5,1.0,0.111,0.016,0.202
-Stat. Ensemble,Drift,0.833,0.583,1.0,0.136,0.017,0.263
-Stat. Ensemble,Moirai-2.0,0.667,0.333,0.917,0.138,0.035,0.235
-Stat. Ensemble,AutoTheta,0.917,0.75,1.0,0.093,0.006,0.186
-Stat. Ensemble,Naive,1.0,1.0,1.0,0.265,0.136,0.387
-Stat. Ensemble,Sundial-Base,1.0,1.0,1.0,0.248,0.146,0.339
-Stat. Ensemble,Seasonal Naive,1.0,1.0,1.0,0.382,0.284,0.478
-Chronos-2,TiRex,0.333,0.083,0.667,-0.019,-0.059,0.025
-Chronos-2,Stat. Ensemble,0.667,0.417,0.917,-0.026,-0.142,0.05
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,Toto-1.0,0.5,0.25,0.833,0.039,-0.05,0.132
-Chronos-2,TimesFM-2.5,0.5,0.167,0.75,0.01,-0.025,0.05
-Chronos-2,AutoETS,0.667,0.417,0.917,0.026,-0.04,0.085
-Chronos-2,TabPFN-TS,0.667,0.417,0.917,0.035,-0.112,0.148
-Chronos-2,AutoARIMA,0.583,0.333,0.833,0.013,-0.106,0.11
-Chronos-2,Chronos-Bolt,0.75,0.5,1.0,0.087,0.014,0.175
-Chronos-2,Drift,0.667,0.417,0.917,0.114,-0.101,0.291
-Chronos-2,Moirai-2.0,0.667,0.333,0.917,0.115,0.018,0.219
-Chronos-2,AutoTheta,0.667,0.417,0.917,0.07,-0.127,0.214
-Chronos-2,Naive,0.833,0.583,1.0,0.246,0.102,0.391
-Chronos-2,Sundial-Base,1.0,1.0,1.0,0.229,0.153,0.304
-Chronos-2,Seasonal Naive,0.917,0.75,1.0,0.366,0.237,0.489
-Toto-1.0,TiRex,0.5,0.167,0.75,-0.061,-0.152,0.013
-Toto-1.0,Stat. Ensemble,0.417,0.167,0.75,-0.068,-0.181,0.025
-Toto-1.0,Chronos-2,0.5,0.167,0.75,-0.041,-0.152,0.047
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,TimesFM-2.5,0.583,0.333,0.833,-0.03,-0.14,0.051
-Toto-1.0,AutoETS,0.5,0.25,0.75,-0.013,-0.1,0.063
-Toto-1.0,TabPFN-TS,0.583,0.333,0.833,-0.005,-0.164,0.134
-Toto-1.0,AutoARIMA,0.583,0.333,0.833,-0.027,-0.14,0.072
-Toto-1.0,Chronos-Bolt,0.75,0.542,0.958,0.05,0.006,0.092
-Toto-1.0,Drift,0.667,0.417,0.917,0.077,-0.097,0.197
-Toto-1.0,Moirai-2.0,0.833,0.625,1.0,0.079,0.026,0.142
-Toto-1.0,AutoTheta,0.667,0.417,0.917,0.032,-0.139,0.166
-Toto-1.0,Naive,1.0,1.0,1.0,0.215,0.104,0.331
-Toto-1.0,Sundial-Base,0.833,0.583,1.0,0.197,0.07,0.294
-Toto-1.0,Seasonal Naive,0.917,0.75,1.0,0.34,0.203,0.463
-TimesFM-2.5,TiRex,0.167,0.0,0.333,-0.03,-0.071,0.01
-TimesFM-2.5,Stat. Ensemble,0.5,0.25,0.75,-0.037,-0.161,0.042
-TimesFM-2.5,Chronos-2,0.5,0.25,0.833,-0.011,-0.052,0.025
-TimesFM-2.5,Toto-1.0,0.417,0.167,0.667,0.029,-0.054,0.123
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,AutoETS,0.583,0.331,0.833,0.016,-0.058,0.087
-TimesFM-2.5,TabPFN-TS,0.5,0.25,0.75,0.024,-0.121,0.142
-TimesFM-2.5,AutoARIMA,0.667,0.417,0.917,0.002,-0.108,0.082
-TimesFM-2.5,Chronos-Bolt,0.667,0.417,0.875,0.078,0.002,0.167
-TimesFM-2.5,Drift,0.75,0.5,1.0,0.104,-0.116,0.272
-TimesFM-2.5,Moirai-2.0,0.583,0.333,0.833,0.106,0.001,0.216
-TimesFM-2.5,AutoTheta,0.75,0.5,1.0,0.06,-0.138,0.209
-TimesFM-2.5,Naive,0.833,0.583,1.0,0.238,0.088,0.384
-TimesFM-2.5,Sundial-Base,1.0,1.0,1.0,0.221,0.142,0.296
-TimesFM-2.5,Seasonal Naive,0.917,0.75,1.0,0.359,0.235,0.479
-AutoETS,TiRex,0.333,0.083,0.583,-0.047,-0.127,0.022
-AutoETS,Stat. Ensemble,0.25,0.0,0.5,-0.054,-0.115,-0.006
-AutoETS,Chronos-2,0.333,0.083,0.583,-0.027,-0.093,0.039
-AutoETS,Toto-1.0,0.5,0.25,0.75,0.013,-0.067,0.091
-AutoETS,TimesFM-2.5,0.417,0.167,0.669,-0.016,-0.095,0.055
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,TabPFN-TS,0.417,0.167,0.75,0.008,-0.094,0.112
-AutoETS,AutoARIMA,0.583,0.333,0.833,-0.014,-0.108,0.063
-AutoETS,Chronos-Bolt,0.667,0.333,0.917,0.063,-0.003,0.135
-AutoETS,Drift,0.583,0.333,0.833,0.09,-0.071,0.231
-AutoETS,Moirai-2.0,0.667,0.333,0.917,0.092,0.012,0.167
-AutoETS,AutoTheta,0.667,0.417,0.917,0.044,-0.086,0.148
-AutoETS,Naive,0.917,0.75,1.0,0.225,0.099,0.346
-AutoETS,Sundial-Base,0.917,0.75,1.0,0.208,0.127,0.279
-AutoETS,Seasonal Naive,1.0,1.0,1.0,0.348,0.248,0.455
-TabPFN-TS,TiRex,0.25,0.0,0.5,-0.056,-0.187,0.073
-TabPFN-TS,Stat. Ensemble,0.5,0.25,0.75,-0.063,-0.156,0.019
-TabPFN-TS,Chronos-2,0.333,0.083,0.583,-0.036,-0.174,0.101
-TabPFN-TS,Toto-1.0,0.417,0.167,0.667,0.005,-0.155,0.141
-TabPFN-TS,TimesFM-2.5,0.5,0.25,0.75,-0.025,-0.165,0.108
-TabPFN-TS,AutoETS,0.583,0.25,0.833,-0.009,-0.126,0.086
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,AutoARIMA,0.5,0.25,0.75,-0.023,-0.135,0.081
-TabPFN-TS,Chronos-Bolt,0.583,0.331,0.833,0.055,-0.107,0.185
-TabPFN-TS,Drift,0.583,0.333,0.833,0.082,-0.067,0.234
-TabPFN-TS,Moirai-2.0,0.583,0.331,0.833,0.084,-0.089,0.226
-TabPFN-TS,AutoTheta,0.583,0.333,0.833,0.036,-0.065,0.138
-TabPFN-TS,Naive,0.833,0.583,1.0,0.219,0.073,0.351
-TabPFN-TS,Sundial-Base,0.917,0.75,1.0,0.201,0.088,0.308
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.343,0.242,0.437
-AutoARIMA,TiRex,0.417,0.167,0.667,-0.033,-0.131,0.07
-AutoARIMA,Stat. Ensemble,0.167,0.0,0.417,-0.04,-0.084,0.014
-AutoARIMA,Chronos-2,0.417,0.167,0.667,-0.013,-0.124,0.096
-AutoARIMA,Toto-1.0,0.417,0.167,0.667,0.027,-0.077,0.123
-AutoARIMA,TimesFM-2.5,0.333,0.083,0.583,-0.003,-0.089,0.098
-AutoARIMA,AutoETS,0.417,0.167,0.667,0.014,-0.067,0.097
-AutoARIMA,TabPFN-TS,0.5,0.25,0.75,0.022,-0.089,0.119
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Chronos-Bolt,0.417,0.167,0.667,0.076,-0.037,0.184
-AutoARIMA,Drift,0.5,0.25,0.75,0.102,-0.037,0.249
-AutoARIMA,Moirai-2.0,0.5,0.25,0.75,0.104,-0.025,0.23
-AutoARIMA,AutoTheta,0.417,0.167,0.667,0.057,-0.064,0.19
-AutoARIMA,Naive,0.75,0.5,0.917,0.236,0.084,0.383
-AutoARIMA,Sundial-Base,0.833,0.583,1.0,0.219,0.084,0.33
-AutoARIMA,Seasonal Naive,1.0,1.0,1.0,0.357,0.24,0.471
-Chronos-Bolt,TiRex,0.083,0.0,0.25,-0.117,-0.213,-0.045
-Chronos-Bolt,Stat. Ensemble,0.25,0.0,0.5,-0.125,-0.253,-0.016
-Chronos-Bolt,Chronos-2,0.25,0.0,0.5,-0.096,-0.213,-0.014
-Chronos-Bolt,Toto-1.0,0.25,0.042,0.458,-0.053,-0.101,-0.006
-Chronos-Bolt,TimesFM-2.5,0.333,0.125,0.583,-0.084,-0.2,-0.002
-Chronos-Bolt,AutoETS,0.333,0.083,0.667,-0.067,-0.156,0.003
-Chronos-Bolt,TabPFN-TS,0.417,0.167,0.669,-0.058,-0.227,0.097
-Chronos-Bolt,AutoARIMA,0.583,0.333,0.833,-0.082,-0.226,0.035
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,Drift,0.75,0.5,1.0,0.029,-0.15,0.149
-Chronos-Bolt,Moirai-2.0,0.417,0.167,0.667,0.031,-0.022,0.094
-Chronos-Bolt,AutoTheta,0.667,0.417,0.917,-0.02,-0.201,0.115
-Chronos-Bolt,Naive,0.833,0.583,1.0,0.174,0.068,0.284
-Chronos-Bolt,Sundial-Base,0.917,0.75,1.0,0.155,0.026,0.239
-Chronos-Bolt,Seasonal Naive,0.833,0.583,1.0,0.305,0.17,0.432
-Drift,TiRex,0.25,0.0,0.5,-0.15,-0.388,0.065
-Drift,Stat. Ensemble,0.167,0.0,0.417,-0.158,-0.356,-0.017
-Drift,Chronos-2,0.333,0.083,0.583,-0.128,-0.41,0.092
-Drift,Toto-1.0,0.333,0.083,0.583,-0.084,-0.246,0.088
-Drift,TimesFM-2.5,0.25,0.0,0.5,-0.116,-0.374,0.104
-Drift,AutoETS,0.417,0.167,0.667,-0.098,-0.3,0.066
-Drift,TabPFN-TS,0.417,0.167,0.667,-0.089,-0.305,0.063
-Drift,AutoARIMA,0.5,0.25,0.75,-0.114,-0.332,0.036
-Drift,Chronos-Bolt,0.25,0.0,0.5,-0.029,-0.175,0.13
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0
-Drift,Moirai-2.0,0.417,0.167,0.667,0.002,-0.146,0.163
-Drift,AutoTheta,0.75,0.5,1.0,-0.05,-0.184,0.016
-Drift,Naive,0.917,0.75,1.0,0.149,0.045,0.251
-Drift,Sundial-Base,0.833,0.583,1.0,0.13,-0.09,0.296
-Drift,Seasonal Naive,0.917,0.75,1.0,0.284,0.11,0.391
-Moirai-2.0,TiRex,0.083,0.0,0.25,-0.152,-0.307,-0.042
-Moirai-2.0,Stat. Ensemble,0.333,0.083,0.667,-0.16,-0.308,-0.037
-Moirai-2.0,Chronos-2,0.333,0.083,0.667,-0.131,-0.281,-0.018
-Moirai-2.0,Toto-1.0,0.167,0.0,0.375,-0.086,-0.166,-0.027
-Moirai-2.0,TimesFM-2.5,0.417,0.167,0.667,-0.119,-0.275,-0.001
-Moirai-2.0,AutoETS,0.333,0.083,0.667,-0.101,-0.201,-0.012
-Moirai-2.0,TabPFN-TS,0.417,0.167,0.669,-0.091,-0.291,0.082
-Moirai-2.0,AutoARIMA,0.5,0.25,0.75,-0.116,-0.299,0.024
-Moirai-2.0,Chronos-Bolt,0.583,0.333,0.833,-0.032,-0.104,0.022
-Moirai-2.0,Drift,0.583,0.333,0.833,-0.002,-0.195,0.127
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,AutoTheta,0.5,0.25,0.75,-0.052,-0.228,0.084
-Moirai-2.0,Naive,0.75,0.5,1.0,0.147,0.051,0.248
-Moirai-2.0,Sundial-Base,0.833,0.583,1.0,0.128,-0.003,0.227
-Moirai-2.0,Seasonal Naive,0.75,0.5,1.0,0.283,0.152,0.405
-AutoTheta,TiRex,0.333,0.083,0.583,-0.096,-0.299,0.084
-AutoTheta,Stat. Ensemble,0.083,0.0,0.25,-0.103,-0.229,-0.006
-AutoTheta,Chronos-2,0.333,0.083,0.583,-0.075,-0.273,0.113
-AutoTheta,Toto-1.0,0.333,0.083,0.583,-0.033,-0.199,0.122
-AutoTheta,TimesFM-2.5,0.25,0.0,0.5,-0.064,-0.263,0.122
-AutoTheta,AutoETS,0.333,0.083,0.583,-0.046,-0.174,0.079
-AutoTheta,TabPFN-TS,0.417,0.167,0.667,-0.038,-0.16,0.061
-AutoTheta,AutoARIMA,0.583,0.333,0.833,-0.061,-0.235,0.061
-AutoTheta,Chronos-Bolt,0.333,0.083,0.583,0.019,-0.13,0.167
-AutoTheta,Drift,0.25,0.0,0.5,0.047,-0.016,0.156
-AutoTheta,Moirai-2.0,0.5,0.25,0.75,0.049,-0.092,0.186
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Naive,0.833,0.583,1.0,0.189,0.084,0.29
-AutoTheta,Sundial-Base,0.833,0.583,1.0,0.171,0.057,0.303
-AutoTheta,Seasonal Naive,1.0,1.0,1.0,0.318,0.237,0.391
-Naive,TiRex,0.083,0.0,0.25,-0.352,-0.649,-0.156
-Naive,Stat. Ensemble,0.0,0.0,0.0,-0.361,-0.631,-0.157
-Naive,Chronos-2,0.167,0.0,0.417,-0.326,-0.642,-0.113
-Naive,Toto-1.0,0.0,0.0,0.0,-0.274,-0.494,-0.116
-Naive,TimesFM-2.5,0.167,0.0,0.417,-0.312,-0.623,-0.097
-Naive,AutoETS,0.083,0.0,0.25,-0.291,-0.53,-0.11
-Naive,TabPFN-TS,0.167,0.0,0.417,-0.28,-0.54,-0.079
-Naive,AutoARIMA,0.25,0.083,0.5,-0.309,-0.62,-0.092
-Naive,Chronos-Bolt,0.167,0.0,0.417,-0.21,-0.396,-0.073
-Naive,Drift,0.083,0.0,0.25,-0.176,-0.335,-0.047
-Naive,Moirai-2.0,0.25,0.0,0.5,-0.173,-0.33,-0.053
-Naive,AutoTheta,0.167,0.0,0.417,-0.234,-0.408,-0.092
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Sundial-Base,0.583,0.333,0.833,-0.023,-0.218,0.11
-Naive,Seasonal Naive,0.75,0.542,0.917,0.159,0.001,0.281
-Sundial-Base,TiRex,0.083,0.0,0.25,-0.322,-0.476,-0.182
-Sundial-Base,Stat. Ensemble,0.0,0.0,0.0,-0.331,-0.513,-0.171
-Sundial-Base,Chronos-2,0.0,0.0,0.0,-0.297,-0.436,-0.181
-Sundial-Base,Toto-1.0,0.167,0.0,0.417,-0.246,-0.416,-0.075
-Sundial-Base,TimesFM-2.5,0.0,0.0,0.0,-0.283,-0.421,-0.165
-Sundial-Base,AutoETS,0.083,0.0,0.25,-0.262,-0.386,-0.145
-Sundial-Base,TabPFN-TS,0.083,0.0,0.25,-0.252,-0.445,-0.097
-Sundial-Base,AutoARIMA,0.167,0.0,0.417,-0.28,-0.493,-0.091
-Sundial-Base,Chronos-Bolt,0.083,0.0,0.25,-0.183,-0.315,-0.026
-Sundial-Base,Drift,0.167,0.0,0.417,-0.149,-0.421,0.082
-Sundial-Base,Moirai-2.0,0.167,0.0,0.417,-0.147,-0.294,0.002
-Sundial-Base,AutoTheta,0.167,0.0,0.417,-0.206,-0.435,-0.061
-Sundial-Base,Naive,0.417,0.167,0.667,0.022,-0.123,0.179
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Seasonal Naive,0.833,0.583,1.0,0.177,0.057,0.284
-Seasonal Naive,TiRex,0.083,0.0,0.25,-0.607,-1.014,-0.337
-Seasonal Naive,Stat. Ensemble,0.0,0.0,0.0,-0.618,-0.915,-0.396
-Seasonal Naive,Chronos-2,0.083,0.0,0.25,-0.576,-0.957,-0.311
-Seasonal Naive,Toto-1.0,0.083,0.0,0.25,-0.514,-0.863,-0.255
-Seasonal Naive,TimesFM-2.5,0.083,0.0,0.25,-0.56,-0.92,-0.308
-Seasonal Naive,AutoETS,0.0,0.0,0.0,-0.535,-0.834,-0.33
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.522,-0.775,-0.319
-Seasonal Naive,AutoARIMA,0.0,0.0,0.0,-0.556,-0.891,-0.315
-Seasonal Naive,Chronos-Bolt,0.167,0.0,0.417,-0.438,-0.76,-0.205
-Seasonal Naive,Drift,0.083,0.0,0.25,-0.397,-0.643,-0.124
-Seasonal Naive,Moirai-2.0,0.25,0.0,0.5,-0.394,-0.681,-0.179
-Seasonal Naive,AutoTheta,0.0,0.0,0.0,-0.467,-0.643,-0.311
-Seasonal Naive,Naive,0.25,0.083,0.458,-0.189,-0.391,-0.001
-Seasonal Naive,Sundial-Base,0.167,0.0,0.417,-0.216,-0.396,-0.06
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_econ/pairwise_WQL.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Chronos-2,0.583,0.333,0.833,0.019,-0.029,0.058
-TiRex,Toto-1.0,0.667,0.333,0.917,0.065,0.009,0.127
-TiRex,TimesFM-2.5,0.833,0.667,1.0,0.047,0.005,0.084
-TiRex,TabPFN-TS,0.833,0.583,1.0,0.062,-0.038,0.149
-TiRex,Stat. Ensemble,0.75,0.5,1.0,0.066,-0.055,0.161
-TiRex,Chronos-Bolt,0.917,0.75,1.0,0.102,0.047,0.167
-TiRex,AutoETS,0.833,0.583,1.0,0.084,0.017,0.145
-TiRex,Moirai-2.0,0.833,0.583,1.0,0.138,0.051,0.236
-TiRex,AutoARIMA,0.75,0.5,1.0,0.117,-0.008,0.227
-TiRex,AutoTheta,0.833,0.583,1.0,0.183,0.017,0.322
-TiRex,Drift,0.917,0.75,1.0,0.24,0.039,0.386
-TiRex,Sundial-Base,0.917,0.75,1.0,0.298,0.205,0.38
-TiRex,Naive,1.0,1.0,1.0,0.347,0.221,0.471
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.421,0.287,0.546
-Chronos-2,TiRex,0.417,0.167,0.667,-0.02,-0.062,0.028
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,Toto-1.0,0.5,0.25,0.833,0.047,-0.03,0.132
-Chronos-2,TimesFM-2.5,0.667,0.333,0.917,0.028,-0.008,0.061
-Chronos-2,TabPFN-TS,0.667,0.417,0.917,0.043,-0.066,0.134
-Chronos-2,Stat. Ensemble,0.833,0.583,1.0,0.048,-0.082,0.141
-Chronos-2,Chronos-Bolt,0.667,0.417,0.917,0.084,0.014,0.167
-Chronos-2,AutoETS,0.917,0.75,1.0,0.066,0.004,0.112
-Chronos-2,Moirai-2.0,0.667,0.333,0.917,0.122,0.031,0.22
-Chronos-2,AutoARIMA,0.75,0.5,1.0,0.099,-0.035,0.22
-Chronos-2,AutoTheta,0.917,0.75,1.0,0.167,-0.006,0.299
-Chronos-2,Drift,0.917,0.75,1.0,0.225,0.01,0.391
-Chronos-2,Sundial-Base,1.0,1.0,1.0,0.284,0.209,0.358
-Chronos-2,Naive,0.917,0.75,1.0,0.334,0.191,0.471
-Chronos-2,Seasonal Naive,0.917,0.75,1.0,0.41,0.281,0.527
-Toto-1.0,TiRex,0.333,0.083,0.667,-0.069,-0.145,-0.009
-Toto-1.0,Chronos-2,0.5,0.167,0.75,-0.049,-0.152,0.029
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,TimesFM-2.5,0.583,0.333,0.833,-0.02,-0.115,0.049
-Toto-1.0,TabPFN-TS,0.583,0.333,0.833,-0.004,-0.135,0.114
-Toto-1.0,Stat. Ensemble,0.583,0.333,0.833,0.001,-0.133,0.109
-Toto-1.0,Chronos-Bolt,0.833,0.667,1.0,0.039,0.004,0.074
-Toto-1.0,AutoETS,0.667,0.417,0.917,0.021,-0.071,0.095
-Toto-1.0,Moirai-2.0,0.917,0.792,1.0,0.079,0.025,0.148
-Toto-1.0,AutoARIMA,0.583,0.333,0.833,0.055,-0.086,0.182
-Toto-1.0,AutoTheta,0.75,0.5,1.0,0.126,-0.046,0.264
-Toto-1.0,Drift,0.917,0.75,1.0,0.187,-0.005,0.316
-Toto-1.0,Sundial-Base,0.917,0.75,1.0,0.249,0.132,0.337
-Toto-1.0,Naive,0.917,0.75,1.0,0.301,0.189,0.415
-Toto-1.0,Seasonal Naive,0.833,0.583,1.0,0.381,0.238,0.505
-TimesFM-2.5,TiRex,0.167,0.0,0.333,-0.049,-0.092,-0.005
-TimesFM-2.5,Chronos-2,0.333,0.083,0.667,-0.029,-0.064,0.008
-TimesFM-2.5,Toto-1.0,0.417,0.167,0.667,0.019,-0.052,0.103
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TabPFN-TS,0.5,0.25,0.75,0.016,-0.106,0.118
-TimesFM-2.5,Stat. Ensemble,0.667,0.417,0.917,0.021,-0.12,0.113
-TimesFM-2.5,Chronos-Bolt,0.583,0.333,0.833,0.058,-0.015,0.143
-TimesFM-2.5,AutoETS,0.75,0.5,1.0,0.04,-0.045,0.104
-TimesFM-2.5,Moirai-2.0,0.583,0.333,0.833,0.096,-0.002,0.206
-TimesFM-2.5,AutoARIMA,0.75,0.5,1.0,0.074,-0.073,0.186
-TimesFM-2.5,AutoTheta,0.833,0.583,1.0,0.143,-0.049,0.286
-TimesFM-2.5,Drift,0.833,0.583,1.0,0.203,-0.028,0.375
-TimesFM-2.5,Sundial-Base,1.0,1.0,1.0,0.264,0.182,0.346
-TimesFM-2.5,Naive,0.917,0.75,1.0,0.315,0.161,0.458
-TimesFM-2.5,Seasonal Naive,0.917,0.75,1.0,0.393,0.257,0.517
-TabPFN-TS,TiRex,0.167,0.0,0.417,-0.066,-0.175,0.037
-TabPFN-TS,Chronos-2,0.333,0.083,0.583,-0.045,-0.154,0.062
-TabPFN-TS,Toto-1.0,0.417,0.167,0.667,0.004,-0.129,0.119
-TabPFN-TS,TimesFM-2.5,0.5,0.25,0.75,-0.016,-0.134,0.096
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Stat. Ensemble,0.417,0.167,0.667,0.005,-0.084,0.091
-TabPFN-TS,Chronos-Bolt,0.583,0.331,0.833,0.043,-0.082,0.159
-TabPFN-TS,AutoETS,0.75,0.5,1.0,0.024,-0.064,0.095
-TabPFN-TS,Moirai-2.0,0.583,0.331,0.833,0.082,-0.067,0.207
-TabPFN-TS,AutoARIMA,0.5,0.25,0.833,0.059,-0.057,0.177
-TabPFN-TS,AutoTheta,0.833,0.583,1.0,0.129,0.023,0.236
-TabPFN-TS,Drift,0.75,0.5,1.0,0.19,0.03,0.353
-TabPFN-TS,Sundial-Base,1.0,1.0,1.0,0.252,0.154,0.347
-TabPFN-TS,Naive,0.917,0.75,1.0,0.304,0.18,0.427
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.383,0.282,0.484
-Stat. Ensemble,TiRex,0.25,0.0,0.5,-0.071,-0.192,0.052
-Stat. Ensemble,Chronos-2,0.167,0.0,0.417,-0.05,-0.164,0.076
-Stat. Ensemble,Toto-1.0,0.417,0.167,0.667,-0.001,-0.123,0.117
-Stat. Ensemble,TimesFM-2.5,0.333,0.083,0.583,-0.021,-0.127,0.107
-Stat. Ensemble,TabPFN-TS,0.583,0.333,0.833,-0.005,-0.1,0.077
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,Chronos-Bolt,0.417,0.167,0.667,0.038,-0.083,0.155
-Stat. Ensemble,AutoETS,0.417,0.167,0.667,0.019,-0.068,0.095
-Stat. Ensemble,Moirai-2.0,0.5,0.25,0.75,0.077,-0.064,0.201
-Stat. Ensemble,AutoARIMA,0.917,0.75,1.0,0.054,-0.0,0.099
-Stat. Ensemble,AutoTheta,0.917,0.75,1.0,0.125,0.037,0.224
-Stat. Ensemble,Drift,0.917,0.75,1.0,0.186,0.041,0.338
-Stat. Ensemble,Sundial-Base,0.833,0.583,1.0,0.248,0.115,0.362
-Stat. Ensemble,Naive,1.0,1.0,1.0,0.3,0.159,0.433
-Stat. Ensemble,Seasonal Naive,1.0,1.0,1.0,0.38,0.283,0.482
-Chronos-Bolt,TiRex,0.083,0.0,0.25,-0.113,-0.2,-0.049
-Chronos-Bolt,Chronos-2,0.333,0.083,0.583,-0.092,-0.2,-0.014
-Chronos-Bolt,Toto-1.0,0.167,0.0,0.333,-0.041,-0.079,-0.004
-Chronos-Bolt,TimesFM-2.5,0.417,0.167,0.667,-0.061,-0.167,0.014
-Chronos-Bolt,TabPFN-TS,0.417,0.167,0.669,-0.045,-0.189,0.075
-Chronos-Bolt,Stat. Ensemble,0.583,0.333,0.833,-0.04,-0.184,0.077
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,AutoETS,0.583,0.333,0.833,-0.019,-0.125,0.057
-Chronos-Bolt,Moirai-2.0,0.5,0.25,0.75,0.041,-0.012,0.105
-Chronos-Bolt,AutoARIMA,0.667,0.417,0.917,0.017,-0.147,0.157
-Chronos-Bolt,AutoTheta,0.833,0.583,1.0,0.09,-0.083,0.223
-Chronos-Bolt,Drift,0.917,0.75,1.0,0.154,-0.033,0.28
-Chronos-Bolt,Sundial-Base,0.917,0.75,1.0,0.218,0.093,0.302
-Chronos-Bolt,Naive,0.917,0.75,1.0,0.273,0.167,0.374
-Chronos-Bolt,Seasonal Naive,0.833,0.583,1.0,0.355,0.212,0.48
-AutoETS,TiRex,0.167,0.0,0.417,-0.092,-0.169,-0.018
-AutoETS,Chronos-2,0.083,0.0,0.25,-0.071,-0.126,-0.004
-AutoETS,Toto-1.0,0.333,0.083,0.583,-0.021,-0.105,0.066
-AutoETS,TimesFM-2.5,0.25,0.0,0.5,-0.041,-0.116,0.043
-AutoETS,TabPFN-TS,0.25,0.0,0.5,-0.025,-0.104,0.06
-AutoETS,Stat. Ensemble,0.583,0.333,0.833,-0.02,-0.105,0.064
-AutoETS,Chronos-Bolt,0.417,0.167,0.667,0.019,-0.06,0.111
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,Moirai-2.0,0.417,0.167,0.667,0.059,-0.034,0.15
-AutoETS,AutoARIMA,0.75,0.5,1.0,0.035,-0.082,0.151
-AutoETS,AutoTheta,0.833,0.583,1.0,0.108,-0.026,0.227
-AutoETS,Drift,0.833,0.583,1.0,0.17,-0.016,0.334
-AutoETS,Sundial-Base,1.0,1.0,1.0,0.233,0.138,0.322
-AutoETS,Naive,0.917,0.75,1.0,0.287,0.148,0.417
-AutoETS,Seasonal Naive,0.917,0.75,1.0,0.368,0.243,0.486
-Moirai-2.0,TiRex,0.167,0.0,0.417,-0.161,-0.309,-0.053
-Moirai-2.0,Chronos-2,0.333,0.083,0.667,-0.138,-0.282,-0.032
-Moirai-2.0,Toto-1.0,0.083,0.0,0.208,-0.085,-0.173,-0.026
-Moirai-2.0,TimesFM-2.5,0.417,0.167,0.667,-0.107,-0.259,0.002
-Moirai-2.0,TabPFN-TS,0.417,0.167,0.669,-0.089,-0.262,0.063
-Moirai-2.0,Stat. Ensemble,0.5,0.25,0.75,-0.084,-0.251,0.061
-Moirai-2.0,Chronos-Bolt,0.5,0.25,0.75,-0.043,-0.118,0.012
-Moirai-2.0,AutoETS,0.583,0.333,0.833,-0.063,-0.177,0.033
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,AutoARIMA,0.583,0.333,0.833,-0.025,-0.226,0.141
-Moirai-2.0,AutoTheta,0.667,0.417,0.917,0.051,-0.119,0.187
-Moirai-2.0,Drift,0.75,0.5,1.0,0.118,-0.083,0.257
-Moirai-2.0,Sundial-Base,0.833,0.583,1.0,0.185,0.061,0.279
-Moirai-2.0,Naive,0.833,0.583,1.0,0.242,0.127,0.337
-Moirai-2.0,Seasonal Naive,0.75,0.5,1.0,0.328,0.183,0.456
-AutoARIMA,TiRex,0.25,0.0,0.5,-0.132,-0.294,0.008
-AutoARIMA,Chronos-2,0.25,0.0,0.5,-0.11,-0.282,0.034
-AutoARIMA,Toto-1.0,0.417,0.167,0.667,-0.059,-0.222,0.079
-AutoARIMA,TimesFM-2.5,0.25,0.0,0.5,-0.079,-0.228,0.068
-AutoARIMA,TabPFN-TS,0.5,0.167,0.75,-0.062,-0.215,0.054
-AutoARIMA,Stat. Ensemble,0.083,0.0,0.25,-0.057,-0.109,0.0
-AutoARIMA,Chronos-Bolt,0.333,0.083,0.583,-0.017,-0.187,0.128
-AutoARIMA,AutoETS,0.25,0.0,0.5,-0.037,-0.178,0.076
-AutoARIMA,Moirai-2.0,0.417,0.167,0.667,0.025,-0.164,0.185
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,AutoTheta,0.5,0.25,0.75,0.075,-0.043,0.214
-AutoARIMA,Drift,0.667,0.417,0.917,0.139,-0.024,0.304
-AutoARIMA,Sundial-Base,0.75,0.5,1.0,0.205,0.026,0.344
-AutoARIMA,Naive,0.833,0.583,1.0,0.26,0.092,0.423
-AutoARIMA,Seasonal Naive,1.0,1.0,1.0,0.344,0.225,0.467
-AutoTheta,TiRex,0.167,0.0,0.417,-0.224,-0.476,-0.017
-AutoTheta,Chronos-2,0.083,0.0,0.25,-0.2,-0.426,0.006
-AutoTheta,Toto-1.0,0.25,0.0,0.5,-0.144,-0.358,0.044
-AutoTheta,TimesFM-2.5,0.167,0.0,0.417,-0.167,-0.401,0.046
-AutoTheta,TabPFN-TS,0.167,0.0,0.417,-0.148,-0.309,-0.024
-AutoTheta,Stat. Ensemble,0.083,0.0,0.25,-0.143,-0.289,-0.038
-AutoTheta,Chronos-Bolt,0.167,0.0,0.417,-0.099,-0.286,0.076
-AutoTheta,AutoETS,0.167,0.0,0.417,-0.12,-0.293,0.025
-AutoTheta,Moirai-2.0,0.333,0.083,0.583,-0.054,-0.229,0.106
-AutoTheta,AutoARIMA,0.5,0.25,0.75,-0.081,-0.273,0.041
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Drift,0.333,0.083,0.583,0.07,-0.028,0.226
-AutoTheta,Sundial-Base,0.833,0.583,1.0,0.141,0.015,0.293
-AutoTheta,Naive,0.917,0.75,1.0,0.201,0.088,0.318
-AutoTheta,Seasonal Naive,1.0,1.0,1.0,0.291,0.222,0.357
-Drift,TiRex,0.083,0.0,0.25,-0.315,-0.628,-0.04
-Drift,Chronos-2,0.083,0.0,0.25,-0.29,-0.643,-0.01
-Drift,Toto-1.0,0.083,0.0,0.25,-0.23,-0.461,0.005
-Drift,TimesFM-2.5,0.167,0.0,0.417,-0.254,-0.6,0.027
-Drift,TabPFN-TS,0.25,0.0,0.5,-0.234,-0.546,-0.031
-Drift,Stat. Ensemble,0.083,0.0,0.25,-0.228,-0.511,-0.043
-Drift,Chronos-Bolt,0.083,0.0,0.25,-0.182,-0.389,0.032
-Drift,AutoETS,0.167,0.0,0.417,-0.205,-0.5,0.015
-Drift,Moirai-2.0,0.25,0.0,0.5,-0.133,-0.347,0.076
-Drift,AutoARIMA,0.333,0.083,0.583,-0.162,-0.437,0.023
-Drift,AutoTheta,0.667,0.417,0.917,-0.075,-0.292,0.027
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0
-Drift,Sundial-Base,0.667,0.417,0.917,0.076,-0.217,0.284
-Drift,Naive,0.917,0.75,1.0,0.141,0.034,0.242
-Drift,Seasonal Naive,0.917,0.75,1.0,0.238,0.002,0.366
-Sundial-Base,TiRex,0.083,0.0,0.25,-0.424,-0.613,-0.258
-Sundial-Base,Chronos-2,0.0,0.0,0.0,-0.397,-0.557,-0.264
-Sundial-Base,Toto-1.0,0.083,0.0,0.25,-0.332,-0.509,-0.153
-Sundial-Base,TimesFM-2.5,0.0,0.0,0.0,-0.358,-0.528,-0.222
-Sundial-Base,TabPFN-TS,0.0,0.0,0.0,-0.336,-0.532,-0.181
-Sundial-Base,Stat. Ensemble,0.167,0.0,0.417,-0.33,-0.567,-0.13
-Sundial-Base,Chronos-Bolt,0.083,0.0,0.25,-0.279,-0.433,-0.103
-Sundial-Base,AutoETS,0.0,0.0,0.0,-0.304,-0.474,-0.159
-Sundial-Base,Moirai-2.0,0.167,0.0,0.417,-0.227,-0.387,-0.066
-Sundial-Base,AutoARIMA,0.25,0.0,0.5,-0.258,-0.524,-0.027
-Sundial-Base,AutoTheta,0.167,0.0,0.417,-0.164,-0.415,-0.015
-Sundial-Base,Drift,0.333,0.083,0.583,-0.083,-0.397,0.178
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Naive,0.583,0.333,0.833,0.07,-0.096,0.23
-Sundial-Base,Seasonal Naive,0.833,0.583,1.0,0.175,0.01,0.298
-Naive,TiRex,0.0,0.0,0.0,-0.53,-0.889,-0.283
-Naive,Chronos-2,0.083,0.0,0.25,-0.501,-0.89,-0.236
-Naive,Toto-1.0,0.083,0.0,0.25,-0.431,-0.708,-0.233
-Naive,TimesFM-2.5,0.083,0.0,0.25,-0.459,-0.846,-0.192
-Naive,TabPFN-TS,0.083,0.0,0.25,-0.436,-0.746,-0.219
-Naive,Stat. Ensemble,0.0,0.0,0.0,-0.429,-0.765,-0.19
-Naive,Chronos-Bolt,0.083,0.0,0.25,-0.375,-0.598,-0.2
-Naive,AutoETS,0.083,0.0,0.25,-0.402,-0.715,-0.174
-Naive,Moirai-2.0,0.167,0.0,0.417,-0.319,-0.508,-0.145
-Naive,AutoARIMA,0.167,0.0,0.417,-0.352,-0.732,-0.102
-Naive,AutoTheta,0.083,0.0,0.25,-0.251,-0.466,-0.096
-Naive,Drift,0.083,0.0,0.25,-0.164,-0.32,-0.035
-Naive,Sundial-Base,0.417,0.167,0.667,-0.075,-0.298,0.087
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Seasonal Naive,0.75,0.542,0.917,0.114,-0.095,0.248
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.727,-1.204,-0.402
-Seasonal Naive,Chronos-2,0.083,0.0,0.25,-0.694,-1.116,-0.391
-Seasonal Naive,Toto-1.0,0.167,0.0,0.417,-0.615,-1.022,-0.312
-Seasonal Naive,TimesFM-2.5,0.083,0.0,0.25,-0.647,-1.071,-0.346
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.621,-0.938,-0.393
-Seasonal Naive,Stat. Ensemble,0.0,0.0,0.0,-0.613,-0.931,-0.394
-Seasonal Naive,Chronos-Bolt,0.167,0.0,0.417,-0.551,-0.924,-0.269
-Seasonal Naive,AutoETS,0.083,0.0,0.25,-0.581,-0.947,-0.321
-Seasonal Naive,Moirai-2.0,0.25,0.0,0.5,-0.488,-0.838,-0.223
-Seasonal Naive,AutoARIMA,0.0,0.0,0.0,-0.525,-0.875,-0.291
-Seasonal Naive,AutoTheta,0.0,0.0,0.0,-0.411,-0.556,-0.286
-Seasonal Naive,Drift,0.083,0.0,0.25,-0.313,-0.578,-0.002
-Seasonal Naive,Sundial-Base,0.167,0.0,0.417,-0.213,-0.424,-0.01
-Seasonal Naive,Naive,0.25,0.083,0.458,-0.128,-0.33,0.087
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_energy/leaderboard_MASE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,91.20879120879124,35.398352594765115,0.0,3.4626783322727275,0.0,0.0
-TiRex,75.41208791208793,25.111125985019022,0.0,1.3772318233333332,0.038461538461538464,0.0
-TimesFM-2.5,74.17582417582418,24.470577245081294,0.0,36.094269344715904,0.15384615384615385,0.0
-Chronos-Bolt,70.05494505494507,24.79493117234646,0.0,1.1812673024242424,0.0,0.0
-TabPFN-TS,67.03296703296704,28.931010322468566,0.0,213.1836282907102,0.0,0.0
-Moirai-2.0,65.65934065934066,23.276855755353097,0.0,2.6483478834375003,0.3076923076923077,0.0
-Sundial-Base,64.69780219780219,25.973734598235776,0.0,8.734264116875,0.038461538461538464,0.0
-Toto-1.0,64.56043956043956,22.175386726803193,0.0,64.5943206675,0.15384615384615385,0.0
-Stat. Ensemble,39.83516483516484,5.123802463450922,0.0,2087.2821065800895,0.0,15.384615384615385
-AutoARIMA,36.53846153846153,3.34881134283036,0.0,1914.2135565203125,0.0,15.384615384615385
-Seasonal Naive,30.35714285714285,0.0,0.0,1.0154424039285712,0.0,0.0
-AutoTheta,28.57142857142857,1.1815957493498508,0.0,6.5463739134469705,0.0,0.0
-AutoETS,17.857142857142858,-34.975498607867614,0.0,13.10401027465909,0.0,0.0
-Naive,17.44505494505494,-44.77392099300393,0.0,1.012792872857143,0.0,0.0
-Drift,6.593406593406594,-52.299123287549286,0.0,1.0261745340625001,0.0,0.0

tables/domain_energy/leaderboard_SQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,93.13186813186815,43.69603844728434,0.0,3.4626783322727275,0.0,0.0
-TiRex,82.28021978021978,34.44791487661235,0.0,1.3772318233333332,0.038461538461538464,0.0
-TimesFM-2.5,75.82417582417582,33.04019456805117,0.0,36.094269344715904,0.15384615384615385,0.0
-TabPFN-TS,73.62637362637363,38.076295973948326,0.0,213.1836282907102,0.0,0.0
-Chronos-Bolt,70.6043956043956,32.88168947733126,0.0,1.1812673024242424,0.0,0.0
-Moirai-2.0,68.13186813186815,31.522824785428526,0.0,2.6483478834375003,0.3076923076923077,0.0
-Toto-1.0,67.03296703296705,31.075422556050736,0.0,64.5943206675,0.15384615384615385,0.0
-Sundial-Base,55.35714285714286,28.52842382801387,0.0,8.734264116875,0.038461538461538464,0.0
-AutoARIMA,39.28571428571428,9.387893004882919,0.0,1914.2135565203125,0.0,15.384615384615385
-Stat. Ensemble,37.08791208791208,7.135433823240489,0.0,2087.2821065800895,0.0,15.384615384615385
-Seasonal Naive,25.96153846153846,0.0,0.0,1.0154424039285712,0.0,0.0
-AutoETS,24.725274725274723,-30.21022947475347,0.0,13.10401027465909,0.0,0.0
-AutoTheta,20.05494505494506,-10.643501735785609,0.0,6.5463739134469705,0.0,0.0
-Naive,13.873626373626374,-60.57154436477256,0.0,1.012792872857143,0.0,0.0
-Drift,3.0219780219780215,-67.3016516015588,0.0,1.0261745340625001,0.0,0.0

tables/domain_energy/leaderboard_WAPE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,88.73626373626375,37.55790132014533,0.0,3.4626783322727275,0.0,0.0
-TimesFM-2.5,69.78021978021978,26.657949622489184,0.0,36.094269344715904,0.15384615384615385,0.0
-TiRex,69.36813186813188,27.32266216368823,0.0,1.3772318233333332,0.038461538461538464,0.0
-Chronos-Bolt,65.93406593406594,26.60685234340968,0.0,1.1812673024242424,0.0,0.0
-TabPFN-TS,65.38461538461539,30.329432122077236,0.0,213.1836282907102,0.0,0.0
-Moirai-2.0,64.01098901098902,25.999880752887982,0.0,2.6483478834375003,0.3076923076923077,0.0
-Sundial-Base,60.85164835164834,28.26554433861671,0.0,8.734264116875,0.038461538461538464,0.0
-Toto-1.0,58.79120879120878,24.17115830675096,0.0,64.5943206675,0.15384615384615385,0.0
-Stat. Ensemble,44.23076923076924,6.664281180112875,0.0,2087.2821065800895,0.0,15.384615384615385
-AutoARIMA,33.51648351648351,2.7340402546184417,0.0,1914.2135565203125,0.0,15.384615384615385
-AutoTheta,29.670329670329664,4.200265512517798,0.0,6.5463739134469705,0.0,0.0
-Naive,27.884615384615387,-29.595393684781012,0.0,1.012792872857143,0.0,0.0
-Seasonal Naive,27.609890109890113,0.0,0.0,1.0154424039285712,0.0,0.0
-AutoETS,27.472527472527474,-24.061974944434183,0.0,13.10401027465909,0.0,0.0
-Drift,16.75824175824176,-36.91555350097882,0.0,1.0261745340625001,0.0,0.0

tables/domain_energy/leaderboard_WQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,93.6813186813187,45.99295406512989,0.0,3.4626783322727275,0.0,0.0
-TiRex,80.35714285714286,37.01592382634481,0.0,1.3772318233333332,0.038461538461538464,0.0
-TabPFN-TS,75.54945054945054,39.82364919499898,0.0,213.1836282907102,0.0,0.0
-TimesFM-2.5,75.27472527472527,35.74711031825937,0.0,36.094269344715904,0.15384615384615385,0.0
-Chronos-Bolt,69.78021978021978,35.346534617319236,0.0,1.1812673024242424,0.0,0.0
-Moirai-2.0,67.85714285714288,34.54029719816187,0.0,2.6483478834375003,0.3076923076923077,0.0
-Toto-1.0,65.10989010989012,33.6346956535905,0.0,64.5943206675,0.15384615384615385,0.0
-Sundial-Base,54.807692307692314,31.42058530075822,0.0,8.734264116875,0.038461538461538464,0.0
-Stat. Ensemble,40.10989010989011,9.81188536974501,0.0,2087.2821065800895,0.0,15.384615384615385
-AutoARIMA,38.18681318681318,9.717573374612886,0.0,1914.2135565203125,0.0,15.384615384615385
-AutoETS,23.626373626373624,-24.269495265740982,0.0,13.10401027465909,0.0,0.0
-Seasonal Naive,23.214285714285715,0.0,0.0,1.0154424039285712,0.0,0.0
-AutoTheta,21.7032967032967,-4.896832263246642,0.0,6.5463739134469705,0.0,0.0
-Naive,14.972527472527469,-50.97673020090387,0.0,1.012792872857143,0.0,0.0
-Drift,5.769230769230769,-57.44553174606462,0.0,1.0261745340625001,0.0,0.0

tables/domain_energy/pairwise_MASE.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,TiRex,0.769,0.615,0.923,0.137,0.066,0.203
-Chronos-2,TimesFM-2.5,0.769,0.577,0.923,0.145,0.072,0.212
-Chronos-2,Chronos-Bolt,0.962,0.885,1.0,0.141,0.078,0.202
-Chronos-2,TabPFN-TS,0.808,0.654,0.962,0.091,0.047,0.134
-Chronos-2,Moirai-2.0,0.923,0.808,1.0,0.158,0.092,0.222
-Chronos-2,Sundial-Base,0.846,0.692,0.962,0.127,0.069,0.195
-Chronos-2,Toto-1.0,0.846,0.692,0.962,0.17,0.105,0.237
-Chronos-2,Stat. Ensemble,0.962,0.885,1.0,0.319,0.239,0.385
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.332,0.249,0.404
-Chronos-2,Seasonal Naive,0.962,0.885,1.0,0.354,0.278,0.422
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.346,0.278,0.408
-Chronos-2,AutoETS,0.962,0.885,1.0,0.521,0.418,0.612
-Chronos-2,Naive,0.962,0.885,1.0,0.554,0.468,0.626
-Chronos-2,Drift,1.0,1.0,1.0,0.576,0.494,0.643
-TiRex,Chronos-2,0.231,0.077,0.385,-0.159,-0.255,-0.07
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,TimesFM-2.5,0.519,0.327,0.731,0.008,-0.015,0.033
-TiRex,Chronos-Bolt,0.558,0.385,0.731,0.004,-0.021,0.032
-TiRex,TabPFN-TS,0.538,0.346,0.731,-0.054,-0.16,0.039
-TiRex,Moirai-2.0,0.558,0.365,0.731,0.024,-0.003,0.056
-TiRex,Sundial-Base,0.75,0.596,0.904,-0.012,-0.103,0.066
-TiRex,Toto-1.0,0.673,0.5,0.846,0.038,0.01,0.068
-TiRex,Stat. Ensemble,0.962,0.885,1.0,0.211,0.152,0.27
-TiRex,AutoARIMA,0.923,0.808,1.0,0.225,0.159,0.292
-TiRex,Seasonal Naive,0.923,0.808,1.0,0.251,0.191,0.308
-TiRex,AutoTheta,1.0,1.0,1.0,0.242,0.194,0.299
-TiRex,AutoETS,0.962,0.885,1.0,0.445,0.339,0.543
-TiRex,Naive,0.962,0.885,1.0,0.483,0.403,0.556
-TiRex,Drift,1.0,1.0,1.0,0.508,0.431,0.578
-TimesFM-2.5,Chronos-2,0.231,0.077,0.423,-0.169,-0.269,-0.077
-TimesFM-2.5,TiRex,0.481,0.269,0.673,-0.009,-0.034,0.014
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,Chronos-Bolt,0.577,0.404,0.769,-0.004,-0.033,0.024
-TimesFM-2.5,TabPFN-TS,0.538,0.346,0.731,-0.063,-0.171,0.028
-TimesFM-2.5,Moirai-2.0,0.615,0.442,0.808,0.016,-0.028,0.057
-TimesFM-2.5,Sundial-Base,0.596,0.423,0.788,-0.02,-0.113,0.062
-TimesFM-2.5,Toto-1.0,0.692,0.519,0.846,0.029,-0.006,0.071
-TimesFM-2.5,Stat. Ensemble,0.885,0.731,1.0,0.204,0.139,0.271
-TimesFM-2.5,AutoARIMA,0.923,0.808,1.0,0.219,0.146,0.291
-TimesFM-2.5,Seasonal Naive,0.923,0.808,1.0,0.245,0.182,0.306
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.236,0.18,0.299
-TimesFM-2.5,AutoETS,0.962,0.885,1.0,0.44,0.337,0.539
-TimesFM-2.5,Naive,0.962,0.885,1.0,0.478,0.392,0.555
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.504,0.424,0.577
-Chronos-Bolt,Chronos-2,0.038,0.0,0.115,-0.164,-0.254,-0.085
-Chronos-Bolt,TiRex,0.442,0.269,0.615,-0.004,-0.034,0.02
-Chronos-Bolt,TimesFM-2.5,0.423,0.231,0.596,0.004,-0.025,0.032
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,TabPFN-TS,0.5,0.308,0.692,-0.058,-0.158,0.022
-Chronos-Bolt,Moirai-2.0,0.577,0.423,0.731,0.02,-0.022,0.064
-Chronos-Bolt,Sundial-Base,0.558,0.365,0.731,-0.016,-0.106,0.062
-Chronos-Bolt,Toto-1.0,0.577,0.404,0.75,0.034,-0.001,0.072
-Chronos-Bolt,Stat. Ensemble,0.885,0.731,1.0,0.207,0.148,0.271
-Chronos-Bolt,AutoARIMA,1.0,1.0,1.0,0.222,0.154,0.29
-Chronos-Bolt,Seasonal Naive,0.923,0.808,1.0,0.248,0.188,0.306
-Chronos-Bolt,AutoTheta,1.0,1.0,1.0,0.239,0.19,0.297
-Chronos-Bolt,AutoETS,0.923,0.808,1.0,0.443,0.333,0.545
-Chronos-Bolt,Naive,0.962,0.885,1.0,0.481,0.396,0.556
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.506,0.425,0.578
-TabPFN-TS,Chronos-2,0.192,0.038,0.346,-0.1,-0.155,-0.049
-TabPFN-TS,TiRex,0.462,0.269,0.654,0.051,-0.04,0.138
-TabPFN-TS,TimesFM-2.5,0.462,0.269,0.654,0.059,-0.029,0.146
-TabPFN-TS,Chronos-Bolt,0.5,0.308,0.692,0.055,-0.023,0.137
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Moirai-2.0,0.538,0.346,0.731,0.074,-0.012,0.16
-TabPFN-TS,Sundial-Base,0.462,0.269,0.654,0.04,-0.055,0.135
-TabPFN-TS,Toto-1.0,0.538,0.346,0.731,0.087,0.004,0.175
-TabPFN-TS,Stat. Ensemble,0.846,0.692,0.962,0.251,0.16,0.326
-TabPFN-TS,AutoARIMA,0.846,0.692,0.962,0.265,0.165,0.351
-TabPFN-TS,Seasonal Naive,0.885,0.731,1.0,0.289,0.198,0.368
-TabPFN-TS,AutoTheta,0.923,0.808,1.0,0.281,0.201,0.352
-TabPFN-TS,AutoETS,0.885,0.731,1.0,0.473,0.356,0.578
-TabPFN-TS,Naive,0.923,0.808,1.0,0.509,0.418,0.588
-TabPFN-TS,Drift,0.923,0.808,1.0,0.533,0.444,0.61
-Moirai-2.0,Chronos-2,0.077,0.0,0.192,-0.188,-0.285,-0.101
-Moirai-2.0,TiRex,0.442,0.269,0.635,-0.024,-0.059,0.003
-Moirai-2.0,TimesFM-2.5,0.385,0.192,0.558,-0.016,-0.061,0.027
-Moirai-2.0,Chronos-Bolt,0.423,0.269,0.577,-0.02,-0.068,0.021
-Moirai-2.0,TabPFN-TS,0.462,0.269,0.654,-0.08,-0.19,0.012
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Sundial-Base,0.635,0.462,0.789,-0.036,-0.136,0.048
-Moirai-2.0,Toto-1.0,0.615,0.462,0.788,0.014,-0.011,0.043
-Moirai-2.0,Stat. Ensemble,0.808,0.654,0.923,0.191,0.118,0.256
-Moirai-2.0,AutoARIMA,0.769,0.577,0.923,0.206,0.128,0.282
-Moirai-2.0,Seasonal Naive,0.808,0.654,0.924,0.233,0.167,0.293
-Moirai-2.0,AutoTheta,0.962,0.885,1.0,0.224,0.168,0.284
-Moirai-2.0,AutoETS,0.885,0.769,1.0,0.432,0.314,0.538
-Moirai-2.0,Naive,0.923,0.808,1.0,0.47,0.388,0.545
-Moirai-2.0,Drift,1.0,1.0,1.0,0.496,0.418,0.568
-Sundial-Base,Chronos-2,0.154,0.038,0.308,-0.146,-0.242,-0.074
-Sundial-Base,TiRex,0.25,0.096,0.404,0.012,-0.071,0.093
-Sundial-Base,TimesFM-2.5,0.404,0.212,0.577,0.02,-0.066,0.101
-Sundial-Base,Chronos-Bolt,0.442,0.269,0.635,0.016,-0.066,0.096
-Sundial-Base,TabPFN-TS,0.538,0.346,0.731,-0.042,-0.156,0.052
-Sundial-Base,Moirai-2.0,0.365,0.211,0.538,0.035,-0.051,0.119
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Toto-1.0,0.481,0.308,0.654,0.049,-0.032,0.129
-Sundial-Base,Stat. Ensemble,0.846,0.692,0.962,0.22,0.136,0.296
-Sundial-Base,AutoARIMA,0.846,0.692,0.962,0.234,0.147,0.311
-Sundial-Base,Seasonal Naive,0.885,0.731,1.0,0.26,0.177,0.334
-Sundial-Base,AutoTheta,0.962,0.885,1.0,0.251,0.176,0.319
-Sundial-Base,AutoETS,0.923,0.808,1.0,0.452,0.336,0.538
-Sundial-Base,Naive,0.962,0.885,1.0,0.489,0.393,0.575
-Sundial-Base,Drift,1.0,1.0,1.0,0.514,0.421,0.597
-Toto-1.0,Chronos-2,0.154,0.038,0.308,-0.205,-0.311,-0.117
-Toto-1.0,TiRex,0.327,0.154,0.5,-0.039,-0.073,-0.01
-Toto-1.0,TimesFM-2.5,0.308,0.154,0.481,-0.03,-0.077,0.006
-Toto-1.0,Chronos-Bolt,0.423,0.25,0.596,-0.035,-0.078,0.001
-Toto-1.0,TabPFN-TS,0.462,0.269,0.654,-0.095,-0.212,-0.004
-Toto-1.0,Moirai-2.0,0.385,0.212,0.538,-0.014,-0.045,0.011
-Toto-1.0,Sundial-Base,0.519,0.346,0.692,-0.051,-0.148,0.031
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,Stat. Ensemble,0.885,0.769,1.0,0.18,0.109,0.243
-Toto-1.0,AutoARIMA,0.885,0.769,1.0,0.195,0.122,0.268
-Toto-1.0,Seasonal Naive,0.885,0.731,1.0,0.222,0.159,0.279
-Toto-1.0,AutoTheta,0.923,0.808,1.0,0.212,0.153,0.276
-Toto-1.0,AutoETS,0.923,0.808,1.0,0.423,0.311,0.528
-Toto-1.0,Naive,0.962,0.885,1.0,0.462,0.385,0.537
-Toto-1.0,Drift,1.0,1.0,1.0,0.489,0.414,0.559
-Stat. Ensemble,Chronos-2,0.038,0.0,0.115,-0.469,-0.625,-0.314
-Stat. Ensemble,TiRex,0.038,0.0,0.115,-0.267,-0.369,-0.179
-Stat. Ensemble,TimesFM-2.5,0.115,0.0,0.269,-0.256,-0.371,-0.162
-Stat. Ensemble,Chronos-Bolt,0.115,0.0,0.269,-0.262,-0.372,-0.173
-Stat. Ensemble,TabPFN-TS,0.154,0.038,0.308,-0.335,-0.483,-0.19
-Stat. Ensemble,Moirai-2.0,0.192,0.077,0.346,-0.237,-0.345,-0.134
-Stat. Ensemble,Sundial-Base,0.154,0.038,0.308,-0.282,-0.419,-0.158
-Stat. Ensemble,Toto-1.0,0.115,0.0,0.231,-0.219,-0.322,-0.123
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoARIMA,0.615,0.442,0.788,0.018,-0.016,0.053
-Stat. Ensemble,Seasonal Naive,0.692,0.538,0.846,0.051,0.016,0.084
-Stat. Ensemble,AutoTheta,0.654,0.462,0.846,0.04,0.001,0.079
-Stat. Ensemble,AutoETS,0.846,0.692,0.962,0.297,0.19,0.407
-Stat. Ensemble,Naive,0.885,0.769,1.0,0.345,0.25,0.433
-Stat. Ensemble,Drift,0.962,0.885,1.0,0.377,0.289,0.468
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-0.496,-0.679,-0.332
-AutoARIMA,TiRex,0.077,0.0,0.192,-0.291,-0.413,-0.189
-AutoARIMA,TimesFM-2.5,0.077,0.0,0.192,-0.28,-0.41,-0.171
-AutoARIMA,Chronos-Bolt,0.0,0.0,0.0,-0.285,-0.409,-0.182
-AutoARIMA,TabPFN-TS,0.154,0.038,0.308,-0.36,-0.541,-0.198
-AutoARIMA,Moirai-2.0,0.231,0.077,0.423,-0.26,-0.393,-0.147
-AutoARIMA,Sundial-Base,0.154,0.038,0.308,-0.306,-0.451,-0.172
-AutoARIMA,Toto-1.0,0.115,0.0,0.231,-0.242,-0.366,-0.139
-AutoARIMA,Stat. Ensemble,0.385,0.212,0.558,-0.019,-0.056,0.015
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Seasonal Naive,0.654,0.481,0.808,0.033,0.006,0.059
-AutoARIMA,AutoTheta,0.654,0.462,0.808,0.022,-0.033,0.071
-AutoARIMA,AutoETS,0.769,0.577,0.923,0.284,0.172,0.395
-AutoARIMA,Naive,0.885,0.769,1.0,0.332,0.237,0.426
-AutoARIMA,Drift,0.962,0.885,1.0,0.365,0.272,0.453
-Seasonal Naive,Chronos-2,0.038,0.0,0.115,-0.548,-0.729,-0.384
-Seasonal Naive,TiRex,0.077,0.0,0.192,-0.335,-0.445,-0.237
-Seasonal Naive,TimesFM-2.5,0.077,0.0,0.192,-0.324,-0.441,-0.223
-Seasonal Naive,Chronos-Bolt,0.077,0.0,0.192,-0.33,-0.44,-0.232
-Seasonal Naive,TabPFN-TS,0.115,0.0,0.269,-0.407,-0.582,-0.246
-Seasonal Naive,Moirai-2.0,0.192,0.076,0.346,-0.303,-0.415,-0.201
-Seasonal Naive,Sundial-Base,0.115,0.0,0.269,-0.351,-0.501,-0.215
-Seasonal Naive,Toto-1.0,0.115,0.0,0.269,-0.285,-0.386,-0.189
-Seasonal Naive,Stat. Ensemble,0.308,0.154,0.462,-0.054,-0.092,-0.016
-Seasonal Naive,AutoARIMA,0.346,0.192,0.519,-0.035,-0.063,-0.006
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,AutoTheta,0.423,0.231,0.615,-0.012,-0.067,0.043
-Seasonal Naive,AutoETS,0.731,0.538,0.885,0.259,0.139,0.381
-Seasonal Naive,Naive,0.75,0.596,0.904,0.309,0.211,0.405
-Seasonal Naive,Drift,0.885,0.731,1.0,0.343,0.25,0.435
-AutoTheta,Chronos-2,0.0,0.0,0.0,-0.53,-0.689,-0.384
-AutoTheta,TiRex,0.0,0.0,0.0,-0.32,-0.427,-0.24
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-0.308,-0.427,-0.22
-AutoTheta,Chronos-Bolt,0.0,0.0,0.0,-0.314,-0.423,-0.234
-AutoTheta,TabPFN-TS,0.077,0.0,0.192,-0.39,-0.542,-0.251
-AutoTheta,Moirai-2.0,0.038,0.0,0.115,-0.288,-0.396,-0.202
-AutoTheta,Sundial-Base,0.038,0.0,0.115,-0.335,-0.468,-0.214
-AutoTheta,Toto-1.0,0.077,0.0,0.192,-0.27,-0.38,-0.18
-AutoTheta,Stat. Ensemble,0.346,0.154,0.538,-0.042,-0.085,-0.001
-AutoTheta,AutoARIMA,0.346,0.192,0.538,-0.022,-0.077,0.032
-AutoTheta,Seasonal Naive,0.577,0.385,0.769,0.012,-0.045,0.062
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,AutoETS,0.769,0.577,0.923,0.268,0.146,0.396
-AutoTheta,Naive,0.808,0.654,0.962,0.317,0.221,0.409
-AutoTheta,Drift,0.923,0.808,1.0,0.351,0.259,0.44
-AutoETS,Chronos-2,0.038,0.0,0.115,-1.089,-1.579,-0.717
-AutoETS,TiRex,0.038,0.0,0.115,-0.802,-1.187,-0.513
-AutoETS,TimesFM-2.5,0.038,0.0,0.115,-0.787,-1.17,-0.507
-AutoETS,Chronos-Bolt,0.077,0.0,0.192,-0.795,-1.196,-0.5
-AutoETS,TabPFN-TS,0.115,0.0,0.269,-0.899,-1.372,-0.554
-AutoETS,Moirai-2.0,0.115,0.0,0.231,-0.759,-1.164,-0.458
-AutoETS,Sundial-Base,0.077,0.0,0.192,-0.823,-1.166,-0.506
-AutoETS,Toto-1.0,0.077,0.0,0.192,-0.734,-1.118,-0.451
-AutoETS,Stat. Ensemble,0.154,0.038,0.308,-0.423,-0.687,-0.234
-AutoETS,AutoARIMA,0.231,0.077,0.423,-0.397,-0.654,-0.208
-AutoETS,Seasonal Naive,0.269,0.115,0.462,-0.35,-0.615,-0.161
-AutoETS,AutoTheta,0.231,0.077,0.423,-0.366,-0.655,-0.17
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,Naive,0.462,0.269,0.654,0.068,-0.078,0.212
-AutoETS,Drift,0.577,0.385,0.732,0.114,-0.03,0.253
-Naive,Chronos-2,0.038,0.0,0.115,-1.241,-1.671,-0.879
-Naive,TiRex,0.038,0.0,0.115,-0.933,-1.254,-0.676
-Naive,TimesFM-2.5,0.038,0.0,0.115,-0.917,-1.248,-0.645
-Naive,Chronos-Bolt,0.038,0.0,0.115,-0.925,-1.252,-0.656
-Naive,TabPFN-TS,0.077,0.0,0.192,-1.037,-1.429,-0.718
-Naive,Moirai-2.0,0.077,0.0,0.192,-0.887,-1.2,-0.635
-Naive,Sundial-Base,0.038,0.0,0.115,-0.956,-1.353,-0.646
-Naive,Toto-1.0,0.038,0.0,0.115,-0.86,-1.159,-0.625
-Naive,Stat. Ensemble,0.115,0.0,0.231,-0.526,-0.765,-0.333
-Naive,AutoARIMA,0.115,0.0,0.231,-0.498,-0.742,-0.31
-Naive,Seasonal Naive,0.25,0.096,0.404,-0.448,-0.68,-0.267
-Naive,AutoTheta,0.192,0.038,0.346,-0.465,-0.692,-0.283
-Naive,AutoETS,0.538,0.346,0.731,-0.073,-0.269,0.072
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Drift,0.846,0.692,0.962,0.049,0.03,0.073
-Drift,Chronos-2,0.0,0.0,0.0,-1.358,-1.804,-0.976
-Drift,TiRex,0.0,0.0,0.0,-1.034,-1.369,-0.759
-Drift,TimesFM-2.5,0.0,0.0,0.0,-1.016,-1.363,-0.736
-Drift,Chronos-Bolt,0.0,0.0,0.0,-1.025,-1.372,-0.739
-Drift,TabPFN-TS,0.077,0.0,0.192,-1.143,-1.562,-0.797
-Drift,Moirai-2.0,0.0,0.0,0.0,-0.985,-1.317,-0.718
-Drift,Sundial-Base,0.0,0.0,0.0,-1.057,-1.483,-0.728
-Drift,Toto-1.0,0.0,0.0,0.0,-0.957,-1.265,-0.707
-Drift,Stat. Ensemble,0.038,0.0,0.115,-0.605,-0.879,-0.406
-Drift,AutoARIMA,0.038,0.0,0.115,-0.576,-0.828,-0.373
-Drift,Seasonal Naive,0.115,0.0,0.269,-0.523,-0.771,-0.333
-Drift,AutoTheta,0.077,0.0,0.192,-0.541,-0.784,-0.35
-Drift,AutoETS,0.423,0.268,0.615,-0.128,-0.339,0.029
-Drift,Naive,0.154,0.038,0.308,-0.052,-0.078,-0.031
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_energy/pairwise_SQL.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,TiRex,0.731,0.538,0.885,0.141,0.072,0.204
-Chronos-2,TimesFM-2.5,0.769,0.577,0.923,0.159,0.083,0.225
-Chronos-2,TabPFN-TS,0.846,0.692,0.962,0.091,0.05,0.131
-Chronos-2,Chronos-Bolt,0.923,0.808,1.0,0.161,0.096,0.219
-Chronos-2,Moirai-2.0,0.923,0.808,1.0,0.178,0.111,0.242
-Chronos-2,Toto-1.0,0.885,0.731,1.0,0.183,0.115,0.25
-Chronos-2,Sundial-Base,0.962,0.885,1.0,0.212,0.157,0.275
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.379,0.297,0.45
-Chronos-2,Stat. Ensemble,1.0,1.0,1.0,0.394,0.317,0.461
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.437,0.368,0.497
-Chronos-2,AutoETS,1.0,1.0,1.0,0.568,0.468,0.65
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.491,0.407,0.574
-Chronos-2,Naive,1.0,1.0,1.0,0.649,0.573,0.716
-Chronos-2,Drift,1.0,1.0,1.0,0.663,0.591,0.728
-TiRex,Chronos-2,0.269,0.115,0.462,-0.164,-0.257,-0.078
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,TimesFM-2.5,0.673,0.481,0.846,0.021,-0.002,0.045
-TiRex,TabPFN-TS,0.577,0.385,0.769,-0.059,-0.164,0.029
-TiRex,Chronos-Bolt,0.75,0.577,0.885,0.023,-0.005,0.052
-TiRex,Moirai-2.0,0.788,0.615,0.923,0.043,0.014,0.075
-TiRex,Toto-1.0,0.788,0.615,0.923,0.049,0.024,0.077
-TiRex,Sundial-Base,0.788,0.635,0.923,0.083,0.009,0.153
-TiRex,AutoARIMA,0.962,0.885,1.0,0.277,0.21,0.351
-TiRex,Stat. Ensemble,0.962,0.885,1.0,0.294,0.229,0.365
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.344,0.286,0.404
-TiRex,AutoETS,0.962,0.885,1.0,0.497,0.4,0.581
-TiRex,AutoTheta,1.0,1.0,1.0,0.408,0.321,0.5
-TiRex,Naive,1.0,1.0,1.0,0.592,0.513,0.666
-TiRex,Drift,1.0,1.0,1.0,0.608,0.534,0.676
-TimesFM-2.5,Chronos-2,0.231,0.077,0.423,-0.189,-0.291,-0.09
-TimesFM-2.5,TiRex,0.327,0.154,0.519,-0.021,-0.047,0.002
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TabPFN-TS,0.538,0.346,0.731,-0.081,-0.195,0.009
-TimesFM-2.5,Chronos-Bolt,0.538,0.365,0.712,0.002,-0.028,0.029
-TimesFM-2.5,Moirai-2.0,0.615,0.442,0.808,0.022,-0.023,0.067
-TimesFM-2.5,Toto-1.0,0.654,0.462,0.808,0.029,-0.01,0.069
-TimesFM-2.5,Sundial-Base,0.788,0.635,0.923,0.063,-0.021,0.143
-TimesFM-2.5,AutoARIMA,1.0,1.0,1.0,0.261,0.191,0.336
-TimesFM-2.5,Stat. Ensemble,0.962,0.885,1.0,0.279,0.207,0.357
-TimesFM-2.5,Seasonal Naive,1.0,1.0,1.0,0.33,0.269,0.394
-TimesFM-2.5,AutoETS,0.962,0.885,1.0,0.486,0.387,0.574
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.395,0.304,0.497
-TimesFM-2.5,Naive,1.0,1.0,1.0,0.583,0.5,0.659
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.6,0.521,0.672
-TabPFN-TS,Chronos-2,0.154,0.038,0.308,-0.1,-0.15,-0.052
-TabPFN-TS,TiRex,0.423,0.231,0.615,0.055,-0.03,0.141
-TabPFN-TS,TimesFM-2.5,0.462,0.269,0.654,0.075,-0.009,0.163
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Chronos-Bolt,0.615,0.423,0.808,0.077,0.005,0.156
-TabPFN-TS,Moirai-2.0,0.538,0.346,0.731,0.096,0.014,0.181
-TabPFN-TS,Toto-1.0,0.577,0.385,0.769,0.102,0.017,0.189
-TabPFN-TS,Sundial-Base,0.769,0.577,0.923,0.134,0.048,0.222
-TabPFN-TS,AutoARIMA,0.923,0.808,1.0,0.317,0.222,0.406
-TabPFN-TS,Stat. Ensemble,0.923,0.808,1.0,0.333,0.243,0.417
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.381,0.298,0.457
-TabPFN-TS,AutoETS,0.923,0.808,1.0,0.524,0.416,0.616
-TabPFN-TS,AutoTheta,1.0,1.0,1.0,0.44,0.342,0.539
-TabPFN-TS,Naive,1.0,1.0,1.0,0.614,0.529,0.691
-TabPFN-TS,Drift,1.0,1.0,1.0,0.63,0.547,0.702
-Chronos-Bolt,Chronos-2,0.077,0.0,0.192,-0.192,-0.28,-0.106
-Chronos-Bolt,TiRex,0.25,0.115,0.423,-0.024,-0.055,0.005
-Chronos-Bolt,TimesFM-2.5,0.462,0.288,0.635,-0.002,-0.03,0.027
-Chronos-Bolt,TabPFN-TS,0.385,0.192,0.577,-0.084,-0.185,-0.005
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,Moirai-2.0,0.5,0.327,0.654,0.02,-0.025,0.07
-Chronos-Bolt,Toto-1.0,0.538,0.365,0.712,0.026,-0.012,0.068
-Chronos-Bolt,Sundial-Base,0.75,0.596,0.904,0.061,-0.024,0.136
-Chronos-Bolt,AutoARIMA,1.0,1.0,1.0,0.259,0.191,0.333
-Chronos-Bolt,Stat. Ensemble,0.962,0.885,1.0,0.277,0.211,0.349
-Chronos-Bolt,Seasonal Naive,1.0,1.0,1.0,0.329,0.273,0.388
-Chronos-Bolt,AutoETS,0.962,0.885,1.0,0.485,0.384,0.576
-Chronos-Bolt,AutoTheta,1.0,1.0,1.0,0.393,0.304,0.492
-Chronos-Bolt,Naive,1.0,1.0,1.0,0.582,0.5,0.654
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.599,0.52,0.667
-Moirai-2.0,Chronos-2,0.077,0.0,0.192,-0.216,-0.319,-0.125
-Moirai-2.0,TiRex,0.212,0.077,0.385,-0.045,-0.081,-0.014
-Moirai-2.0,TimesFM-2.5,0.385,0.192,0.558,-0.023,-0.071,0.023
-Moirai-2.0,TabPFN-TS,0.462,0.269,0.654,-0.106,-0.221,-0.014
-Moirai-2.0,Chronos-Bolt,0.5,0.346,0.673,-0.02,-0.075,0.025
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Toto-1.0,0.577,0.423,0.75,0.006,-0.016,0.035
-Moirai-2.0,Sundial-Base,0.75,0.596,0.904,0.042,-0.051,0.124
-Moirai-2.0,AutoARIMA,0.923,0.808,1.0,0.244,0.163,0.328
-Moirai-2.0,Stat. Ensemble,0.923,0.808,1.0,0.263,0.18,0.345
-Moirai-2.0,Seasonal Naive,0.923,0.808,1.0,0.315,0.249,0.385
-Moirai-2.0,AutoETS,0.923,0.808,1.0,0.474,0.366,0.571
-Moirai-2.0,AutoTheta,0.923,0.808,1.0,0.381,0.287,0.48
-Moirai-2.0,Naive,0.962,0.885,1.0,0.574,0.491,0.651
-Moirai-2.0,Drift,1.0,1.0,1.0,0.591,0.511,0.663
-Toto-1.0,Chronos-2,0.115,0.0,0.269,-0.224,-0.333,-0.13
-Toto-1.0,TiRex,0.212,0.077,0.385,-0.051,-0.084,-0.025
-Toto-1.0,TimesFM-2.5,0.346,0.192,0.538,-0.029,-0.075,0.01
-Toto-1.0,TabPFN-TS,0.423,0.231,0.615,-0.113,-0.232,-0.018
-Toto-1.0,Chronos-Bolt,0.462,0.288,0.635,-0.027,-0.073,0.012
-Toto-1.0,Moirai-2.0,0.423,0.25,0.577,-0.007,-0.036,0.016
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,Sundial-Base,0.75,0.596,0.904,0.036,-0.053,0.113
-Toto-1.0,AutoARIMA,0.923,0.808,1.0,0.239,0.163,0.318
-Toto-1.0,Stat. Ensemble,0.923,0.808,1.0,0.258,0.178,0.337
-Toto-1.0,Seasonal Naive,0.962,0.885,1.0,0.311,0.247,0.377
-Toto-1.0,AutoETS,0.923,0.808,1.0,0.471,0.366,0.564
-Toto-1.0,AutoTheta,0.923,0.808,1.0,0.377,0.277,0.477
-Toto-1.0,Naive,1.0,1.0,1.0,0.571,0.49,0.645
-Toto-1.0,Drift,1.0,1.0,1.0,0.588,0.51,0.658
-Sundial-Base,Chronos-2,0.038,0.0,0.115,-0.269,-0.379,-0.186
-Sundial-Base,TiRex,0.212,0.077,0.365,-0.09,-0.18,-0.009
-Sundial-Base,TimesFM-2.5,0.212,0.077,0.365,-0.067,-0.168,0.021
-Sundial-Base,TabPFN-TS,0.231,0.077,0.423,-0.154,-0.285,-0.05
-Sundial-Base,Chronos-Bolt,0.25,0.096,0.404,-0.065,-0.157,0.023
-Sundial-Base,Moirai-2.0,0.25,0.096,0.404,-0.044,-0.142,0.048
-Sundial-Base,Toto-1.0,0.25,0.096,0.404,-0.037,-0.127,0.05
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,AutoARIMA,0.769,0.577,0.923,0.211,0.119,0.291
-Sundial-Base,Stat. Ensemble,0.808,0.654,0.923,0.23,0.145,0.306
-Sundial-Base,Seasonal Naive,0.962,0.885,1.0,0.285,0.209,0.359
-Sundial-Base,AutoETS,0.846,0.692,0.962,0.451,0.336,0.535
-Sundial-Base,AutoTheta,0.923,0.808,1.0,0.354,0.246,0.448
-Sundial-Base,Naive,1.0,1.0,1.0,0.555,0.46,0.641
-Sundial-Base,Drift,1.0,1.0,1.0,0.573,0.482,0.653
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-0.609,-0.817,-0.423
-AutoARIMA,TiRex,0.038,0.0,0.115,-0.382,-0.541,-0.266
-AutoARIMA,TimesFM-2.5,0.0,0.0,0.0,-0.353,-0.505,-0.237
-AutoARIMA,TabPFN-TS,0.077,0.0,0.192,-0.463,-0.683,-0.285
-AutoARIMA,Chronos-Bolt,0.0,0.0,0.0,-0.35,-0.499,-0.236
-AutoARIMA,Moirai-2.0,0.077,0.0,0.192,-0.323,-0.488,-0.195
-AutoARIMA,Toto-1.0,0.077,0.0,0.192,-0.315,-0.466,-0.195
-AutoARIMA,Sundial-Base,0.231,0.077,0.423,-0.268,-0.41,-0.135
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,Stat. Ensemble,0.615,0.423,0.788,0.024,-0.008,0.057
-AutoARIMA,Seasonal Naive,0.885,0.769,0.962,0.094,0.068,0.121
-AutoARIMA,AutoETS,0.731,0.538,0.885,0.304,0.194,0.406
-AutoARIMA,AutoTheta,0.885,0.731,1.0,0.181,0.078,0.284
-AutoARIMA,Naive,0.923,0.808,1.0,0.436,0.348,0.519
-AutoARIMA,Drift,0.962,0.885,1.0,0.458,0.375,0.539
-Stat. Ensemble,Chronos-2,0.0,0.0,0.0,-0.649,-0.854,-0.464
-Stat. Ensemble,TiRex,0.038,0.0,0.115,-0.417,-0.575,-0.297
-Stat. Ensemble,TimesFM-2.5,0.038,0.0,0.115,-0.387,-0.556,-0.262
-Stat. Ensemble,TabPFN-TS,0.077,0.0,0.192,-0.5,-0.715,-0.32
-Stat. Ensemble,Chronos-Bolt,0.038,0.0,0.115,-0.384,-0.536,-0.268
-Stat. Ensemble,Moirai-2.0,0.077,0.0,0.192,-0.356,-0.526,-0.219
-Stat. Ensemble,Toto-1.0,0.077,0.0,0.192,-0.347,-0.509,-0.217
-Stat. Ensemble,Sundial-Base,0.192,0.077,0.346,-0.299,-0.441,-0.169
-Stat. Ensemble,AutoARIMA,0.385,0.212,0.577,-0.025,-0.061,0.008
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,Seasonal Naive,0.731,0.577,0.865,0.071,0.03,0.11
-Stat. Ensemble,AutoETS,0.769,0.577,0.923,0.287,0.178,0.394
-Stat. Ensemble,AutoTheta,0.885,0.731,1.0,0.161,0.074,0.253
-Stat. Ensemble,Naive,0.923,0.808,1.0,0.422,0.329,0.507
-Stat. Ensemble,Drift,0.962,0.885,1.0,0.445,0.359,0.526
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-0.776,-0.987,-0.581
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.526,-0.679,-0.401
-Seasonal Naive,TimesFM-2.5,0.0,0.0,0.0,-0.493,-0.65,-0.368
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.615,-0.841,-0.425
-Seasonal Naive,Chronos-Bolt,0.0,0.0,0.0,-0.49,-0.633,-0.376
-Seasonal Naive,Moirai-2.0,0.077,0.0,0.192,-0.46,-0.627,-0.331
-Seasonal Naive,Toto-1.0,0.038,0.0,0.115,-0.451,-0.605,-0.329
-Seasonal Naive,Sundial-Base,0.038,0.0,0.115,-0.399,-0.56,-0.264
-Seasonal Naive,AutoARIMA,0.115,0.038,0.231,-0.104,-0.138,-0.073
-Seasonal Naive,Stat. Ensemble,0.269,0.135,0.423,-0.077,-0.124,-0.031
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,AutoETS,0.615,0.423,0.77,0.232,0.1,0.359
-Seasonal Naive,AutoTheta,0.615,0.423,0.808,0.096,-0.022,0.215
-Seasonal Naive,Naive,0.865,0.75,0.962,0.377,0.284,0.463
-Seasonal Naive,Drift,1.0,1.0,1.0,0.402,0.311,0.485
-AutoETS,Chronos-2,0.0,0.0,0.0,-1.313,-1.857,-0.881
-AutoETS,TiRex,0.038,0.0,0.115,-0.986,-1.387,-0.666
-AutoETS,TimesFM-2.5,0.038,0.0,0.115,-0.945,-1.345,-0.631
-AutoETS,TabPFN-TS,0.077,0.0,0.192,-1.103,-1.606,-0.713
-AutoETS,Chronos-Bolt,0.038,0.0,0.115,-0.94,-1.357,-0.624
-AutoETS,Moirai-2.0,0.077,0.0,0.192,-0.902,-1.33,-0.578
-AutoETS,Toto-1.0,0.077,0.0,0.192,-0.889,-1.292,-0.576
-AutoETS,Sundial-Base,0.154,0.038,0.308,-0.822,-1.149,-0.507
-AutoETS,AutoARIMA,0.269,0.115,0.462,-0.437,-0.684,-0.241
-AutoETS,Stat. Ensemble,0.231,0.077,0.423,-0.402,-0.651,-0.217
-AutoETS,Seasonal Naive,0.385,0.23,0.577,-0.302,-0.559,-0.111
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,AutoTheta,0.5,0.308,0.692,-0.177,-0.448,0.013
-AutoETS,Naive,0.692,0.5,0.846,0.189,0.022,0.343
-AutoETS,Drift,0.885,0.731,1.0,0.222,0.059,0.368
-AutoTheta,Chronos-2,0.0,0.0,0.0,-0.965,-1.35,-0.685
-AutoTheta,TiRex,0.0,0.0,0.0,-0.688,-0.999,-0.474
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-0.652,-0.988,-0.437
-AutoTheta,TabPFN-TS,0.0,0.0,0.0,-0.787,-1.168,-0.519
-AutoTheta,Chronos-Bolt,0.0,0.0,0.0,-0.648,-0.97,-0.437
-AutoTheta,Moirai-2.0,0.077,0.0,0.192,-0.616,-0.923,-0.402
-AutoTheta,Toto-1.0,0.077,0.0,0.192,-0.605,-0.913,-0.384
-AutoTheta,Sundial-Base,0.077,0.0,0.192,-0.548,-0.811,-0.327
-AutoTheta,AutoARIMA,0.115,0.0,0.269,-0.221,-0.397,-0.085
-AutoTheta,Stat. Ensemble,0.115,0.0,0.269,-0.191,-0.339,-0.08
-AutoTheta,Seasonal Naive,0.385,0.192,0.577,-0.106,-0.273,0.022
-AutoTheta,AutoETS,0.5,0.308,0.692,0.15,-0.013,0.309
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Naive,0.654,0.462,0.846,0.311,0.19,0.426
-AutoTheta,Drift,0.808,0.654,0.923,0.339,0.219,0.449
-Naive,Chronos-2,0.0,0.0,0.0,-1.852,-2.524,-1.341
-Naive,TiRex,0.0,0.0,0.0,-1.45,-1.993,-1.054
-Naive,TimesFM-2.5,0.0,0.0,0.0,-1.398,-1.932,-1.0
-Naive,TabPFN-TS,0.0,0.0,0.0,-1.593,-2.233,-1.123
-Naive,Chronos-Bolt,0.0,0.0,0.0,-1.392,-1.887,-1.001
-Naive,Moirai-2.0,0.038,0.0,0.115,-1.345,-1.866,-0.964
-Naive,Toto-1.0,0.0,0.0,0.0,-1.33,-1.815,-0.961
-Naive,Sundial-Base,0.0,0.0,0.0,-1.247,-1.782,-0.853
-Naive,AutoARIMA,0.077,0.0,0.192,-0.772,-1.079,-0.535
-Naive,Stat. Ensemble,0.077,0.0,0.192,-0.729,-1.027,-0.491
-Naive,Seasonal Naive,0.135,0.038,0.25,-0.606,-0.862,-0.397
-Naive,AutoETS,0.308,0.154,0.5,-0.233,-0.521,-0.023
-Naive,AutoTheta,0.346,0.154,0.538,-0.451,-0.743,-0.234
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Drift,0.962,0.885,1.0,0.04,0.025,0.057
-Drift,Chronos-2,0.0,0.0,0.0,-1.971,-2.673,-1.447
-Drift,TiRex,0.0,0.0,0.0,-1.552,-2.088,-1.148
-Drift,TimesFM-2.5,0.0,0.0,0.0,-1.499,-2.051,-1.089
-Drift,TabPFN-TS,0.0,0.0,0.0,-1.702,-2.354,-1.208
-Drift,Chronos-Bolt,0.0,0.0,0.0,-1.493,-2.007,-1.085
-Drift,Moirai-2.0,0.0,0.0,0.0,-1.443,-1.963,-1.045
-Drift,Toto-1.0,0.0,0.0,0.0,-1.427,-1.926,-1.041
-Drift,Sundial-Base,0.0,0.0,0.0,-1.341,-1.884,-0.929
-Drift,AutoARIMA,0.038,0.0,0.115,-0.846,-1.169,-0.599
-Drift,Stat. Ensemble,0.038,0.0,0.115,-0.802,-1.111,-0.559
-Drift,Seasonal Naive,0.0,0.0,0.0,-0.673,-0.942,-0.452
-Drift,AutoETS,0.115,0.0,0.269,-0.285,-0.583,-0.063
-Drift,AutoTheta,0.192,0.077,0.346,-0.512,-0.816,-0.28
-Drift,Naive,0.038,0.0,0.115,-0.042,-0.06,-0.026
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_energy/pairwise_WAPE.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,TimesFM-2.5,0.731,0.538,0.885,0.149,0.075,0.214
-Chronos-2,TiRex,0.846,0.692,0.962,0.141,0.074,0.204
-Chronos-2,Chronos-Bolt,0.923,0.808,1.0,0.149,0.081,0.219
-Chronos-2,TabPFN-TS,0.846,0.692,0.962,0.104,0.034,0.174
-Chronos-2,Moirai-2.0,0.885,0.731,1.0,0.156,0.089,0.221
-Chronos-2,Sundial-Base,0.846,0.692,0.962,0.13,0.071,0.194
-Chronos-2,Toto-1.0,0.846,0.692,0.962,0.177,0.109,0.245
-Chronos-2,Stat. Ensemble,0.962,0.885,1.0,0.331,0.242,0.409
-Chronos-2,AutoARIMA,0.962,0.885,1.0,0.358,0.267,0.438
-Chronos-2,AutoTheta,0.962,0.885,1.0,0.348,0.276,0.418
-Chronos-2,Naive,0.885,0.731,1.0,0.518,0.403,0.613
-Chronos-2,Seasonal Naive,0.923,0.808,1.0,0.376,0.282,0.449
-Chronos-2,AutoETS,0.885,0.731,1.0,0.497,0.365,0.605
-Chronos-2,Drift,0.923,0.808,1.0,0.544,0.439,0.636
-TimesFM-2.5,Chronos-2,0.269,0.115,0.462,-0.175,-0.273,-0.081
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,TiRex,0.442,0.25,0.635,-0.009,-0.043,0.023
-TimesFM-2.5,Chronos-Bolt,0.5,0.327,0.674,0.001,-0.039,0.042
-TimesFM-2.5,TabPFN-TS,0.5,0.308,0.692,-0.053,-0.161,0.048
-TimesFM-2.5,Moirai-2.0,0.577,0.404,0.75,0.009,-0.046,0.062
-TimesFM-2.5,Sundial-Base,0.596,0.423,0.788,-0.022,-0.12,0.067
-TimesFM-2.5,Toto-1.0,0.654,0.481,0.808,0.033,-0.012,0.081
-TimesFM-2.5,Stat. Ensemble,0.808,0.615,0.962,0.214,0.126,0.294
-TimesFM-2.5,AutoARIMA,0.923,0.808,1.0,0.246,0.159,0.324
-TimesFM-2.5,AutoTheta,0.962,0.885,1.0,0.234,0.168,0.307
-TimesFM-2.5,Naive,0.885,0.731,1.0,0.434,0.317,0.538
-TimesFM-2.5,Seasonal Naive,0.923,0.808,1.0,0.267,0.184,0.338
-TimesFM-2.5,AutoETS,0.846,0.692,0.962,0.409,0.27,0.532
-TimesFM-2.5,Drift,0.885,0.731,1.0,0.464,0.352,0.562
-TiRex,Chronos-2,0.154,0.038,0.308,-0.164,-0.256,-0.08
-TiRex,TimesFM-2.5,0.558,0.365,0.75,0.009,-0.023,0.041
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,Chronos-Bolt,0.481,0.308,0.673,0.01,-0.028,0.06
-TiRex,TabPFN-TS,0.538,0.346,0.731,-0.043,-0.165,0.065
-TiRex,Moirai-2.0,0.442,0.269,0.615,0.018,-0.013,0.051
-TiRex,Sundial-Base,0.635,0.462,0.808,-0.013,-0.12,0.076
-TiRex,Toto-1.0,0.712,0.538,0.885,0.042,0.009,0.077
-TiRex,Stat. Ensemble,0.808,0.654,0.924,0.221,0.134,0.303
-TiRex,AutoARIMA,0.885,0.731,1.0,0.253,0.169,0.337
-TiRex,AutoTheta,1.0,1.0,1.0,0.241,0.179,0.313
-TiRex,Naive,0.885,0.731,1.0,0.439,0.321,0.545
-TiRex,Seasonal Naive,0.885,0.731,1.0,0.273,0.184,0.352
-TiRex,AutoETS,0.846,0.692,0.962,0.414,0.268,0.54
-TiRex,Drift,0.885,0.731,1.0,0.469,0.354,0.568
-Chronos-Bolt,Chronos-2,0.077,0.0,0.192,-0.175,-0.28,-0.089
-Chronos-Bolt,TimesFM-2.5,0.5,0.326,0.673,-0.001,-0.044,0.038
-Chronos-Bolt,TiRex,0.519,0.327,0.692,-0.01,-0.064,0.027
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,TabPFN-TS,0.462,0.269,0.654,-0.053,-0.157,0.036
-Chronos-Bolt,Moirai-2.0,0.577,0.404,0.731,0.008,-0.066,0.068
-Chronos-Bolt,Sundial-Base,0.519,0.327,0.692,-0.023,-0.128,0.067
-Chronos-Bolt,Toto-1.0,0.615,0.442,0.788,0.032,-0.022,0.081
-Chronos-Bolt,Stat. Ensemble,0.731,0.538,0.885,0.214,0.122,0.3
-Chronos-Bolt,AutoARIMA,0.846,0.692,0.962,0.245,0.16,0.331
-Chronos-Bolt,AutoTheta,0.962,0.885,1.0,0.234,0.166,0.312
-Chronos-Bolt,Naive,0.846,0.692,0.962,0.434,0.311,0.538
-Chronos-Bolt,Seasonal Naive,0.923,0.808,1.0,0.266,0.183,0.339
-Chronos-Bolt,AutoETS,0.808,0.654,0.924,0.408,0.263,0.533
-Chronos-Bolt,Drift,0.846,0.692,0.962,0.464,0.347,0.563
-TabPFN-TS,Chronos-2,0.154,0.038,0.308,-0.116,-0.211,-0.035
-TabPFN-TS,TimesFM-2.5,0.5,0.308,0.692,0.05,-0.05,0.139
-TabPFN-TS,TiRex,0.462,0.269,0.654,0.041,-0.07,0.142
-TabPFN-TS,Chronos-Bolt,0.538,0.346,0.731,0.051,-0.037,0.135
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,Moirai-2.0,0.538,0.346,0.731,0.059,-0.054,0.165
-TabPFN-TS,Sundial-Base,0.577,0.385,0.769,0.029,-0.091,0.135
-TabPFN-TS,Toto-1.0,0.538,0.346,0.731,0.081,-0.023,0.177
-TabPFN-TS,Stat. Ensemble,0.846,0.692,0.962,0.254,0.151,0.343
-TabPFN-TS,AutoARIMA,0.808,0.654,0.924,0.284,0.181,0.375
-TabPFN-TS,AutoTheta,0.846,0.692,0.962,0.273,0.178,0.359
-TabPFN-TS,Naive,0.808,0.654,0.923,0.462,0.33,0.57
-TabPFN-TS,Seasonal Naive,0.885,0.731,1.0,0.303,0.205,0.384
-TabPFN-TS,AutoETS,0.808,0.654,0.962,0.438,0.285,0.563
-TabPFN-TS,Drift,0.846,0.692,0.962,0.491,0.367,0.59
-Moirai-2.0,Chronos-2,0.115,0.0,0.269,-0.185,-0.284,-0.098
-Moirai-2.0,TimesFM-2.5,0.423,0.25,0.596,-0.009,-0.067,0.044
-Moirai-2.0,TiRex,0.558,0.385,0.731,-0.018,-0.054,0.013
-Moirai-2.0,Chronos-Bolt,0.423,0.269,0.596,-0.008,-0.073,0.062
-Moirai-2.0,TabPFN-TS,0.462,0.269,0.654,-0.062,-0.197,0.052
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Sundial-Base,0.596,0.423,0.769,-0.032,-0.143,0.065
-Moirai-2.0,Toto-1.0,0.577,0.404,0.75,0.024,-0.008,0.058
-Moirai-2.0,Stat. Ensemble,0.692,0.5,0.846,0.207,0.115,0.288
-Moirai-2.0,AutoARIMA,0.808,0.654,0.923,0.239,0.149,0.328
-Moirai-2.0,AutoTheta,0.962,0.885,1.0,0.228,0.162,0.3
-Moirai-2.0,Naive,0.846,0.692,0.962,0.429,0.314,0.532
-Moirai-2.0,Seasonal Naive,0.808,0.654,0.924,0.26,0.164,0.341
-Moirai-2.0,AutoETS,0.808,0.654,0.923,0.404,0.253,0.532
-Moirai-2.0,Drift,0.885,0.731,1.0,0.46,0.35,0.554
-Sundial-Base,Chronos-2,0.154,0.038,0.308,-0.149,-0.241,-0.077
-Sundial-Base,TimesFM-2.5,0.404,0.212,0.577,0.022,-0.072,0.107
-Sundial-Base,TiRex,0.365,0.192,0.538,0.013,-0.083,0.107
-Sundial-Base,Chronos-Bolt,0.481,0.308,0.673,0.023,-0.072,0.113
-Sundial-Base,TabPFN-TS,0.423,0.231,0.615,-0.03,-0.156,0.083
-Sundial-Base,Moirai-2.0,0.404,0.231,0.577,0.031,-0.069,0.125
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Toto-1.0,0.519,0.327,0.712,0.054,-0.039,0.141
-Sundial-Base,Stat. Ensemble,0.769,0.577,0.923,0.231,0.132,0.318
-Sundial-Base,AutoARIMA,0.808,0.654,0.924,0.262,0.168,0.346
-Sundial-Base,AutoTheta,0.846,0.692,0.962,0.251,0.169,0.332
-Sundial-Base,Naive,0.846,0.692,0.962,0.446,0.316,0.562
-Sundial-Base,Seasonal Naive,0.846,0.692,0.962,0.283,0.183,0.364
-Sundial-Base,AutoETS,0.808,0.654,0.924,0.422,0.278,0.534
-Sundial-Base,Drift,0.846,0.692,0.962,0.476,0.35,0.586
-Toto-1.0,Chronos-2,0.154,0.038,0.308,-0.214,-0.324,-0.122
-Toto-1.0,TimesFM-2.5,0.346,0.192,0.519,-0.034,-0.089,0.012
-Toto-1.0,TiRex,0.288,0.115,0.462,-0.043,-0.084,-0.009
-Toto-1.0,Chronos-Bolt,0.385,0.212,0.558,-0.033,-0.088,0.021
-Toto-1.0,TabPFN-TS,0.462,0.269,0.654,-0.088,-0.216,0.022
-Toto-1.0,Moirai-2.0,0.423,0.25,0.596,-0.025,-0.061,0.008
-Toto-1.0,Sundial-Base,0.481,0.288,0.673,-0.057,-0.164,0.038
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,Stat. Ensemble,0.692,0.5,0.846,0.188,0.094,0.273
-Toto-1.0,AutoARIMA,0.846,0.692,0.962,0.22,0.129,0.309
-Toto-1.0,AutoTheta,0.846,0.692,0.962,0.208,0.137,0.284
-Toto-1.0,Naive,0.808,0.654,0.924,0.415,0.296,0.517
-Toto-1.0,Seasonal Naive,0.846,0.654,0.962,0.242,0.151,0.321
-Toto-1.0,AutoETS,0.769,0.614,0.923,0.389,0.242,0.513
-Toto-1.0,Drift,0.885,0.769,1.0,0.446,0.337,0.542
-Stat. Ensemble,Chronos-2,0.038,0.0,0.115,-0.495,-0.692,-0.319
-Stat. Ensemble,TimesFM-2.5,0.192,0.038,0.385,-0.273,-0.416,-0.144
-Stat. Ensemble,TiRex,0.192,0.076,0.346,-0.284,-0.434,-0.154
-Stat. Ensemble,Chronos-Bolt,0.269,0.115,0.462,-0.272,-0.429,-0.139
-Stat. Ensemble,TabPFN-TS,0.154,0.038,0.308,-0.34,-0.522,-0.178
-Stat. Ensemble,Moirai-2.0,0.308,0.154,0.5,-0.261,-0.405,-0.13
-Stat. Ensemble,Sundial-Base,0.231,0.077,0.423,-0.301,-0.467,-0.152
-Stat. Ensemble,Toto-1.0,0.308,0.154,0.5,-0.231,-0.376,-0.104
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoARIMA,0.654,0.481,0.827,0.04,0.008,0.076
-Stat. Ensemble,AutoTheta,0.731,0.538,0.885,0.026,-0.029,0.075
-Stat. Ensemble,Naive,0.808,0.654,0.962,0.28,0.162,0.399
-Stat. Ensemble,Seasonal Naive,0.692,0.538,0.846,0.067,0.029,0.103
-Stat. Ensemble,AutoETS,0.808,0.654,0.923,0.248,0.12,0.381
-Stat. Ensemble,Drift,0.808,0.654,0.962,0.318,0.206,0.435
-AutoARIMA,Chronos-2,0.038,0.0,0.115,-0.558,-0.781,-0.363
-AutoARIMA,TimesFM-2.5,0.077,0.0,0.192,-0.326,-0.48,-0.19
-AutoARIMA,TiRex,0.115,0.0,0.269,-0.338,-0.509,-0.203
-AutoARIMA,Chronos-Bolt,0.154,0.038,0.308,-0.325,-0.496,-0.19
-AutoARIMA,TabPFN-TS,0.192,0.076,0.346,-0.396,-0.601,-0.222
-AutoARIMA,Moirai-2.0,0.192,0.077,0.346,-0.314,-0.489,-0.175
-AutoARIMA,Sundial-Base,0.192,0.076,0.346,-0.356,-0.529,-0.202
-AutoARIMA,Toto-1.0,0.154,0.038,0.308,-0.283,-0.447,-0.148
-AutoARIMA,Stat. Ensemble,0.346,0.173,0.519,-0.042,-0.082,-0.008
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,AutoTheta,0.462,0.269,0.654,-0.015,-0.079,0.04
-AutoARIMA,Naive,0.731,0.577,0.885,0.249,0.12,0.379
-AutoARIMA,Seasonal Naive,0.692,0.519,0.846,0.027,-0.012,0.062
-AutoARIMA,AutoETS,0.615,0.423,0.808,0.216,0.079,0.35
-AutoARIMA,Drift,0.731,0.577,0.885,0.29,0.167,0.413
-AutoTheta,Chronos-2,0.038,0.0,0.115,-0.534,-0.718,-0.381
-AutoTheta,TimesFM-2.5,0.038,0.0,0.115,-0.306,-0.443,-0.202
-AutoTheta,TiRex,0.0,0.0,0.0,-0.318,-0.455,-0.218
-AutoTheta,Chronos-Bolt,0.038,0.0,0.115,-0.305,-0.452,-0.2
-AutoTheta,TabPFN-TS,0.154,0.038,0.308,-0.375,-0.561,-0.216
-AutoTheta,Moirai-2.0,0.038,0.0,0.115,-0.295,-0.428,-0.193
-AutoTheta,Sundial-Base,0.154,0.038,0.308,-0.335,-0.496,-0.204
-AutoTheta,Toto-1.0,0.154,0.038,0.308,-0.263,-0.397,-0.158
-AutoTheta,Stat. Ensemble,0.269,0.115,0.462,-0.026,-0.081,0.029
-AutoTheta,AutoARIMA,0.538,0.346,0.731,0.015,-0.042,0.073
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Naive,0.654,0.462,0.846,0.261,0.138,0.381
-AutoTheta,Seasonal Naive,0.615,0.423,0.808,0.042,-0.02,0.097
-AutoTheta,AutoETS,0.654,0.462,0.808,0.228,0.087,0.37
-AutoTheta,Drift,0.808,0.654,0.962,0.3,0.184,0.415
-Naive,Chronos-2,0.115,0.0,0.269,-1.075,-1.586,-0.674
-Naive,TimesFM-2.5,0.115,0.0,0.269,-0.767,-1.166,-0.465
-Naive,TiRex,0.115,0.0,0.269,-0.783,-1.198,-0.474
-Naive,Chronos-Bolt,0.154,0.038,0.308,-0.766,-1.162,-0.452
-Naive,TabPFN-TS,0.192,0.077,0.346,-0.86,-1.326,-0.492
-Naive,Moirai-2.0,0.154,0.038,0.308,-0.751,-1.137,-0.458
-Naive,Sundial-Base,0.154,0.038,0.308,-0.807,-1.283,-0.462
-Naive,Toto-1.0,0.192,0.076,0.346,-0.709,-1.072,-0.42
-Naive,Stat. Ensemble,0.192,0.038,0.346,-0.388,-0.664,-0.193
-Naive,AutoARIMA,0.269,0.115,0.423,-0.332,-0.611,-0.137
-Naive,AutoTheta,0.346,0.154,0.538,-0.353,-0.616,-0.16
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Seasonal Naive,0.404,0.231,0.558,-0.296,-0.551,-0.107
-Naive,AutoETS,0.577,0.385,0.769,-0.045,-0.271,0.122
-Naive,Drift,0.923,0.808,1.0,0.053,0.032,0.078
-Seasonal Naive,Chronos-2,0.077,0.0,0.192,-0.601,-0.814,-0.393
-Seasonal Naive,TimesFM-2.5,0.077,0.0,0.192,-0.363,-0.511,-0.225
-Seasonal Naive,TiRex,0.115,0.0,0.269,-0.376,-0.544,-0.225
-Seasonal Naive,Chronos-Bolt,0.077,0.0,0.192,-0.363,-0.513,-0.225
-Seasonal Naive,TabPFN-TS,0.115,0.0,0.269,-0.435,-0.622,-0.257
-Seasonal Naive,Moirai-2.0,0.192,0.076,0.346,-0.351,-0.517,-0.196
-Seasonal Naive,Sundial-Base,0.154,0.038,0.308,-0.394,-0.572,-0.225
-Seasonal Naive,Toto-1.0,0.154,0.038,0.346,-0.319,-0.473,-0.178
-Seasonal Naive,Stat. Ensemble,0.308,0.154,0.462,-0.071,-0.115,-0.03
-Seasonal Naive,AutoARIMA,0.308,0.154,0.481,-0.028,-0.066,0.012
-Seasonal Naive,AutoTheta,0.385,0.192,0.577,-0.044,-0.107,0.019
-Seasonal Naive,Naive,0.596,0.442,0.769,0.228,0.097,0.355
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,AutoETS,0.615,0.423,0.808,0.194,0.048,0.336
-Seasonal Naive,Drift,0.692,0.538,0.885,0.27,0.146,0.393
-AutoETS,Chronos-2,0.115,0.0,0.269,-0.987,-1.529,-0.575
-AutoETS,TimesFM-2.5,0.154,0.038,0.308,-0.692,-1.136,-0.369
-AutoETS,TiRex,0.154,0.038,0.308,-0.707,-1.173,-0.365
-AutoETS,Chronos-Bolt,0.192,0.076,0.346,-0.69,-1.143,-0.356
-AutoETS,TabPFN-TS,0.192,0.038,0.346,-0.781,-1.288,-0.399
-AutoETS,Moirai-2.0,0.192,0.077,0.346,-0.677,-1.137,-0.34
-AutoETS,Sundial-Base,0.192,0.076,0.346,-0.729,-1.145,-0.385
-AutoETS,Toto-1.0,0.231,0.077,0.386,-0.636,-1.055,-0.318
-AutoETS,Stat. Ensemble,0.192,0.077,0.346,-0.329,-0.614,-0.136
-AutoETS,AutoARIMA,0.385,0.192,0.577,-0.275,-0.538,-0.086
-AutoETS,AutoTheta,0.346,0.192,0.538,-0.295,-0.587,-0.095
-AutoETS,Naive,0.423,0.231,0.615,0.043,-0.139,0.213
-AutoETS,Seasonal Naive,0.385,0.192,0.577,-0.241,-0.505,-0.051
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,Drift,0.692,0.5,0.846,0.094,-0.078,0.255
-Drift,Chronos-2,0.077,0.0,0.192,-1.193,-1.748,-0.782
-Drift,TimesFM-2.5,0.115,0.0,0.269,-0.867,-1.283,-0.544
-Drift,TiRex,0.115,0.0,0.269,-0.884,-1.313,-0.548
-Drift,Chronos-Bolt,0.154,0.038,0.308,-0.866,-1.29,-0.531
-Drift,TabPFN-TS,0.154,0.038,0.308,-0.965,-1.439,-0.579
-Drift,Moirai-2.0,0.115,0.0,0.269,-0.85,-1.241,-0.539
-Drift,Sundial-Base,0.154,0.038,0.308,-0.909,-1.417,-0.539
-Drift,Toto-1.0,0.115,0.0,0.231,-0.806,-1.183,-0.509
-Drift,Stat. Ensemble,0.192,0.038,0.346,-0.467,-0.771,-0.259
-Drift,AutoARIMA,0.269,0.115,0.423,-0.408,-0.704,-0.2
-Drift,AutoTheta,0.192,0.038,0.346,-0.429,-0.71,-0.225
-Drift,Naive,0.077,0.0,0.192,-0.056,-0.085,-0.033
-Drift,Seasonal Naive,0.308,0.115,0.462,-0.369,-0.648,-0.171
-Drift,AutoETS,0.308,0.154,0.5,-0.104,-0.342,0.073
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_energy/pairwise_WQL.csv

@@ -1,226 +0,0 @@
-model_1,model_2,win_rate,win_rate_lower,win_rate_upper,skill_score,skill_score_lower,skill_score_upper
-Chronos-2,Chronos-2,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-2,TiRex,0.808,0.654,0.962,0.143,0.076,0.204
-Chronos-2,TabPFN-TS,0.808,0.654,0.962,0.103,0.043,0.165
-Chronos-2,TimesFM-2.5,0.808,0.654,0.962,0.159,0.086,0.224
-Chronos-2,Chronos-Bolt,0.962,0.885,1.0,0.165,0.099,0.23
-Chronos-2,Moirai-2.0,0.923,0.808,1.0,0.175,0.108,0.243
-Chronos-2,Toto-1.0,0.885,0.769,1.0,0.186,0.119,0.253
-Chronos-2,Sundial-Base,0.962,0.885,1.0,0.212,0.16,0.273
-Chronos-2,Stat. Ensemble,1.0,1.0,1.0,0.401,0.322,0.47
-Chronos-2,AutoARIMA,1.0,1.0,1.0,0.402,0.319,0.474
-Chronos-2,AutoETS,0.962,0.885,1.0,0.565,0.464,0.652
-Chronos-2,Seasonal Naive,1.0,1.0,1.0,0.46,0.384,0.522
-Chronos-2,AutoTheta,1.0,1.0,1.0,0.485,0.417,0.553
-Chronos-2,Naive,1.0,1.0,1.0,0.642,0.566,0.711
-Chronos-2,Drift,1.0,1.0,1.0,0.657,0.583,0.723
-TiRex,Chronos-2,0.192,0.038,0.346,-0.166,-0.256,-0.083
-TiRex,TiRex,0.5,0.5,0.5,0.0,0.0,0.0
-TiRex,TabPFN-TS,0.462,0.269,0.654,-0.047,-0.16,0.056
-TiRex,TimesFM-2.5,0.673,0.5,0.846,0.02,-0.015,0.053
-TiRex,Chronos-Bolt,0.712,0.538,0.865,0.026,-0.018,0.077
-TiRex,Moirai-2.0,0.75,0.577,0.904,0.038,0.007,0.071
-TiRex,Toto-1.0,0.827,0.673,0.962,0.051,0.023,0.082
-TiRex,Sundial-Base,0.788,0.635,0.923,0.082,-0.005,0.162
-TiRex,Stat. Ensemble,0.923,0.808,1.0,0.302,0.222,0.379
-TiRex,AutoARIMA,0.962,0.885,1.0,0.302,0.222,0.38
-TiRex,AutoETS,0.962,0.885,1.0,0.493,0.379,0.594
-TiRex,Seasonal Naive,1.0,1.0,1.0,0.37,0.299,0.439
-TiRex,AutoTheta,1.0,1.0,1.0,0.4,0.33,0.472
-TiRex,Naive,1.0,1.0,1.0,0.583,0.496,0.662
-TiRex,Drift,1.0,1.0,1.0,0.6,0.519,0.674
-TabPFN-TS,Chronos-2,0.192,0.038,0.346,-0.114,-0.198,-0.045
-TabPFN-TS,TiRex,0.538,0.346,0.731,0.045,-0.06,0.138
-TabPFN-TS,TabPFN-TS,0.5,0.5,0.5,0.0,0.0,0.0
-TabPFN-TS,TimesFM-2.5,0.5,0.308,0.692,0.063,-0.03,0.15
-TabPFN-TS,Chronos-Bolt,0.577,0.385,0.769,0.069,-0.012,0.148
-TabPFN-TS,Moirai-2.0,0.615,0.423,0.808,0.081,-0.022,0.18
-TabPFN-TS,Toto-1.0,0.577,0.385,0.769,0.093,-0.007,0.189
-TabPFN-TS,Sundial-Base,0.769,0.615,0.923,0.123,0.016,0.219
-TabPFN-TS,Stat. Ensemble,0.962,0.885,1.0,0.333,0.245,0.415
-TabPFN-TS,AutoARIMA,0.885,0.731,1.0,0.333,0.24,0.418
-TabPFN-TS,AutoETS,0.962,0.885,1.0,0.516,0.398,0.612
-TabPFN-TS,Seasonal Naive,1.0,1.0,1.0,0.398,0.321,0.473
-TabPFN-TS,AutoTheta,1.0,1.0,1.0,0.426,0.336,0.513
-TabPFN-TS,Naive,1.0,1.0,1.0,0.601,0.507,0.685
-TabPFN-TS,Drift,1.0,1.0,1.0,0.618,0.526,0.695
-TimesFM-2.5,Chronos-2,0.192,0.038,0.346,-0.19,-0.289,-0.094
-TimesFM-2.5,TiRex,0.327,0.154,0.5,-0.02,-0.056,0.015
-TimesFM-2.5,TabPFN-TS,0.5,0.308,0.692,-0.068,-0.177,0.029
-TimesFM-2.5,TimesFM-2.5,0.5,0.5,0.5,0.0,0.0,0.0
-TimesFM-2.5,Chronos-Bolt,0.577,0.404,0.75,0.006,-0.031,0.046
-TimesFM-2.5,Moirai-2.0,0.615,0.442,0.788,0.018,-0.038,0.072
-TimesFM-2.5,Toto-1.0,0.654,0.481,0.808,0.032,-0.016,0.081
-TimesFM-2.5,Sundial-Base,0.75,0.596,0.904,0.063,-0.029,0.147
-TimesFM-2.5,Stat. Ensemble,0.962,0.885,1.0,0.288,0.209,0.366
-TimesFM-2.5,AutoARIMA,1.0,1.0,1.0,0.288,0.212,0.365
-TimesFM-2.5,AutoETS,0.962,0.885,1.0,0.483,0.376,0.581
-TimesFM-2.5,Seasonal Naive,1.0,1.0,1.0,0.357,0.288,0.423
-TimesFM-2.5,AutoTheta,1.0,1.0,1.0,0.387,0.31,0.47
-TimesFM-2.5,Naive,1.0,1.0,1.0,0.574,0.483,0.657
-TimesFM-2.5,Drift,1.0,1.0,1.0,0.592,0.507,0.67
-Chronos-Bolt,Chronos-2,0.038,0.0,0.115,-0.197,-0.299,-0.11
-Chronos-Bolt,TiRex,0.288,0.135,0.462,-0.027,-0.084,0.018
-Chronos-Bolt,TabPFN-TS,0.423,0.231,0.615,-0.074,-0.174,0.012
-Chronos-Bolt,TimesFM-2.5,0.423,0.25,0.596,-0.006,-0.048,0.03
-Chronos-Bolt,Chronos-Bolt,0.5,0.5,0.5,0.0,0.0,0.0
-Chronos-Bolt,Moirai-2.0,0.462,0.308,0.635,0.012,-0.062,0.076
-Chronos-Bolt,Toto-1.0,0.577,0.404,0.75,0.026,-0.03,0.079
-Chronos-Bolt,Sundial-Base,0.75,0.596,0.904,0.057,-0.039,0.142
-Chronos-Bolt,Stat. Ensemble,0.923,0.808,1.0,0.283,0.207,0.358
-Chronos-Bolt,AutoARIMA,0.923,0.808,1.0,0.284,0.204,0.362
-Chronos-Bolt,AutoETS,0.962,0.885,1.0,0.48,0.368,0.58
-Chronos-Bolt,Seasonal Naive,1.0,1.0,1.0,0.353,0.287,0.417
-Chronos-Bolt,AutoTheta,1.0,1.0,1.0,0.384,0.307,0.466
-Chronos-Bolt,Naive,1.0,1.0,1.0,0.572,0.485,0.651
-Chronos-Bolt,Drift,1.0,1.0,1.0,0.589,0.506,0.663
-Moirai-2.0,Chronos-2,0.077,0.0,0.192,-0.212,-0.32,-0.122
-Moirai-2.0,TiRex,0.25,0.096,0.423,-0.039,-0.077,-0.007
-Moirai-2.0,TabPFN-TS,0.385,0.192,0.577,-0.088,-0.219,0.022
-Moirai-2.0,TimesFM-2.5,0.385,0.212,0.558,-0.019,-0.077,0.037
-Moirai-2.0,Chronos-Bolt,0.538,0.365,0.692,-0.012,-0.082,0.058
-Moirai-2.0,Moirai-2.0,0.5,0.5,0.5,0.0,0.0,0.0
-Moirai-2.0,Toto-1.0,0.538,0.365,0.712,0.014,-0.016,0.046
-Moirai-2.0,Sundial-Base,0.788,0.654,0.923,0.045,-0.054,0.134
-Moirai-2.0,Stat. Ensemble,0.885,0.768,1.0,0.274,0.181,0.36
-Moirai-2.0,AutoARIMA,0.923,0.808,1.0,0.275,0.183,0.363
-Moirai-2.0,AutoETS,0.923,0.808,1.0,0.473,0.354,0.58
-Moirai-2.0,Seasonal Naive,0.923,0.808,1.0,0.345,0.266,0.421
-Moirai-2.0,AutoTheta,0.923,0.808,1.0,0.376,0.293,0.457
-Moirai-2.0,Naive,0.962,0.885,1.0,0.566,0.482,0.648
-Moirai-2.0,Drift,1.0,1.0,1.0,0.584,0.502,0.661
-Toto-1.0,Chronos-2,0.115,0.0,0.231,-0.229,-0.339,-0.136
-Toto-1.0,TiRex,0.173,0.038,0.327,-0.054,-0.089,-0.024
-Toto-1.0,TabPFN-TS,0.423,0.231,0.615,-0.103,-0.233,0.007
-Toto-1.0,TimesFM-2.5,0.346,0.192,0.519,-0.033,-0.088,0.016
-Toto-1.0,Chronos-Bolt,0.423,0.25,0.596,-0.026,-0.086,0.029
-Toto-1.0,Moirai-2.0,0.462,0.288,0.635,-0.014,-0.048,0.016
-Toto-1.0,Toto-1.0,0.5,0.5,0.5,0.0,0.0,0.0
-Toto-1.0,Sundial-Base,0.712,0.557,0.865,0.032,-0.064,0.118
-Toto-1.0,Stat. Ensemble,0.846,0.692,0.962,0.264,0.176,0.347
-Toto-1.0,AutoARIMA,0.885,0.731,1.0,0.265,0.173,0.351
-Toto-1.0,AutoETS,0.885,0.731,1.0,0.466,0.343,0.573
-Toto-1.0,Seasonal Naive,0.962,0.885,1.0,0.336,0.258,0.412
-Toto-1.0,AutoTheta,0.962,0.885,1.0,0.367,0.282,0.45
-Toto-1.0,Naive,0.962,0.885,1.0,0.56,0.475,0.64
-Toto-1.0,Drift,0.962,0.885,1.0,0.578,0.497,0.654
-Sundial-Base,Chronos-2,0.038,0.0,0.115,-0.27,-0.375,-0.19
-Sundial-Base,TiRex,0.212,0.077,0.365,-0.089,-0.193,0.005
-Sundial-Base,TabPFN-TS,0.231,0.077,0.385,-0.14,-0.281,-0.016
-Sundial-Base,TimesFM-2.5,0.25,0.096,0.404,-0.067,-0.172,0.029
-Sundial-Base,Chronos-Bolt,0.25,0.096,0.404,-0.061,-0.166,0.037
-Sundial-Base,Moirai-2.0,0.212,0.077,0.346,-0.048,-0.154,0.052
-Sundial-Base,Toto-1.0,0.288,0.135,0.443,-0.033,-0.133,0.06
-Sundial-Base,Sundial-Base,0.5,0.5,0.5,0.0,0.0,0.0
-Sundial-Base,Stat. Ensemble,0.769,0.577,0.923,0.24,0.14,0.328
-Sundial-Base,AutoARIMA,0.808,0.654,0.962,0.24,0.142,0.324
-Sundial-Base,AutoETS,0.846,0.692,0.962,0.448,0.328,0.539
-Sundial-Base,Seasonal Naive,0.885,0.731,1.0,0.314,0.227,0.393
-Sundial-Base,AutoTheta,0.923,0.808,1.0,0.346,0.251,0.424
-Sundial-Base,Naive,0.962,0.885,1.0,0.546,0.446,0.638
-Sundial-Base,Drift,1.0,1.0,1.0,0.564,0.467,0.653
-Stat. Ensemble,Chronos-2,0.0,0.0,0.0,-0.67,-0.887,-0.475
-Stat. Ensemble,TiRex,0.077,0.0,0.192,-0.432,-0.61,-0.286
-Stat. Ensemble,TabPFN-TS,0.038,0.0,0.115,-0.499,-0.711,-0.324
-Stat. Ensemble,TimesFM-2.5,0.038,0.0,0.115,-0.404,-0.576,-0.265
-Stat. Ensemble,Chronos-Bolt,0.077,0.0,0.192,-0.395,-0.558,-0.261
-Stat. Ensemble,Moirai-2.0,0.115,0.0,0.232,-0.378,-0.563,-0.221
-Stat. Ensemble,Toto-1.0,0.154,0.038,0.308,-0.359,-0.531,-0.214
-Stat. Ensemble,Sundial-Base,0.231,0.077,0.423,-0.315,-0.489,-0.162
-Stat. Ensemble,Stat. Ensemble,0.5,0.5,0.5,0.0,0.0,0.0
-Stat. Ensemble,AutoARIMA,0.5,0.308,0.692,0.001,-0.03,0.03
-Stat. Ensemble,AutoETS,0.808,0.654,0.923,0.274,0.156,0.394
-Stat. Ensemble,Seasonal Naive,0.808,0.673,0.923,0.098,0.062,0.134
-Stat. Ensemble,AutoTheta,0.846,0.692,0.962,0.14,0.054,0.228
-Stat. Ensemble,Naive,0.962,0.885,1.0,0.403,0.302,0.5
-Stat. Ensemble,Drift,0.962,0.885,1.0,0.427,0.332,0.521
-AutoARIMA,Chronos-2,0.0,0.0,0.0,-0.672,-0.901,-0.469
-AutoARIMA,TiRex,0.038,0.0,0.115,-0.433,-0.614,-0.285
-AutoARIMA,TabPFN-TS,0.115,0.0,0.269,-0.5,-0.719,-0.316
-AutoARIMA,TimesFM-2.5,0.0,0.0,0.0,-0.405,-0.574,-0.269
-AutoARIMA,Chronos-Bolt,0.077,0.0,0.192,-0.396,-0.568,-0.256
-AutoARIMA,Moirai-2.0,0.077,0.0,0.192,-0.379,-0.569,-0.224
-AutoARIMA,Toto-1.0,0.115,0.0,0.269,-0.36,-0.54,-0.209
-AutoARIMA,Sundial-Base,0.192,0.038,0.346,-0.316,-0.48,-0.166
-AutoARIMA,Stat. Ensemble,0.5,0.308,0.692,-0.001,-0.031,0.029
-AutoARIMA,AutoARIMA,0.5,0.5,0.5,0.0,0.0,0.0
-AutoARIMA,AutoETS,0.731,0.538,0.885,0.273,0.152,0.393
-AutoARIMA,Seasonal Naive,0.846,0.712,0.962,0.097,0.065,0.133
-AutoARIMA,AutoTheta,0.846,0.692,0.962,0.139,0.048,0.225
-AutoARIMA,Naive,0.885,0.769,1.0,0.402,0.299,0.502
-AutoARIMA,Drift,0.923,0.808,1.0,0.427,0.331,0.523
-AutoETS,Chronos-2,0.038,0.0,0.115,-1.301,-1.876,-0.865
-AutoETS,TiRex,0.038,0.0,0.115,-0.973,-1.463,-0.611
-AutoETS,TabPFN-TS,0.038,0.0,0.115,-1.065,-1.575,-0.661
-AutoETS,TimesFM-2.5,0.038,0.0,0.115,-0.934,-1.389,-0.604
-AutoETS,Chronos-Bolt,0.038,0.0,0.115,-0.922,-1.378,-0.581
-AutoETS,Moirai-2.0,0.077,0.0,0.192,-0.898,-1.383,-0.547
-AutoETS,Toto-1.0,0.115,0.0,0.269,-0.873,-1.342,-0.522
-AutoETS,Sundial-Base,0.154,0.038,0.308,-0.812,-1.168,-0.489
-AutoETS,Stat. Ensemble,0.192,0.077,0.346,-0.378,-0.651,-0.185
-AutoETS,AutoARIMA,0.269,0.115,0.462,-0.376,-0.647,-0.18
-AutoETS,AutoETS,0.5,0.5,0.5,0.0,0.0,0.0
-AutoETS,Seasonal Naive,0.423,0.231,0.615,-0.243,-0.498,-0.059
-AutoETS,AutoTheta,0.423,0.231,0.615,-0.185,-0.496,0.024
-AutoETS,Naive,0.692,0.5,0.846,0.177,-0.005,0.343
-AutoETS,Drift,0.769,0.614,0.923,0.211,0.031,0.372
-Seasonal Naive,Chronos-2,0.0,0.0,0.0,-0.852,-1.093,-0.622
-Seasonal Naive,TiRex,0.0,0.0,0.0,-0.588,-0.782,-0.427
-Seasonal Naive,TabPFN-TS,0.0,0.0,0.0,-0.662,-0.896,-0.473
-Seasonal Naive,TimesFM-2.5,0.0,0.0,0.0,-0.556,-0.734,-0.404
-Seasonal Naive,Chronos-Bolt,0.0,0.0,0.0,-0.547,-0.716,-0.403
-Seasonal Naive,Moirai-2.0,0.077,0.0,0.192,-0.528,-0.727,-0.362
-Seasonal Naive,Toto-1.0,0.038,0.0,0.115,-0.507,-0.7,-0.348
-Seasonal Naive,Sundial-Base,0.115,0.0,0.269,-0.458,-0.646,-0.293
-Seasonal Naive,Stat. Ensemble,0.192,0.077,0.327,-0.109,-0.155,-0.066
-Seasonal Naive,AutoARIMA,0.154,0.038,0.288,-0.108,-0.153,-0.069
-Seasonal Naive,AutoETS,0.577,0.385,0.769,0.195,0.055,0.332
-Seasonal Naive,Seasonal Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Seasonal Naive,AutoTheta,0.5,0.308,0.692,0.047,-0.058,0.151
-Seasonal Naive,Naive,0.75,0.596,0.885,0.338,0.225,0.448
-Seasonal Naive,Drift,0.846,0.692,0.962,0.365,0.257,0.469
-AutoTheta,Chronos-2,0.0,0.0,0.0,-0.942,-1.236,-0.714
-AutoTheta,TiRex,0.0,0.0,0.0,-0.665,-0.895,-0.491
-AutoTheta,TabPFN-TS,0.0,0.0,0.0,-0.743,-1.051,-0.506
-AutoTheta,TimesFM-2.5,0.0,0.0,0.0,-0.633,-0.887,-0.449
-AutoTheta,Chronos-Bolt,0.0,0.0,0.0,-0.622,-0.874,-0.442
-AutoTheta,Moirai-2.0,0.077,0.0,0.192,-0.602,-0.84,-0.415
-AutoTheta,Toto-1.0,0.038,0.0,0.115,-0.581,-0.819,-0.392
-AutoTheta,Sundial-Base,0.077,0.0,0.192,-0.53,-0.735,-0.335
-AutoTheta,Stat. Ensemble,0.154,0.038,0.308,-0.163,-0.296,-0.057
-AutoTheta,AutoARIMA,0.154,0.038,0.308,-0.162,-0.29,-0.05
-AutoTheta,AutoETS,0.577,0.385,0.769,0.156,-0.025,0.332
-AutoTheta,Seasonal Naive,0.5,0.308,0.692,-0.049,-0.178,0.054
-AutoTheta,AutoTheta,0.5,0.5,0.5,0.0,0.0,0.0
-AutoTheta,Naive,0.692,0.5,0.846,0.305,0.169,0.434
-AutoTheta,Drift,0.769,0.615,0.923,0.334,0.2,0.458
-Naive,Chronos-2,0.0,0.0,0.0,-1.796,-2.464,-1.302
-Naive,TiRex,0.0,0.0,0.0,-1.397,-1.954,-0.985
-Naive,TabPFN-TS,0.0,0.0,0.0,-1.509,-2.175,-1.028
-Naive,TimesFM-2.5,0.0,0.0,0.0,-1.35,-1.917,-0.935
-Naive,Chronos-Bolt,0.0,0.0,0.0,-1.335,-1.866,-0.941
-Naive,Moirai-2.0,0.038,0.0,0.115,-1.306,-1.84,-0.93
-Naive,Toto-1.0,0.038,0.0,0.115,-1.275,-1.776,-0.904
-Naive,Sundial-Base,0.038,0.0,0.115,-1.201,-1.761,-0.804
-Naive,Stat. Ensemble,0.038,0.0,0.115,-0.674,-1.0,-0.432
-Naive,AutoARIMA,0.115,0.0,0.231,-0.672,-1.007,-0.426
-Naive,AutoETS,0.308,0.154,0.5,-0.215,-0.523,0.005
-Naive,Seasonal Naive,0.25,0.115,0.404,-0.51,-0.811,-0.291
-Naive,AutoTheta,0.308,0.154,0.5,-0.439,-0.768,-0.203
-Naive,Naive,0.5,0.5,0.5,0.0,0.0,0.0
-Naive,Drift,0.962,0.885,1.0,0.041,0.026,0.058
-Drift,Chronos-2,0.0,0.0,0.0,-1.915,-2.609,-1.396
-Drift,TiRex,0.0,0.0,0.0,-1.5,-2.068,-1.08
-Drift,TabPFN-TS,0.0,0.0,0.0,-1.616,-2.276,-1.108
-Drift,TimesFM-2.5,0.0,0.0,0.0,-1.45,-2.03,-1.028
-Drift,Chronos-Bolt,0.0,0.0,0.0,-1.435,-1.97,-1.024
-Drift,Moirai-2.0,0.0,0.0,0.0,-1.405,-1.953,-1.008
-Drift,Toto-1.0,0.038,0.0,0.115,-1.372,-1.889,-0.986
-Drift,Sundial-Base,0.0,0.0,0.0,-1.296,-1.882,-0.876
-Drift,Stat. Ensemble,0.038,0.0,0.115,-0.746,-1.089,-0.497
-Drift,AutoARIMA,0.077,0.0,0.192,-0.744,-1.097,-0.495
-Drift,AutoETS,0.231,0.077,0.386,-0.267,-0.592,-0.032
-Drift,Seasonal Naive,0.154,0.038,0.308,-0.574,-0.885,-0.347
-Drift,AutoTheta,0.231,0.077,0.385,-0.501,-0.843,-0.25
-Drift,Naive,0.038,0.0,0.115,-0.043,-0.062,-0.027
-Drift,Drift,0.5,0.5,0.5,0.0,0.0,0.0

tables/domain_health/leaderboard_MASE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,74.99999999999999,31.785976470244737,0.0,0.33713188125000004,0.0,0.0
-TimesFM-2.5,74.99999999999999,32.87474900010455,0.0,1.1526229114583333,0.0,0.0
-TiRex,70.0,32.475887751126365,0.0,0.3170949572916667,0.0,0.0
-TabPFN-TS,68.57142857142857,30.065972963196263,0.0,28.343891684635416,0.0,0.0
-Moirai-2.0,65.35714285714286,33.79488514595498,0.0,0.34801128166666667,0.1,0.0
-Toto-1.0,60.71428571428571,31.61987589396246,0.0,9.245004833333333,0.0,0.0
-Chronos-Bolt,56.785714285714285,30.041136141125392,0.0,0.406559469375,0.0,0.0
-Stat. Ensemble,50.71428571428572,29.22957519821191,0.0,193.737679803125,0.0,0.0
-AutoETS,49.28571428571429,28.071655389223615,0.0,2.600062978125,0.0,0.0
-Sundial-Base,48.57142857142857,22.707876070444478,0.0,8.29280267,0.0,0.0
-AutoARIMA,36.42857142857142,5.881738788340107,0.0,7.909155262500001,0.0,0.0
-AutoTheta,30.71428571428571,15.209167110886634,0.0,2.196897983333333,0.0,0.0
-Seasonal Naive,21.78571428571429,0.0,0.0,1.1288162752343749,0.0,0.0
-Naive,21.071428571428573,0.30982946320681215,0.0,1.196338605,0.0,0.0
-Drift,20.0,7.491051478030797,0.0,1.1894594236458333,0.0,0.0

tables/domain_health/leaderboard_SQL.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-Chronos-2,81.42857142857143,38.088755291215016,0.0,0.33713188125000004,0.0,0.0
-TimesFM-2.5,76.42857142857142,38.03594076932182,0.0,1.1526229114583333,0.0,0.0
-TiRex,70.71428571428572,36.908809578202614,0.0,0.3170949572916667,0.0,0.0
-Moirai-2.0,69.64285714285714,38.69101656633476,0.0,0.34801128166666667,0.1,0.0
-Toto-1.0,66.42857142857143,37.84147796075432,0.0,9.245004833333333,0.0,0.0
-TabPFN-TS,65.71428571428571,35.21500006276798,0.0,28.343891684635416,0.0,0.0
-Chronos-Bolt,61.07142857142858,36.49963485983284,0.0,0.406559469375,0.0,0.0
-AutoETS,52.85714285714287,31.12404362920507,0.0,2.600062978125,0.0,0.0
-Stat. Ensemble,49.285714285714285,31.66665055693333,0.0,193.737679803125,0.0,0.0
-Sundial-Base,38.57142857142858,23.555776015856367,0.0,8.29280267,0.0,0.0
-AutoARIMA,37.857142857142854,8.155432249884543,0.0,7.909155262500001,0.0,0.0
-AutoTheta,26.42857142857143,17.733419423817608,0.0,2.196897983333333,0.0,0.0
-Seasonal Naive,21.071428571428573,0.0,0.0,1.1288162752343749,0.0,0.0
-Naive,17.5,-7.297793633399574,0.0,1.196338605,0.0,0.0
-Drift,15.0,0.4041900778196994,0.0,1.1894594236458333,0.0,0.0

tables/domain_health/leaderboard_WAPE.csv

@@ -1,16 +0,0 @@
-model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures
-TimesFM-2.5,72.14285714285714,30.23643324970241,0.0,1.1526229114583333,0.0,0.0
-TiRex,67.14285714285715,29.759653338440085,0.0,0.3170949572916667,0.0,0.0
-Chronos-2,67.14285714285714,29.12048426647892,0.0,0.33713188125000004,0.0,0.0
-Moirai-2.0,66.07142857142858,31.59592125809332,0.0,0.34801128166666667,0.1,0.0
-Toto-1.0,64.2857142857143,28.503622669499972,0.0,9.245004833333333,0.0,0.0
-Stat. Ensemble,57.85714285714286,29.695571630620133,0.0,193.737679803125,0.0,0.0
-TabPFN-TS,57.85714285714286,22.16773042840192,0.0,28.343891684635416,0.0,0.0
-Chronos-Bolt,57.50000000000001,25.88326333574207,0.0,0.406559469375,0.0,0.0
-AutoETS,57.14285714285714,29.452418664318316,0.0,2.600062978125,0.0,0.0
-Sundial-Base,47.14285714285714,16.462367071514482,0.0,8.29280267,0.0,0.0
-AutoARIMA,35.0,15.909991992310612,0.0,7.909155262500001,0.0,0.0
-AutoTheta,30.0,14.265515884511814,0.0,2.196897983333333,0.0,0.0
-Naive,24.642857142857146,0.09410339815076885,0.0,1.196338605,0.0,0.0
-Seasonal Naive,23.92857142857143,0.0,0.0,1.1288162752343749,0.0,0.0
-Drift,22.142857142857146,6.9951032330727525,0.0,1.1894594236458333,0.0,0.0