fev-bench published: win rate 0.855 (1st), skill score 48.1

README.md

@@ -35,12 +35,15 @@ model-index:
           name: fev-bench
           type: autogluon/fev_datasets
         metrics:
+          - type: win rate
+            name: Win rate (all 100 tasks)
+            value: 0.855
           - type: SQL skill score
             name: Scaled quantile loss skill score (all 100 tasks)
             value: 48.1
         source:
           name: fev-bench leaderboard
-          url: https://huggingface.co/spaces/autogluon/fev-bench
+          url: https://huggingface.co/spaces/autogluon/fev-leaderboard
       - task:
           type: time-series-forecasting
         dataset:
@@ -67,7 +70,7 @@ Send a batch of series and a horizon, get back nine quantiles (10% to 90%) for e
 - Website: https://yhatlabs.com
 - Request API access: https://yhatlabs.com/access
 - Code, benchmark files and notebooks: https://github.com/yhatlabs/yhatlabs
-- Leaderboards: [TIME](https://huggingface.co/spaces/Real-TSF/TIME-leaderboard) · [GIFT-Eval](https://huggingface.co/spaces/Salesforce/GIFT-Eval) (submission under review) · [fev-bench](https://huggingface.co/spaces/autogluon/fev-bench) (submission under review)
+- Leaderboards: [TIME](https://huggingface.co/spaces/Real-TSF/TIME-leaderboard) · [GIFT-Eval](https://huggingface.co/spaces/Salesforce/GIFT-Eval) (submission under review) · [fev-bench](https://huggingface.co/spaces/autogluon/fev-leaderboard)
 - Research configuration (non-commercial): [yhatlabs/ChakraTS-Lab](https://huggingface.co/yhatlabs/ChakraTS-Lab)
 - Tabular model: [yhatlabs/ChakraTab](https://huggingface.co/yhatlabs/ChakraTab)
 
@@ -134,7 +137,7 @@ Quantiles are monotone in the level. The median (level 0.5) is the point forecas
 
 ## Evaluation
 
-*The TIME entry is published on the official leaderboard; the GIFT-Eval and fev-bench submissions are under review, and their numbers below are from our runs of the official protocols.*
+*The TIME and fev-bench entries are published on the official leaderboards; the GIFT-Eval submission is under review, and its numbers below are from our run of the official protocol.*
 
 All numbers come from the official benchmark protocols on the full benchmarks, scored against the official seasonal-naive baselines. No benchmark test data is used at any point, and no benchmark training split is used to fit anything.
 
@@ -165,19 +168,20 @@ Geometric mean over configurations, relative to seasonal naive; lower is better.
 | Chronos-2 | 0.698 | 0.485 |
 | TimesFM-2.5 | 0.705 | 0.490 |
 
-### fev-bench (100 tasks) — submission under review
+### fev-bench (100 tasks) — published
 
-Scaled quantile loss skill score vs seasonal naive; higher is better. The ChakraTS entry is its fev-bench evaluation configuration (`chakra-ts-fev`), which uses no component pretrained on fev-bench data, so no task is flagged for leakage.
+[fev-bench](https://huggingface.co/spaces/autogluon/fev-leaderboard) ranks models by win rate (how often a model beats each other model across the 100 tasks) and reports the scaled quantile loss skill score vs seasonal naive; higher is better for both. The ChakraTS entry is its fev-bench evaluation configuration (`chakra-ts-fev`), which uses no component pretrained on fev-bench data, so no task is flagged for leakage. Rows are the official leaderboard at merge time (October 2026, [PR #192](https://github.com/autogluon/fev/pull/192)).
 
-| Model | Skill score |
-|---|---|
-| **ChakraTS** | **48.1** |
-| TimesFM-3 (non-commercial) | 48.7 |
-| Chronos-2 | 47.3 |
-| TimesFM-2.5 | 42.2 (leaderboard figure, after its leakage replacement) |
-| T0-beta | 46.7 |
-| TiRex-2 | 45.5 |
-| Toto-2.0-2.5B | 44.4 |
+| Model | Win rate | Skill score |
+|---|---|---|
+| **ChakraTS** | **0.855** | **48.1** |
+| TimesFM-3 (non-commercial) | 0.849 | 49.0 |
+| Lingjiang 2.0 | 0.816 | 47.0 |
+| Chronos-2 | 0.778 | 47.3 |
+| T0-beta | 0.752 | 46.7 |
+| Toto-2.0-2.5B | 0.745 | 44.4 |
+| TiRex-2 | 0.741 | 45.5 |
+| TimesFM-2.5 | 0.678 | 46.7 (leaderboard figure, after its leakage replacement) |
 
 Per-task result files and the evaluation notebooks are in the [GitHub repository](https://github.com/yhatlabs/yhatlabs).