initial UQ and Claude interface

.cache/known_methods_snapshot.json

@@ -22,7 +22,8 @@
       "family": "Kalman",
       "aliases": [
         "iekf",
-        "gnsl"
+        "gnsl",
+        "gnki"
       ]
     },
     "uki": {
@@ -48,6 +49,14 @@
       "aliases": [
         "hm"
       ]
+    },
+    "ces-eki-dmc": {
+      "abbreviation": "CES-EKI-DMC",
+      "Method": "Calibrate Emulate Sample (EKI-DataMisfitController)",
+      "family": "calibrate_then_emulate",
+      "aliases": [
+        "ces-eki-dmc"
+      ]
     }
   },
   "observed_methods": [

.claude/settings.json

@@ -0,0 +1,79 @@
+{
+    "$schema": "https://json.schemastore.org/claude-code-settings.json",
+    "permissions": {
+        "allow": [
+            "Bash(julia --project *)",
+            "Read(*)",
+            "FileWrite(src/*, test/*, docs/*, examples/*, ai/*)",
+            "Edit(src/*)",
+            "Write(src/*)",
+            "Grep",
+            "Glob",
+            "Bash(find *)",
+            "Bash(ls *)",
+            "Bash(tree *)",
+            "Bash(wc *)",
+            "Bash(head *)",
+            "Bash(tail *)",
+            "Bash(stat *)",
+            "Bash(grep *)",
+            "Bash(git log*)",
+            "Bash(git diff*)",
+            "Bash(git status*)",
+            "Bash(git show*)",
+            "Bash(git blame*)",
+            "Bash(git ls-files*)",
+            "Bash(git rev-parse*)",
+            "Bash(git describe*)",
+            "Bash(git shortlog*)",
+            "Bash(git tag -l*)",
+            "Bash(git tag --list*)",
+            "Bash(git stash list*)",
+            "Bash(git stash show*)",
+            "Bash(git remote -v*)",
+            "Bash(git remote show*)",
+            "Bash(git branch -l*)",
+            "Bash(git branch --list*)",
+            "Bash(git branch -a*)",
+            "Bash(git branch -r*)"
+        ],
+        "deny": [
+            "Read(./.env)",
+            "Bash(git commit*)",
+            "Bash(git add*)",
+            "Bash(git push*)",
+            "Bash(git pull*)",
+            "Bash(git merge*)",
+            "Bash(git rebase*)",
+            "Bash(git checkout*)",
+            "Bash(git switch*)",
+            "Bash(git restore*)",
+            "Bash(git reset*)",
+            "Bash(git cherry-pick*)",
+            "Bash(git clean*)",
+            "Bash(git rm*)"
+        ]
+    },
+    "hooks": {
+        "UserPromptSubmit": [
+            {
+                "hooks": [
+                    {
+                        "type": "command",
+                        "command": "printf '\\n---\\nSKILL ROUTING: Review the available skills listed in the system context. If one clearly matches this task, invoke it via the Skill tool before proceeding. If no skill clearly matches, use AskUserQuestion to ask the user: (a) whether they would like to apply any of the listed skills to this task anyway, (b) whether they would like to create a new skill using the skill-creator skill — in which case propose a suggested skill name and one-line description tailored to this task — or (c) proceed without a skill. Also use AskUserQuestion to ask the user whether they would like to use subagents (e.g., Explore, Plan, general-purpose) for this task, and if so, how (e.g., for research, parallel work, or isolated execution). Do NOT proceed with the task until the user has answered.\\n'"
+                    }
+                ]
+            }
+        ],
+        "Stop": [
+            {
+                "hooks": [
+                    {
+                        "type": "command",
+                        "command": "python3 -c \"import sys,json; d=json.load(sys.stdin); [sys.stdout.write(json.dumps(dict(decision='block',reason='If this task changed the repo structurally (added, removed, or renamed files; changed architecture, run/build commands, data layout, or method-registry/page conventions), ask the user whether to update root CLAUDE.md; otherwise stop normally.'))+chr(10))] if not d.get('stop_hook_active') else None\""
+                    }
+                ]
+            }
+        ]
+    }
+}

.claude/skills/leaderboard-manager/SKILL.md

@@ -0,0 +1,200 @@
+---
+name: leaderboard-manager
+description: >
+  Manages the Calibration Benchmark Streamlit leaderboard dashboard — the repository at
+  calibration_benchmark/ that ranks parameter-calibration methods on forward-model-run
+  efficiency. Use this skill for ANY of the following tasks, even if the user doesn't use
+  the exact words below:
+
+  • Adding a new method to the leaderboard (e.g. "add EKS to the dashboard", "register
+    new method", "wire up the new NetCDF results file", "include algorithm X")
+  • Registering or replacing benchmark result data ("add new L96 results", "update the
+    data file", "point the app at the new .nc file")
+  • Registering UQ ensemble-results data ("add CES-EKI-DMC to the UQ leaderboard",
+    "register UQ results", "wire up the new ensemble_results.nc", "add UQ data for L96")
+  • Changing how the leaderboard table or figure is displayed ("change column order",
+    "rename a column", "show failure rate in the table", "update the chart colours",
+    "add a scoring mode", "tweak the Altair chart", "fix the x-axis ticks")
+  • Moving or duplicating a method between the Optimization and UQ leaderboards ("put
+    TEKI on the UQ board too", "populate UQ leaderboard", "register UQ results")
+  • Filling in or correcting method metadata ("add the citation for UKI", "fix the
+    summary text", "update the reference URL", "ETKI citation is wrong")
+  • Assessing or auditing the leaderboard for bugs, staleness, or dead code ("is
+    the leaderboard stale?", "check for bugs", "what's broken", "audit the dashboard")
+
+  Always invoke this skill when the user is working on this Streamlit leaderboard
+  dashboard, even if they phrase the task as a general code edit.
+---
+
+# Leaderboard Manager
+
+This skill covers maintenance tasks for the Calibration Benchmark Streamlit dashboard.
+Read the **shared model** section below (it applies to every task), then open the
+reference file for the specific task.
+
+---
+
+## Task router
+
+| User intent | Reference file to open |
+|---|---|
+| Add a method (+ its result NetCDF) | `references/add-method.md` |
+| Register new UQ ensemble-results NC | `references/add-uq-data.md` |
+| Change table columns, chart, scoring modes | `references/manage-display.md` |
+| Move/duplicate a method to the UQ leaderboard | `references/move-between-boards.md` |
+| Fill in or correct citation / summary / URL | `references/populate-metadata.md` |
+| Assess for staleness / bugs | `references/assess.md` |
+
+Open only the relevant file — the shared model here covers what all tasks need.
+
+---
+
+## Shared data model
+
+### Two leaderboards, one registry
+
+| Leaderboard | Page | Data loader | Target coord | Data dict |
+|---|---|---|---|---|
+| Optimization | `src/pages/OptimizationLeaderboard.py` | `load_metric_store()` | `rmse_target` | `DATASET_FILES` |
+| UQ | `src/pages/UQLeaderboard.py` | `load_uq_store()` → `load_uq_budget_store()` | `uq_target` | `UQ_BUDGET_FILES` |
+
+Both call `render_leaderboard()` from `src/common/leaderboard.py` — any change there
+affects both boards simultaneously.
+
+The UQ leaderboard uses a **different data pipeline** from the optimization one:
+- Optimization: `DATASET_FILES` → `_load_store()` reads NetCDF `metric` variable directly.
+- UQ: `UQ_BUDGET_FILES` → `load_uq_budget_store()` reads `output_coverage` and computes
+  budget-for-coverage on the fly. See `references/add-uq-data.md` for the schema.
+
+### Method identity chain
+
+A method's identity flows through three layers that must stay in sync:
+
+```
+NetCDF file
+  └─ algorithm_type dim (raw string, e.g. "gnki", "teki")
+       └─ normalize_method_name()  → lowercase alias
+            └─ canonicalize_method_name()  → canonical key (e.g. "iekf", "teki")
+                 └─ KNOWN_METHODS[canonical]  → { abbreviation, Method, family, aliases }
+                      └─ method_meta[abbreviation]  → { citation, url, summary }
+```
+
+`KNOWN_METHODS` lives in `src/common/method_registry.py`.
+`method_meta` lives in `src/pages/MethodDetails.py` (keyed by **abbreviation**, not the
+canonical key).
+
+### Metric semantics
+
+**Optimization leaderboard:**
+- `metric` = mean forward-model runs to reach the RMSE target (lower is better).
+- `metric == -1` encodes a **failed** run. `_load_store` strips these before averaging and
+  records their fraction as `failure_rate`.
+
+**UQ leaderboard:**
+- `metric` = mean budget N_ens·k_iter to reach the coverage tolerance (lower is better).
+- `NaN` (not -1) encodes a failed run — target never reached within k_iter range.
+- `failure_rate` = % of seeds where coverage condition was never satisfied.
+- See `references/add-uq-data.md` for details on the coverage computation.
+
+### Required NetCDF dimensions
+
+**Optimization files:**
+```python
+EXPECTED_DIMS = ("algorithm_type", "rmse_target", "ensemble_size", "random_seed")
+```
+
+**UQ files** use a completely different schema — `output_coverage` variable, not `metric`.
+See `references/add-uq-data.md` for the full spec.
+
+---
+
+## Critical wiring / gotchas
+
+These traps apply to every task. Read them before touching any file.
+
+**1. Method metadata is split across three files.**
+Adding or editing a method requires changes in all three:
+- `src/common/method_registry.py` — canonical key, abbreviation, family, aliases
+- `src/pages/MethodDetails.py` — citation, URL, summary (in `method_meta` dict, keyed by abbreviation)
+- `src/data_store.py` — `DATASET_FILES` (optimization) or `UQ_BUDGET_FILES` (UQ) data path registration
+
+**2. Sidebar navigation is duplicated across five files.**
+`st.sidebar.page_link(...)` blocks are copy-pasted in:
+`streamlit_app.py`, `pages/OptimizationLeaderboard.py`, `pages/UQLeaderboard.py`,
+`pages/MethodDetails.py`, `pages/RawData.py`.
+Adding a new page means editing all five. (This is known tech debt.)
+
+**3. `_load_store` has filename-based special-casing.**
+`src/data_store.py` infers `algorithm_type` and `ensemble_size` from the **filename**
+when they are missing from the NetCDF. This only works for filenames containing `"abc"` or
+`"hm"`. Every other method must carry `algorithm_type` and `ensemble_size` as dimensions
+in the NetCDF itself, or you must add a new branch in `_load_store`.
+
+**4. `@st.cache_data` — clear the cache after data changes.**
+`load_metric_store()`, `load_uq_store()`, and `load_uq_budget_store()` are all cached.
+After registering a new file or changing `DATASET_FILES`/`UQ_BUDGET_FILES`, the user must
+clear the Streamlit cache (top-right menu → "Clear cache", or restart the app).
+
+**5. `git lfs pull` is required before the first run.**
+`data/*.nc` are Git LFS pointers (130 bytes) until pulled. The app will show no data if
+LFS files haven't been fetched.
+
+**6. Streamlit LaTeX rendering: `st.latex()` for display math, not `st.markdown()`.**
+Streamlit's KaTeX parser does not render multi-line `$...$` blocks inside `st.markdown`.
+For display-math equations, always use `st.latex(r"...")` (single-line raw string). For
+surrounding text with inline math, `st.markdown(r"$x$")` works fine. Mixing them:
+```python
+st.markdown(r"The condition is:")
+st.latex(r"\left|S(q) - q\right| \leq c\sqrt{\frac{q(1-q)}{N_y}}")
+st.markdown(r"where $c$ is the target-scaling level.")
+```
+
+---
+
+## Known antipatterns
+
+**`width="stretch"` is invalid Streamlit API.**
+`st.dataframe` and `st.altair_chart` accept `use_container_width=True` (bool) or
+`width=<int>` (pixels). Use `use_container_width=True` for full-width rendering.
+
+**Hardcoded RMSE target list in `MethodDetails.py`.**
+`target_options` must be derived from the loaded data:
+```python
+sorted(slice_df["rmse_target"].astype(str).unique().tolist())
+```
+A hardcoded list silently hides any target level present in the data but absent from the
+list (e.g. ABC has a `2.0` target).
+
+**`UQ_DATASET_FILES` is intentionally empty — do not add UQ files there.**
+UQ data is loaded via `UQ_BUDGET_FILES` and `load_uq_budget_store()`, not via
+`UQ_DATASET_FILES` / `_load_store()`. Adding a file to `UQ_DATASET_FILES` will have no
+effect on the UQ leaderboard because `load_uq_store()` derives from the budget store only.
+
+---
+
+## Smoke-test after any change
+
+```bash
+# From the repo root
+source .venv/bin/activate
+streamlit run src/streamlit_app.py
+```
+
+Then visit `http://localhost:8501` and confirm:
+- The new method (or updated display) appears on the relevant leaderboard.
+- `.cache/known_methods_snapshot.json` → `unmapped_observed_methods` is empty.
+- The Methods page (`📘 Methods`) shows the method with correct citation/summary.
+
+If data files haven't been LFS-pulled the app will be blank — run `git lfs pull` first.
+
+---
+
+## Final step
+
+After completing the task, offer the user a chance to improve the **`leaderboard-manager`**
+skill itself using skill-creator. For example:
+
+> "Would you like to improve the **leaderboard-manager** skill itself using skill-creator?
+> You can share suggestions, or I can analyse patterns from this session — edge cases that
+> came up, steps that were unclear, anything that felt awkward — to refine the skill for
+> next time."

.claude/skills/leaderboard-manager/references/add-method.md

@@ -0,0 +1,138 @@
+# Add a Method (+ its Data)
+
+Use this checklist whenever a new calibration algorithm needs to appear on the
+Optimization leaderboard. For the UQ leaderboard, complete this checklist first, then
+follow `move-between-boards.md`.
+
+---
+
+## Pre-flight: understand the incoming NetCDF
+
+Open the collaborator's file (e.g. from `From_Rob_6-10/`) and verify its structure:
+
+```python
+import xarray as xr
+ds = xr.open_dataset("path/to/file.nc")
+print(ds)          # check dimensions and variables
+print(ds["metric"])
+```
+
+The file must have:
+- A `metric` variable (float; shape over the dims below).
+- Dimensions: `algorithm_type`, `rmse_target`, `ensemble_size`, `random_seed`.
+  - `algorithm_type` is a string coordinate that names the algorithm.
+- All dim names must match exactly (case-sensitive).
+
+**If `algorithm_type` is missing from the NetCDF:**
+The loader (`src/data_store.py:129–137`) only infers it from the filename for `"abc"` and
+`"hm"` files. For any other method you must either:
+(a) add the `algorithm_type` dim to the NetCDF before placing it in `data/`, or
+(b) add a new `elif` branch in `_load_store()` analogous to the existing `abc`/`hm` block.
+
+**If `ensemble_size` is missing:**
+Same situation — currently only inferred for `"abc"` files (hardcoded to 1). Fix or add a
+branch.
+
+---
+
+## Step 1 — Register the method in `src/common/method_registry.py`
+
+Open `src/common/method_registry.py` and add an entry to `KNOWN_METHODS` (the dict starts
+at line 7). Follow the existing pattern exactly:
+
+```python
+"new-key": {
+    "abbreviation": "NEWABBR",          # Short uppercase display name
+    "Method": "Full Method Name",       # Human-readable name for tables
+    "family": "Kalman",                 # One of: Kalman | Bayesian | calibrate_then_emulate
+    "aliases": ["new-key", "alias2"],   # All raw strings that may appear in NetCDF algorithm_type
+},
+```
+
+**Canonical key** (`"new-key"`): lowercase, hyphen-separated. Must match what
+`canonicalize_method_name()` will resolve to from the NetCDF's `algorithm_type` strings.
+
+**Aliases**: include every spelling variant that could appear in the raw NetCDF, both
+lowercase and the exact case used. The `build_alias_lookup()` function (line 53) also
+registers `canonical.upper()` automatically, but doesn't handle mixed-case; be thorough.
+
+After saving, `ALIAS_TO_CANONICAL` is rebuilt at import time — no extra step needed.
+
+---
+
+## Step 2 — Add display metadata in `src/pages/MethodDetails.py`
+
+Open `src/pages/MethodDetails.py`. Find the `method_meta` dict (line 40) and add an entry
+keyed by the **abbreviation** (not the canonical key):
+
+```python
+"NEWABBR": {
+    "citation": "Author et al., Journal, Year",
+    "url": "https://doi.org/...",
+    "summary": "One-sentence description of what this algorithm does.",
+},
+```
+
+If you don't have the citation/URL yet, use placeholder strings and follow up with
+`populate-metadata.md`. The app will show "Citation pending" / a broken link if omitted,
+but won't crash.
+
+---
+
+## Step 3 — Place the NetCDF and register it in `src/data_store.py`
+
+1. **Copy the file** into `data/` (or a subdirectory like `data/bayesian/`). Use
+   `git lfs track "data/**/*.nc"` if it isn't already tracked by LFS.
+
+2. **Add the path** to `DATASET_FILES` in `src/data_store.py` (starts at line 29):
+
+```python
+DATASET_FILES = {
+    "L63": [
+        "l63_ensemble_results.nc",
+        ...
+        "my_subdir/new_method_l63.nc",   # ← add here
+    ],
+    ...
+}
+```
+
+Keys (`"L63"`, `"L96"`, etc.) are the benchmark labels. Add the file to every benchmark
+key it covers. Paths are relative to the `data/` directory.
+
+---
+
+## Step 4 — Verify
+
+Run the smoke-test from `SKILL.md`:
+
+```bash
+streamlit run src/streamlit_app.py
+```
+
+Then check:
+
+1. **Leaderboard table** (Home page): the new method's abbreviation appears as a row.
+2. **Methods page** (`📘 Methods`): the method is in the dropdown; citation/summary display.
+3. **Snapshot file**: open `.cache/known_methods_snapshot.json` after the app loads and
+   confirm:
+   - The new canonical key is in `known_methods`.
+   - It does **not** appear in `unmapped_observed_methods` (which lists any `algorithm_type`
+     strings from the data that didn't match any known alias).
+
+If the method appears in `unmapped_observed_methods`, the raw `algorithm_type` in the
+NetCDF doesn't match any alias you registered. Add the exact string (check
+`observed_methods` in the snapshot for the raw value) to the `aliases` list in
+`method_registry.py` and reload.
+
+---
+
+## Common failure modes
+
+| Symptom | Likely cause |
+|---|---|
+| Method missing from leaderboard table | `algorithm_type` in NetCDF doesn't match any alias in `KNOWN_METHODS` — check `unmapped_observed_methods` |
+| Method shown as uppercase canonical key (e.g. `NEW-KEY`) | `method_registry.py` entry added but `abbreviation` not picked up — check `get_method_meta()` path |
+| "Skipping … missing expected columns" warning | NetCDF lacks `algorithm_type` or `ensemble_size` dims and no special-case branch exists in `_load_store` |
+| Old data still shown after file change | Streamlit cache — clear it (top-right menu or restart) |
+| `data/*.nc` is 130 bytes | `git lfs pull` not run |

.claude/skills/leaderboard-manager/references/add-uq-data.md

@@ -0,0 +1,149 @@
+# Add UQ Ensemble-Results Data
+
+Use this reference when a collaborator provides new ensemble-results NetCDF files that
+should appear on the UQ leaderboard. This pipeline is entirely separate from the
+optimization leaderboard's `DATASET_FILES` / `_load_store()` path.
+
+---
+
+## How the UQ pipeline works
+
+The UQ leaderboard does not read a pre-computed `metric` variable. Instead it:
+
+1. Reads `output_coverage` from the NC file — the empirical marginal coverage fraction
+   `S(q)` at each iteration, ensemble size, and random seed.
+2. In `load_uq_budget_store()`, finds the minimum iteration k where
+   `|S(q) − q| ≤ c·√(q(1−q)/N_y)` holds for **all three** quantiles
+   `q ∈ [0.15, 0.5, 0.85]` simultaneously.
+3. Budget = N_ens · k_iter; NaN when the condition is never satisfied.
+4. Aggregates over seeds: `mean_budget`, `mean_iters`, `failure_count`, `failure_rate`.
+5. `load_uq_store()` calls `load_uq_budget_store()` and exposes `mean_budget` as the
+   leaderboard `metric`.
+
+---
+
+## Required NC file schema
+
+The file must carry:
+
+| Variable / coordinate | Dims / shape | Notes |
+|---|---|---|
+| `output_coverage` | `(coverage_quantile, k_iter, ensemble_size, random_seed)` | Empirical coverage fractions in [0, 1] |
+| `coverage_quantile` | `(n_cov_q,)` | Must include 0.15, 0.5, and 0.85 (exactly) |
+| `k_iter` | `(n_k,)` | Iteration indices (1-indexed integers) |
+| `ensemble_size` | `(n_ens,)` | Ensemble sizes (integers) |
+| `random_seed` | `(n_seeds,)` | Seed indices |
+| `target_scaling` | `(n_ts,)` | The c values, e.g. [1.0, 1.25, 1.5, 1.75, 2.0, 2.25, 2.5] |
+| `output_dim` | scalar or dimension | N_y — number of output dimensions |
+
+**Check the file structure before registering:**
+```python
+import xarray as xr
+ds = xr.open_dataset("data/ces-eki-dmc_results/my_file.nc")
+print(ds)
+print("coverage_quantile:", ds["coverage_quantile"].values)
+print("target_scaling:", ds["target_scaling"].values)
+print("output_dim:", ds.sizes.get("output_dim") or ds["output_dim"].values)
+```
+
+Confirm that 0.15, 0.5, and 0.85 are all present in `coverage_quantile`. If they are
+absent, the budget computation will silently use the nearest available quantile — check
+via `np.argmin(np.abs(cov_q - 0.15))` etc.
+
+---
+
+## Step 1 — Place the file in `data/`
+
+Place the NC file in `data/ces-eki-dmc_results/` (or an appropriate subdirectory).
+Ensure it is tracked by Git LFS:
+
+```bash
+git lfs track "data/**/*.nc"   # already set up; just confirm
+git add data/ces-eki-dmc_results/my_file.nc
+```
+
+---
+
+## Step 2 — Register in `UQ_BUDGET_FILES`
+
+Open `src/data_store.py` and add the file to `UQ_BUDGET_FILES`:
+
+```python
+UQ_BUDGET_FILES: dict[str, list[tuple[str, str]]] = {
+    "L63": [
+        ("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l63_ensemble_results_2026-06-15.nc"),
+    ],
+    "L96": [
+        ("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l96_ensemble_results_2026-06-15.nc"),
+        ("new-method", "new-method_results/new_method_l96.nc"),   # ← add here
+    ],
+    ...
+}
+```
+
+Each entry is a `(algorithm_type, relative_path)` tuple. `algorithm_type` must match an
+entry in `KNOWN_METHODS` (or its aliases list) — see `add-method.md` for how to register
+a new method.
+
+Paths are relative to the `data/` directory.
+
+---
+
+## Step 3 — Ensure the method is registered
+
+The `algorithm_type` string in the tuple must resolve through the method identity chain
+to a known canonical key. If this is a new method:
+
+1. Add an entry to `KNOWN_METHODS` in `src/common/method_registry.py` (see `add-method.md`).
+2. Add display metadata in `src/pages/MethodDetails.py` `method_meta` dict.
+
+If the method is already on the optimization leaderboard, its registry entry exists —
+no changes to `method_registry.py` or `MethodDetails.py` are needed.
+
+---
+
+## Step 4 — Clear cache and verify
+
+Clear the Streamlit cache (top-right menu → "Clear cache", or restart the app).
+
+Then check:
+
+1. **UQ Leaderboard** (`🎯 UQ Leaderboard`): the method appears in the ranked table at
+   target levels where coverage was achievable. At very tight tolerances (low c), all
+   seeds may fail — this is expected due to grid coarseness (N_y output dims means
+   coverage fractions are multiples of 1/N_y).
+
+2. **Iterations chart**: "Mean Iterations for Coverage vs Ensemble Size" should show a
+   line for the new method.
+
+3. **Failure rate chart**: "Failure Rate of Hitting Target ..." should show bars for
+   the new method. 100% failure at c=1.0 is often expected for small N_y.
+
+4. **Snapshot**: `.cache/known_methods_snapshot.json` → `unmapped_observed_methods`
+   should not include the new method's algorithm_type string.
+
+---
+
+## Failure-rate expectations by N_y
+
+With N_y output dimensions, coverage fractions are multiples of 1/N_y. The first c level
+that permits convergence depends on N_y:
+
+| N_y | First achievable c | Reason |
+|---|---|---|
+| 9 (L63) | ≈ 1.5 | tol(q=0.5) = 0.25; nearest coverage 3/9 = 0.333 satisfies |0.333−0.5| ≤ 0.25 |
+| 80 (L96) | ≈ 1.0 | finer grid; easier to satisfy all three quantiles |
+
+Expect 100% failure at the tightest c levels for small N_y. This is data behavior, not a
+pipeline bug.
+
+---
+
+## What NOT to do
+
+- **Do not add UQ files to `UQ_DATASET_FILES`** — that dict is intentionally empty. The
+  UQ leaderboard only reads from `UQ_BUDGET_FILES` via `load_uq_budget_store()`.
+- **Do not rename `rmse_target` to `uq_target`** — UQ files do not use a `uq_target`
+  dimension at all; the target-scaling levels come from the `target_scaling` coordinate.
+- **Do not put a `metric` variable in the UQ NC file** — `load_uq_budget_store()` ignores
+  it; the metric is computed from `output_coverage`.

.claude/skills/leaderboard-manager/references/assess.md

@@ -0,0 +1,84 @@
+# Assess for Staleness / Bugs
+
+Use this checklist when the user asks for an audit, bug check, or staleness review of the
+leaderboard dashboard. Read all relevant source files first (in parallel), then work
+through each item below and report findings grouped by severity.
+
+---
+
+## Files to read
+
+Read these in a single parallel batch before starting the checklist:
+
+- `src/common/method_registry.py` — `KNOWN_METHODS`, aliases
+- `src/common/leaderboard.py` — `render_leaderboard()`
+- `src/data_store.py` — `DATASET_FILES`, `UQ_DATASET_FILES`, `_load_store()`
+- `src/streamlit_app.py` — home page wiring
+- `src/pages/UQLeaderboard.py`
+- `src/pages/MethodDetails.py` — `method_meta` dict, target list
+- `src/pages/RawData.py`
+- `.cache/known_methods_snapshot.json` — observed vs known methods
+
+---
+
+## Checklist
+
+### API correctness
+
+- [ ] **`width="stretch"` in `st.dataframe` or `st.altair_chart`** — invalid parameter.
+  Should be `use_container_width=True`. Grep for `width="stretch"` across `src/`.
+
+### Data integrity
+
+- [ ] **Methods in `KNOWN_METHODS` with no entry in `DATASET_FILES`** — registered but
+  will never appear on any leaderboard (no data). Check each canonical key in
+  `KNOWN_METHODS` against the file paths listed in `DATASET_FILES`; confirm at least one
+  registered file would produce rows for that `algorithm_type`.
+- [ ] **`.cache/known_methods_snapshot.json` → `unmapped_observed_methods` not empty** —
+  means a NetCDF carries an `algorithm_type` string that has no entry in `KNOWN_METHODS`
+  (or its aliases list). Those rows silently fall through with a raw uppercase fallback
+  abbreviation and `family = "Kalman"` default.
+- [ ] **`DATASET_FILES` references a file path that doesn't exist on disk** — `_load_store`
+  emits a `warnings.warn` and skips it silently; run
+  `python -c "from src.data_store import load_metric_store; load_metric_store()"` and
+  watch for warnings, or just `ls` the registered paths.
+
+### Display correctness
+
+- [ ] **Hardcoded `target_options` in `MethodDetails.py`** — should be derived from data:
+  `sorted(slice_df["rmse_target"].astype(str).unique().tolist())`. A hardcoded list
+  silently hides any target present in the data but absent from the list.
+- [ ] **`method_meta` entries with placeholder citations or `example.com` URLs** —
+  grep for `example.com` and `placeholder` / `citation to be added` in `MethodDetails.py`.
+  See `references/populate-metadata.md` for confirmed citations.
+
+### Dead code
+
+- [ ] **Unused columns written to `merged` in `_load_store`** — e.g. any column created
+  but never read downstream. Check columns added after the `dropna` step in `data_store.py`.
+- [ ] **Session-state guards that are always `True`** — e.g.
+  `if st.session_state.get("show_home", True):` where nothing ever sets the key to `False`.
+  These wrap live code in an unreachable conditional.
+- [ ] **Dead functions** — any function in `data_store.py` or elsewhere that is defined but
+  never called. Check for `def build_*` or similar remnants of superseded pipelines.
+
+### Sidebar nav consistency
+
+- [ ] **All four files have identical sidebar `page_link` blocks** — the block is
+  copy-pasted in `streamlit_app.py`, `UQLeaderboard.py`, `MethodDetails.py`, `RawData.py`.
+  Confirm they are identical (same links, same icons, same order).
+
+---
+
+## Reporting format
+
+Group findings by severity:
+
+- **Bug (functional impact)** — something that causes incorrect output, a runtime error,
+  or silently hides data (e.g. hardcoded targets, invalid Streamlit API).
+- **Staleness** — registry entries, metadata, or cached snapshots that no longer match
+  current code or data (e.g. placeholder citations, phantom method entries).
+- **Dead code** — functions, columns, or conditionals that do nothing.
+- **Informational** — minor inconsistencies that don't affect correctness.
+
+For each finding, include: file, line number(s), description, and recommended fix.

.claude/skills/leaderboard-manager/references/manage-display.md

@@ -0,0 +1,218 @@
+# Manage Display — Table, Chart, and Scoring
+
+This file covers every display surface in the leaderboard dashboard and how to change it.
+The key architectural fact: **both the Optimization and UQ leaderboards share a single
+renderer** (`render_leaderboard()` in `src/common/leaderboard.py`). Any change to that
+function affects both boards simultaneously. Method-specific charts live in
+`src/pages/MethodDetails.py`.
+
+---
+
+## Leaderboard table
+
+### Column visibility and order
+
+The columns shown depend on whether "All" or a specific benchmark is selected. Both lists
+are defined in `src/common/leaderboard.py`:
+
+- **All-benchmarks view** (`table_column_order`, line 296–305):
+  ```python
+  ["Placement", "abbreviation", "Method", "family", "Score",
+   "Targets Used", "Ensemble Sizes Used", "Benchmarks Used"]
+  ```
+- **Single-benchmark view** (`table_column_order`, line 307–319):
+  ```python
+  ["Placement", "abbreviation", "Method", "family", "Score",
+   "Mean Forward Model Runs", "Minimum Forward Model Runs",
+   "Optimal Ensemble Size", "Targets Used", "Ensemble Sizes Used", "Benchmarks Used"]
+  ```
+
+To **show or hide** a column, add/remove it from the relevant list.
+To **reorder** columns, rearrange entries within the list.
+
+### Column labels and formatting
+
+Column display names and number formats are set in `st.dataframe(... column_config=...)`
+at lines 326–339. Each column has an entry like:
+
+```python
+"Mean Forward Model Runs": st.column_config.NumberColumn(
+    "Mean Forward Model Runs", format="%.4f"
+),
+"Score": st.column_config.ProgressColumn(
+    "Score (0–100)", min_value=0.0, max_value=100.0, format="%.1f"
+),
+```
+
+To **rename** a displayed column header, change the string argument (second positional arg)
+in the `column_config` entry. The dict key must still match the DataFrame column name.
+
+To **add a new column to the table**, you also need to produce it in `build_scored_table()`
+(line 111) — add it to the `.agg()` call or compute it after the groupby, then add a
+`column_config` entry and include it in `table_column_order`.
+
+### Adding `failure_rate` to the leaderboard table
+
+`failure_rate` is available in `metric_store` but not surfaced in the scored table.
+To add it:
+1. In `build_scored_table()`, add it to the `.groupby(...).agg(...)` call, e.g.:
+   `**{"Mean Failure Rate (%)": ("failure_rate", "mean")}`.
+2. Add a `column_config` entry (e.g. `st.column_config.NumberColumn(..., format="%.1f")`).
+3. Add `"Mean Failure Rate (%)"` to `table_column_order` where desired.
+
+---
+
+## Scoring modes
+
+The four scoring modes are defined in `scoring_options` (line 82):
+```python
+scoring_options = [
+    "Mean Forward Model Runs",
+    "Minimum Forward Model Runs",
+    "Smallest Optimal Ensemble Size",
+    "Custom Blend",
+]
+```
+
+Each mode maps to a sort key and a score column assignment in `build_scored_table()` at
+lines 185–203. To **add a new scoring mode**, add its name to `scoring_options` and add
+an `elif` branch in the scoring logic block that sets `scored_df["Score"]`,
+`sort_columns`, and `ascending`.
+
+The "Custom Blend" slider weight (lines 357–368) is keyed to the session-state key
+`k_weight` — no change needed when adding a non-blend mode.
+
+---
+
+## Leaderboard line chart
+
+The chart that appears in single-benchmark view ("Mean Forward Model Runs vs Ensemble
+Size") is at lines 387–396 (inside the `if selected_benchmark != "All":` guard at line
+376):
+
+```python
+ens_ticks = sorted(chart_df["ensemble_size"].unique().tolist())
+chart = (
+    alt.Chart(chart_df)
+    .mark_line(point=True)
+    .encode(
+        x=alt.X(
+            "ensemble_size:Q",
+            title="Ensemble Size",
+            axis=alt.Axis(values=ens_ticks, format="d"),   # integer ticks only
+        ),
+        y=alt.Y("mean_forward_runs:Q", title="Mean Forward Model Runs"),
+        color=alt.Color("abbreviation:N", title="Method"),
+        tooltip=["abbreviation", "ensemble_size",
+                 alt.Tooltip("mean_forward_runs:Q", format=".4f")],
+    )
+)
+st.altair_chart(chart, use_container_width=True)
+```
+
+The `axis=alt.Axis(values=ens_ticks, format="d")` pattern is important: without it,
+Altair may interpolate non-integer ticks (e.g. 5.5, 10.5) on the ensemble-size axis.
+Always derive `ens_ticks` from the actual data rather than hardcoding.
+
+Common changes:
+- **Log scale on Y**: add `scale=alt.Scale(type="log")` inside `alt.Y(...)`.
+- **Add failure rate as a second Y-axis or tooltip**: add `failure_rate` to `chart_df`
+  groupby and include it in `tooltip`.
+- **Change mark type** (e.g. bars): replace `mark_line(point=True)` with `mark_bar()`.
+- **Color palette**: add `.configure_range(category={"scheme": "tableau10"})` after the
+  chart construction.
+
+The chart only renders when a single benchmark is selected (`if selected_benchmark != "All":`
+guard at line 376).
+
+---
+
+## Method Details page surfaces
+
+The per-method page (`src/pages/MethodDetails.py`) has its own controls.
+
+### Target level radio
+
+Line 111:
+```python
+target_options = ["1.0", "1.1", "1.2"]
+selected_target = st.radio("RMSE Target Level", options=target_options, horizontal=True)
+```
+
+These are **hardcoded strings**. If the benchmark gains new target levels, update this
+list. A more robust approach is to derive them from the data:
+```python
+target_options = sorted(metric_store["rmse_target"].astype(str).unique().tolist())
+```
+
+### Per-method special-case charts
+
+The HM (History Matching) failure-analysis chart (lines 122–138) is the canonical
+example of a per-method detail chart:
+
+```python
+if sel == "HM":
+    st.markdown("### Failure Analysis")
+    chart = (alt.Chart(best_table_view).mark_bar().encode(...))
+    st.altair_chart(chart, use_container_width=True)
+```
+
+To **add a chart for another method**, add an `elif sel == "ABBR":` block immediately
+after the `if sel == "HM":` block, following the same pattern.
+
+To **generalize** (show failure rate for all methods when non-zero), replace the `if sel
+== "HM":` guard with a condition on the data, e.g.:
+```python
+if best_table_view["failure_rate"].max() > 0:
+    # render the failure chart for any method that has failures
+```
+
+### Scaling chart (ensemble size vs forward-model runs per benchmark)
+
+The `"Scaling by Benchmark"` chart is already generic across all methods. To modify
+encoding, change the `.encode()` arguments analogously to the leaderboard chart above.
+
+---
+
+## UQ-specific charts (`budget_store` parameter)
+
+The UQ leaderboard passes a `budget_store` DataFrame to `render_leaderboard()` via the
+`budget_store` keyword argument. This enables two additional chart sections rendered in
+`src/common/leaderboard.py` after the main performance chart:
+
+1. **"Mean Iterations for Coverage vs Ensemble Size"** — line chart of `mean_iters` vs
+   ensemble size. Uses the same integer-tick pattern as the main chart.
+
+2. **"Failure Rate of Hitting Target ..."** — grouped bar chart with `ensemble_size:O`
+   on the x-axis (ordinal, not quantitative) and `mean_failure_rate:Q` on y. The title
+   reflects the currently selected target level.
+
+To **add or modify a UQ chart**, edit the `if budget_store is not None` block in
+`src/common/leaderboard.py`. The `budget_store` DataFrame has columns: `benchmark`,
+`algorithm_type`, `abbreviation`, `family`, `uq_target`, `ensemble_size`, `mean_budget`,
+`mean_iters`, `failure_count`, `failure_rate`, `n_seeds`.
+
+**Bar chart integer x-axis:** Use `ensemble_size:O` (ordinal) with an explicit sort list
+and `axis=alt.Axis(labelAngle=0)` to suppress diagonal labels:
+```python
+ens_ticks_fail = sorted(failure_df["ensemble_size"].unique().tolist())
+x=alt.X(
+    "ensemble_size:O",
+    title="Ensemble Size",
+    sort=[str(e) for e in ens_ticks_fail],
+    axis=alt.Axis(labelAngle=0),
+)
+```
+
+---
+
+## Display change checklist
+
+1. Edit the relevant file (`leaderboard.py` for both leaderboards; `MethodDetails.py` for
+   per-method surfaces).
+2. If adding a column: produce it in the aggregation AND add a `column_config` entry AND
+   add it to `table_column_order`.
+3. Restart or clear Streamlit cache to pick up Python changes (code changes take effect on
+   restart; cache only matters for data changes).
+4. Smoke-test on both the "All" view and a single-benchmark view to catch the different
+   `table_column_order` branches.

.claude/skills/leaderboard-manager/references/move-between-boards.md

@@ -0,0 +1,116 @@
+# Move or Duplicate a Method Between Leaderboards
+
+This file covers wiring a method's results onto the UQ leaderboard, or removing it from
+one board while keeping it on the other. The key insight is that **the method registry
+and the MethodDetails metadata are shared** — you never need to touch `method_registry.py`
+or `MethodDetails.py` for this task. It is purely a **data registration** operation.
+
+---
+
+## How the two boards relate
+
+```
+method_registry.py          ← shared; controls identity on BOTH boards
+pages/MethodDetails.py       ← shared; controls the Methods page for BOTH boards
+data_store.py
+  DATASET_FILES              ← Optimization board only (rmse_target, metric variable)
+  UQ_BUDGET_FILES            ← UQ board only (output_coverage variable, budget pipeline)
+```
+
+A method is "on" a leaderboard if and only if a NetCDF containing its `algorithm_type`
+is registered in the corresponding data dict.
+
+**Critical difference:** the two boards use entirely different NetCDF schemas and loading
+pipelines. Do not try to use the same NC file for both boards.
+
+---
+
+## UQ vs optimization NC schemas
+
+| Aspect | Optimization (`DATASET_FILES`) | UQ (`UQ_BUDGET_FILES`) |
+|---|---|---|
+| Key variable | `metric` (pre-computed runs) | `output_coverage` (coverage fractions) |
+| Target coord | `rmse_target` dimension | `target_scaling` coordinate (not a dim) |
+| Failure encoding | `metric == -1` | NaN (never reached) |
+| Loader | `_load_store()` | `load_uq_budget_store()` computes budget on the fly |
+
+For the full UQ NC schema, see `references/add-uq-data.md`.
+
+---
+
+## Duplicate a method (appear on both boards)
+
+"Duplicate" means the method shows up on the Optimization **and** UQ leaderboards.
+
+1. Obtain (or produce) a UQ-schema NC file for the method (must have `output_coverage`,
+   `coverage_quantile`, `k_iter`, `ensemble_size`, `random_seed`, `target_scaling`,
+   `output_dim` — see `references/add-uq-data.md` for the full spec).
+2. Place it in `data/` (e.g. `data/method_results/method_l63_ensemble_results.nc`).
+3. Add the path to `UQ_BUDGET_FILES` in `src/data_store.py`:
+
+```python
+UQ_BUDGET_FILES: dict[str, list[tuple[str, str]]] = {
+    "L63": [
+        ("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l63_ensemble_results_2026-06-15.nc"),
+        ("new-method", "new-method_results/new_method_l63.nc"),  # ← add here
+    ],
+    ...
+}
+```
+
+The method's optimization results in `DATASET_FILES` remain untouched.
+No changes to `method_registry.py`, `MethodDetails.py`, or `UQLeaderboard.py` are needed.
+
+**Clear the Streamlit cache** after editing `data_store.py` — all three loaders are cached.
+
+---
+
+## Move a method (UQ only, remove from optimization)
+
+"Move" means the method should appear on the UQ leaderboard **and not** the Optimization
+leaderboard.
+
+1. Complete the duplicate steps above (register UQ NC in `UQ_BUDGET_FILES`).
+2. Remove the method's path(s) from `DATASET_FILES` in `src/data_store.py`.
+
+The registry entry and `method_meta` in `MethodDetails.py` stay — the Methods page still
+shows the method, but performance data comes only from `load_uq_store()`. The method won't
+appear in the optimization leaderboard table.
+
+---
+
+## Populate the UQ leaderboard for a new benchmark (first time)
+
+When a benchmark key is absent from `UQ_BUDGET_FILES`, that benchmark has no UQ data.
+Add the key and the NC file tuple — see the "Duplicate" section above. The UQ page calls
+`render_leaderboard(load_uq_store(), ...)` which automatically includes all registered
+benchmarks; no page-level changes needed.
+
+---
+
+## Verify
+
+After editing `UQ_BUDGET_FILES` and clearing the cache:
+
+1. Visit the **UQ Leaderboard** page (`🎯 UQ Leaderboard` in the sidebar).
+2. Confirm the method appears in the ranked table for the benchmark you registered.
+3. Check `.cache/known_methods_snapshot.json` → `unmapped_observed_methods` is empty for
+   the new method.
+4. Expect 100% failure at tight c values (e.g. c=1.0) for benchmarks with small N_y —
+   see `references/add-uq-data.md` for the coverage-coarseness explanation.
+
+If the UQ leaderboard still shows "No metric data found" after clearing cache, confirm:
+- The NC path in `UQ_BUDGET_FILES` is correct relative to `data/`.
+- The file has been `git lfs pull`-ed (not a 130-byte pointer stub).
+- `output_coverage` is present in the file and the `coverage_quantile` coordinate
+  includes 0.15, 0.5, and 0.85.
+
+---
+
+## What does NOT need changing
+
+- `src/common/leaderboard.py` — fully generic; handles both leaderboards via parameters.
+- `src/pages/UQLeaderboard.py` — passes `budget_store=load_uq_budget_store()` already.
+- `src/common/method_registry.py` — no change if the method is already registered.
+- `src/pages/MethodDetails.py` — no change; the Methods page is shared.
+- The five sidebar nav blocks — no new pages are being added.

.claude/skills/leaderboard-manager/references/populate-metadata.md

@@ -0,0 +1,146 @@
+# Populate Method and Data Metadata
+
+This file covers filling in or correcting text that appears about a method in the
+dashboard — citations, reference URLs, and summary descriptions — as well as understanding
+where benchmark (dataset) labels come from.
+
+---
+
+## EKP documentation — fetch warning
+
+The canonical citation source for Kalman-family methods is:
+`https://clima.github.io/EnsembleKalmanProcesses.jl/dev/`
+
+**This URL frequently times out in WebFetch.** If it does, fall back to the confirmed
+citations below rather than retrying repeatedly.
+
+### Confirmed citations (verified in prior sessions)
+
+| Abbreviation | Citation | DOI / URL |
+|---|---|---|
+| TEKI | Chada et al., SIAM/ASA J. UQ, 2020 | https://doi.org/10.1137/17M114402X |
+| IEKF | Iglesias, Inverse Problems, 2016 | https://doi.org/10.1088/0266-5611/32/2/025002 |
+| CES-EKI-DMC | Cleary et al., J. Comput. Phys., 2021 | https://doi.org/10.1016/j.jcp.2020.109716 |
+| ETKI | Schillings & Stuart, Numer. Math., 2017 | link to EKP docs (sub-page DOI unconfirmed) |
+| UKI | Huang, Huang & Stuart, Physica D, 2022 | link to EKP docs (sub-page DOI unconfirmed) |
+| HM | Williamson et al. 2013; King et al. 2025 | — |
+
+Use `https://clima.github.io/EnsembleKalmanProcesses.jl/dev/` as the URL for ETKI and UKI
+until their sub-page DOIs are confirmed.
+
+---
+
+---
+
+## The metadata split
+
+Method information lives in **two separate places** that serve different purposes:
+
+| File | Key used | What it stores |
+|---|---|---|
+| `src/common/method_registry.py` | canonical key (e.g. `"uki"`) | Abbreviation, full method name, family, aliases — used for **data processing and identity** |
+| `src/pages/MethodDetails.py` `method_meta` dict | abbreviation (e.g. `"UKI"`) | Citation, URL, summary — used for **display copy only** |
+
+Both must be kept in sync. When you edit one, check the other. The most common mistake is
+adding a method to `method_registry.py` without adding a corresponding `method_meta`
+entry — the app won't crash, but the Methods page will show "Citation pending" and a
+broken link.
+
+---
+
+## Editing method display copy
+
+Open `src/pages/MethodDetails.py`. The `method_meta` dict starts at line 40:
+
+```python
+method_meta = {
+    "TEKI": {
+        "citation": "Chada et al., SIAM/ASA J. UQ, 2020",
+        "url": "https://doi.org/10.1137/17M114402X",
+        "summary": "EKI variant with Tikhonov regularization for stability.",
+    },
+    ...
+}
+```
+
+**To update a citation or URL**, find the entry by abbreviation and edit the values.
+**To add a missing entry**, add a new key matching the `abbreviation` in `method_registry.py`.
+
+The three fields:
+- `citation` — free-form string; typically "Author et al., Journal, Year" or a short title.
+- `url` — a full `https://...` URL. Displayed as an "Open reference" button.
+- `summary` — one or two sentences. Keep it short enough to fit in a narrow column.
+
+If you don't have a value yet, use `"(citation pending)"` / `"https://example.com"` /
+`"(summary pending)"` as placeholders so the app renders gracefully.
+
+---
+
+## Remaining placeholder backlog
+
+Most entries have been filled in. The following still need real values:
+
+| Abbreviation | Field | Status |
+|---|---|---|
+| `ABC` | url | `"https://example.com/abc"` — no canonical URL identified yet |
+| `HM` | url | `"https://example.com/hm"` — no canonical URL identified yet |
+| `ETKI` | url | Points to EKP docs index — sub-page DOI not yet confirmed |
+| `UKI` | url | Points to EKP docs index — sub-page DOI not yet confirmed |
+
+When filling in `ABC` or `HM` URLs, check with the collaborator for the preferred
+reference (History Matching has many implementations; the right URL depends on which
+variant is used here).
+
+---
+
+## Editing method identity / naming
+
+If you need to correct the full method name, family classification, or abbreviation (the
+short label that appears in table columns), edit `KNOWN_METHODS` in
+`src/common/method_registry.py`:
+
+```python
+"uki": {
+    "abbreviation": "UKI",           # Short label in table columns
+    "Method": "Unscented Kalman Inversion",  # Full name in Method column
+    "family": "Kalman",              # Kalman | Bayesian | calibrate_then_emulate
+    "aliases": ["uki"],              # Raw algorithm_type strings from NetCDF
+},
+```
+
+**If you change `abbreviation`**: also update the corresponding key in `method_meta` in
+`MethodDetails.py` (the keys must match). The app looks up `method_meta[abbreviation]`.
+
+**If you change `family`**: the "Family" column in the leaderboard table updates
+automatically — no other change needed.
+
+**If you add an alias**: new alias strings start matching immediately after reload. Use
+this when a collaborator's NetCDF uses a variant spelling not currently in the alias list.
+
+---
+
+## Benchmark / dataset labels
+
+Benchmark names (`L63`, `L96`, `L96_NN_FORCING`, `L96_SPATIAL_FORCING`) come from the
+keys of `DATASET_FILES` in `src/data_store.py`. They appear:
+- In the "Benchmark" selectbox on both leaderboards.
+- In the `benchmark` column of every row in `metric_store`.
+- As chart axis labels on the Methods detail page ("Scaling by Benchmark").
+
+To **rename a benchmark label**, change its key in `DATASET_FILES` (and `UQ_DATASET_FILES`
+if present). The label propagates everywhere automatically because it's read from the data
+at runtime. There is no separate label registry.
+
+To **add a new benchmark**, see `add-method.md` → Step 3 (the same pattern applies even
+without a new method — just add a new benchmark key with its file list).
+
+---
+
+## Verify
+
+After editing metadata:
+1. Restart the app (code changes don't require cache clearing, only data changes do).
+2. Open the `📘 Methods` page and select the edited method.
+3. Confirm the citation text, URL button target, and summary box show the new values.
+4. Check no `method_meta` key is misspelled (the abbreviation in `method_meta` must match
+   `KNOWN_METHODS[...]["abbreviation"]` exactly, including capitalisation).

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 *.nc filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -1,3 +1,34 @@
-*.pyc
-.venv
-__pycache__
+# Emacs
+*~
+\#*\#
+.\#*
+*.elc
+
+# Python
+*.py[cod]
+*.so
+__pycache__/
+*.egg-info/
+.eggs/
+dist/
+build/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# Streamlit
+.streamlit/secrets.toml
+
+# App cache
+.cache/
+
+# Jupyter
+.ipynb_checkpoints/
+
+# OS
+.DS_Store

CLAUDE.md

@@ -0,0 +1,185 @@
+# Calibration Benchmark
+
+## Project
+Streamlit dashboard hosting a **calibration / UQ leaderboard**. Compares parameter-calibration
+methods on Lorenz-63/96 dynamical-system benchmarks by ranking them on *forward-model run*
+efficiency (cost to hit a target across ensemble sizes). Two leaderboards:
+
+- **Optimization leaderboard** (home page) — race to reach an RMSE target.
+- **UQ leaderboard** (`pages/UQLeaderboard.py`) — budget-for-coverage metric: N_ens·k_iter to
+  satisfy a marginal-coverage condition at quantiles [0.15, 0.5, 0.85] for a given tolerance
+  scaling c. Data comes from ensemble-results NetCDF files registered in `UQ_BUDGET_FILES`.
+
+Deployed as a Hugging Face Space via Docker (port 8501).
+
+**Key metric (optimization):** `metric` = mean forward-model runs to reach an RMSE target
+(lower is better). `metric == -1` encodes a failed run.
+
+**Key metric (UQ):** `metric` = mean budget N_ens·k_iter to reach marginal-coverage
+tolerance `c·√(q(1−q)/N_y)` for all q ∈ [0.15, 0.5, 0.85] simultaneously (lower = better).
+`uq_target` = the scaling c ∈ {1.0, 1.25, 1.5, 1.75, 2.0, 2.25, 2.5}. NaN encodes failure
+(target never reached within the k_iter range).
+
+## Quick start
+
+```sh
+# First-time setup
+python3.11 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+git lfs pull          # required — data/*.nc are LFS pointers; app shows nothing without this
+
+# Run
+streamlit run src/streamlit_app.py
+```
+
+Docker:
+```sh
+docker build -t calibration-benchmark .
+docker run -p 8501:8501 calibration-benchmark    # → http://localhost:8501
+```
+
+## File map
+
+```
+calibration_benchmark/
+├── src/
+│   ├── streamlit_app.py          Entrypoint + Home/"Optimization Leaderboard"
+│   ├── data_store.py             load_metric_store() / load_uq_store() / load_uq_budget_store()
+│   │                             → @st.cache_data DataFrames; DATASET_FILES, UQ_BUDGET_FILES
+│   ├── common/
+│   │   ├── leaderboard.py       render_leaderboard() — shared scored-table + chart pipeline
+│   │   │                        optional budget_store param adds budget/iters chart section
+│   │   └── method_registry.py   KNOWN_METHODS registry + name canonicalization
+│   └── pages/
+│       ├── UQLeaderboard.py     🎯 UQ Leaderboard — calls render_leaderboard(load_uq_store())
+│       ├── MethodDetails.py     📘 Methods — per-method charts + metadata
+│       └── RawData.py           🧾 Get Data — raw table + CSV export
+├── data/                        NetCDF result files (git-LFS — run `git lfs pull`)
+│   ├── *.nc                     Kalman results (TEKI, ETKI, IEKF)
+│   ├── bayesian/                ABC + HM results
+│   ├── UKI_results/             UKI results
+│   └── ces-eki-dmc_results/     CES-EKI-DMC ensemble-results files (UQ source)
+│       ├── ces-eki-dmc_l63_ensemble_results_2026-06-15.nc
+│       ├── ces-eki-dmc_l96_ensemble_results_2026-06-15.nc
+│       ├── ces-eki-dmc_l96_nn_forcing_ensemble_results_2026-06-15.nc
+│       └── ces-eki-dmc_l96_spatial_forcing_ensemble_results_2026-06-15.nc
+├── examples/                    lorenz_demo.ipynb + lorenz.py (reference, not app code)
+├── From_Rob_6-10/               Raw collaborator data drop — NOT read by the app
+├── .cache/                      known_methods_snapshot.json (auto-generated, safe to delete)
+├── .streamlit/config.toml       showSidebarNavigation = false
+├── Dockerfile                   python:3.13.5-slim; ENTRYPOINT streamlit run src/...
+├── requirements.txt             altair, pandas, streamlit, xarray, netcdf4, h5netcdf, jax, gpjax
+├── test_parse.py                Ad-hoc NetCDF parse check (not a test suite)
+└── test_failures.py             Ad-hoc failure-count check (not a test suite)
+```
+
+## Data model
+
+### Optimization store (`load_metric_store()`)
+
+| Field | Meaning |
+|---|---|
+| `benchmark` | `L63`, `L96`, `L96_NN_FORCING`, `L96_SPATIAL_FORCING` |
+| `algorithm_type` | Canonical method key (e.g. `teki`) |
+| `algorithm_alias` | Normalized raw name from NetCDF |
+| `abbreviation` | Display name (e.g. `TEKI`) |
+| `family` | `Kalman` · `Bayesian` · `calibrate_then_emulate` |
+| `rmse_target` | Target RMSE level |
+| `ensemble_size` | Ensemble / particle count |
+| `metric` | Mean forward-model runs over `random_seed` (lower = better; −1 = failed) |
+| `failure_rate` | % seeds that failed (`metric == -1`) |
+
+`EXPECTED_DIMS = ("algorithm_type", "rmse_target", "ensemble_size", "random_seed")`
+
+### UQ store (`load_uq_store()`)
+
+Derived from `load_uq_budget_store()` — same schema as the optimization store but with
+`uq_target` in place of `rmse_target`, and `metric` = mean budget (N_ens·k_iter). Failure
+rate = % of seeds that never satisfied the coverage condition (NaN metric, not −1).
+
+### Budget store (`load_uq_budget_store()`)
+
+| Field | Meaning |
+|---|---|
+| `benchmark` | `L63`, `L96`, `L96_NN_FORCING`, `L96_SPATIAL_FORCING` |
+| `algorithm_type` | Canonical method key |
+| `abbreviation` / `Method` / `family` | From method registry |
+| `uq_target` | Target-scaling level c |
+| `ensemble_size` | Ensemble size N |
+| `mean_budget` | Mean N·k_iter over converged seeds (NaN if all failed) |
+| `mean_iters` | Mean k_iter over converged seeds |
+| `failure_count` | # seeds that never reached the target |
+| `failure_rate` | % seeds that never reached the target |
+| `n_seeds` | Total seeds (20) |
+
+Source NC files are registered in `UQ_BUDGET_FILES` in `src/data_store.py`. Each file must
+carry `output_coverage (coverage_quantile, k_iter, ensemble_size, random_seed)`,
+`target_scaling`, and `output_dim`.
+
+`UQ_COVERAGE_QUANTILES = [0.15, 0.5, 0.85]` — the three quantile levels used for the
+all-quantiles-satisfied condition.
+
+All three stores are `@st.cache_data` — clear Streamlit cache to pick up new data files.
+
+## Method registry  (`src/common/method_registry.py`)
+
+| Canonical | Abbreviation | Family | Aliases |
+|---|---|---|---|
+| `teki` | TEKI | Kalman | — |
+| `etki` | ETKI | Kalman | — |
+| `iekf` | IEKF | Kalman | gnsl, gnki |
+| `uki` | UKI | Kalman | — |
+| `abc` | ABC | Bayesian | — |
+| `hm` | HM | Bayesian | — |
+| `ces-eki-dmc` | CES-EKI-DMC | calibrate_then_emulate | — |
+
+## How to extend
+
+### Add a method
+1. Add entry to `KNOWN_METHODS` in `src/common/method_registry.py`
+   (keys: `abbreviation`, `Method`, `family`, `aliases`).
+2. Add detail-page metadata to the inline `method_meta` dict in `src/pages/MethodDetails.py`
+   keyed by **abbreviation** (e.g. `"TEKI"`). ⚠️ Metadata is split across both files.
+3. Place result NetCDF(s) in `data/` and add paths to `DATASET_FILES` in `src/data_store.py`.
+
+### Add an optimization benchmark dataset
+Add a `"BENCHMARK_NAME": [list_of_nc_paths]` entry to `DATASET_FILES` in `src/data_store.py`.
+Paths are relative to the `data/` directory.
+
+### Add UQ data
+Add an `(algorithm_type, nc_path)` tuple to the appropriate benchmark key in `UQ_BUDGET_FILES`
+in `src/data_store.py`. The NC file must carry:
+- `output_coverage` with dims `(coverage_quantile, k_iter, ensemble_size, random_seed)`
+- coordinates `coverage_quantile`, `k_iter`, `ensemble_size`, `target_scaling`
+- dimension `output_dim`
+
+`load_uq_budget_store()` and `load_uq_store()` will pick it up automatically after a
+Streamlit cache clear. No changes to `UQ_DATASET_FILES` (kept empty) are needed.
+
+The budget/iters chart and coverage-failure bar chart in the UQ leaderboard are rendered via
+the `budget_store` parameter of `render_leaderboard()` in `src/common/leaderboard.py`.
+
+### Add a page
+1. Create `src/pages/NewPage.py` — copy the sidebar nav block + `data_store` import shim
+   from an existing page (e.g. `RawData.py`).
+2. Add `st.sidebar.page_link("pages/NewPage.py", ...)` to **all four** existing files
+   (`streamlit_app.py`, `UQLeaderboard.py`, `MethodDetails.py`, `RawData.py`). Sidebar nav
+   is duplicated — there is no central nav module.
+
+## Known gotchas
+- **`git lfs pull` is mandatory** before first run — without it `data/*.nc` are 130-byte stubs.
+- **Sidebar nav is not centralized** — adding/renaming pages requires editing four files
+  (`streamlit_app.py`, `UQLeaderboard.py`, `MethodDetails.py`, `RawData.py`).
+- **Method metadata is split** — `method_registry.py` (scoring) and `pages/MethodDetails.py`
+  `method_meta` dict (display copy) must both be updated when adding a new method.
+- **`build_leaderboard()` in `data_store.py` is dead code** — both leaderboards are rendered
+  via `render_leaderboard()` in `src/common/leaderboard.py`.
+- **No CI / no formal test suite** — `test_parse.py` and `test_failures.py` are manual scripts.
+- **Cross-page linking** uses `st.query_params["method"]` (URL param = abbreviation).
+- **`UQ_DATASET_FILES` is intentionally empty** — UQ data is loaded via `UQ_BUDGET_FILES`
+  and the `load_uq_budget_store()` pipeline, not via `_load_store()`.
+- **UQ failure ≠ optimization failure** — in the UQ store, NaN metric means coverage target
+  never reached (no `metric == -1` sentinel); `failure_rate` is computed differently.
+- **`render_leaderboard` budget_store param** — must be passed explicitly from
+  `UQLeaderboard.py`; `leaderboard.py` is not aware of UQ-specific data paths.

README.md

@@ -17,3 +17,123 @@ Edit `/src/streamlit_app.py` to customize this app to your heart's desire. :hear
 
 If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
 forums](https://discuss.streamlit.io).
+
+## Path to running the app locally (without docker)
+Install the huggingface authentication (via  `hf`) add security key
+
+Use `python (e.g., 3.11)`
+
+Then call
+```
+python3.11 -m venv .venv
+pip install -r requirements.txt
+```
+
+As the data is just a pointer to the repository, install, and pull the large files
+```
+sudo apt install git-lfs
+git lfs pull
+```
+then run the streamlit app
+```
+streamlit run src/streamlit_app.py
+```
+
+## Via docker (untested)
+
+install docker
+```
+sudo snap install docker
+docker build -t calibration-benchmark .
+docker run -p 8501:8501 calibration-benchmark
+```
+Open
+```
+http://localhost:8501
+```
+
+## Adding new data files (git LFS)
+
+All `*.nc` files in `data/` are tracked via git LFS (configured in `.gitattributes`).
+To add a new result file:
+
+**Step 1 — Ensure git LFS is installed and initialised**
+```
+sudo apt install git-lfs   # or: brew install git-lfs
+git lfs install
+```
+
+**Step 2 — Copy the file into `data/`**
+```
+cp /path/to/your_results.nc data/your_results.nc
+```
+Sub-directories are fine (`data/UKI_results/`, `data/bayesian/`, etc.).
+
+**Step 3 — Stage and commit — LFS handles the rest**
+```
+git add data/your_results.nc
+git commit -m "Add <method> results for <benchmark>"
+```
+Git LFS intercepts the add automatically because `.gitattributes` already contains
+`*.nc filter=lfs diff=lfs merge=lfs -text`.  You can verify the file is tracked with:
+```
+git lfs ls-files | grep your_results.nc
+```
+
+**Step 4 — Register the file in the app**
+
+Open `src/data_store.py` and add the path to `DATASET_FILES` (optimization) or
+`UQ_DATASET_FILES` (UQ leaderboard):
+```python
+DATASET_FILES = {
+    "L96": [
+        "existing_file.nc",
+        "your_results.nc",   # <-- add here
+    ],
+    ...
+}
+```
+Paths are relative to the `data/` directory.
+
+**Step 5 — Push**
+```
+git push
+```
+LFS objects are pushed to the LFS store automatically alongside the pointer commit.
+Collaborators get the data with `git lfs pull` after cloning or fetching.
+
+---
+
+## Contributing via a Hugging Face pull request
+
+Hugging Face Spaces uses a non-standard PR workflow: you create the PR online first, then push to the branch it creates.
+
+**Step 1 — Clone the repo (if you haven't already)**
+```
+git clone https://huggingface.co/spaces/<owner>/<space-name>
+cd <space-name>
+```
+
+**Step 2 — Create and work on a local branch**
+```
+git checkout -b <local-branch-name>
+```
+Make your changes, then commit them:
+```
+git add <changed-files>
+git commit -m "Your commit message"
+```
+
+**Step 3 — Open the pull request online**
+
+Go to the Space on huggingface.co → **Community** tab → **New Pull Request**.
+Fill in the title and description and submit. Hugging Face will create a remote PR branch
+(shown on the PR page, e.g. `refs/pr/1` or a named branch).
+
+**Step 4 — Push your local branch to the PR branch**
+```
+git push --set-upstream origin <local-branch-name>:<pr-branch-name>
+```
+Replace `<pr-branch-name>` with the branch name shown on the HF PR page (often `refs/pr/N`).
+
+The PR page will update automatically once the push lands.

data/ces-eki-dmc_results/ces-eki-dmc_l63_ensemble_results_2026-06-15_minimal.nc

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea0c583aa3b7ffd39d8b00ab476b4fe7fcf3b15b54e0e15377cbc2245385d8e7
+size 287190

data/ces-eki-dmc_results/ces-eki-dmc_l96_ensemble_results_2026-06-15_minimal.nc

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eda2163d7dfc7e4a851aea5d6055b1f6cc1e1c4ec8496cf4072679739101fd39
+size 123030

data/ces-eki-dmc_results/ces-eki-dmc_l96_nn_forcing_ensemble_results_2026-06-15_minimal.nc

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d36761e0de6abb50ddd3054d2adf6100a6e730ba9ea8e2240ed7dd6a2c970892
+size 150390

data/ces-eki-dmc_results/ces-eki-dmc_l96_spatial_forcing_ensemble_results_2026-06-15_minimal.nc

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a402f8dc6d39b34bb1243e878bb34505846fe176a7e21a1024a8c1727d55dc1
+size 177750

requirements.txt

@@ -1,3 +1,4 @@
+hf
 altair
 pandas
 streamlit
@@ -5,4 +6,6 @@ jax
 gpjax
 matplotlib
 xarray
-netCDF4
+netcdf4
+h5netcdf
+h5py

src/common/leaderboard.py

@@ -0,0 +1,660 @@
+"""Shared leaderboard renderer for the Calibration Benchmark dashboard.
+
+Extracts the scored-table pipeline from the home page so both the Optimization
+leaderboard (home) and the UQ leaderboard (pages/UQLeaderboard.py) can reuse
+it without duplicating code.
+
+Usage::
+
+    from common.leaderboard import render_leaderboard
+
+    render_leaderboard(
+        metric_store,
+        target_col="rmse_target",
+        target_label="RMSE Target Level",
+        title="Optimization Leaderboard",
+        state_prefix="opt",
+        raw_page="pages/RawData.py",
+    )
+"""
+from __future__ import annotations
+
+import altair as alt
+import pandas as pd
+import streamlit as st
+
+# Stable categorical palette (Vega tableau10). Methods are assigned colors by their
+# sorted position in the full available-method list, so toggling visibility never
+# reassigns colors to other methods.
+_METHOD_PALETTE = [
+    "#4c78a8", "#f58518", "#e45756", "#72b7b2", "#54a24b",
+    "#eeca3b", "#b279a2", "#ff9da6", "#9d755d", "#bab0ac",
+]
+
+
+def render_leaderboard(
+    metric_store: pd.DataFrame,
+    *,
+    target_col: str,
+    target_label: str,
+    title: str,
+    state_prefix: str,
+    raw_page: str | None = None,
+    show_failure_panel: bool = False,
+    show_scoring_modes: bool = True,
+    canonical_target_levels: list[float] | None = None,
+    budget_store: pd.DataFrame | None = None,
+) -> None:
+    """Render a scored leaderboard backed by *metric_store*.
+
+    Parameters
+    ----------
+    metric_store:
+        DataFrame produced by ``load_metric_store()`` or ``load_uq_store()``.
+        Must contain at least the columns ``benchmark``, ``algorithm_type``,
+        ``abbreviation``, ``Method``, ``family``, ``ensemble_size``, ``metric``,
+        ``failure_rate``, and *target_col*.
+    target_col:
+        Name of the target-coordinate column, e.g. ``"rmse_target"`` or
+        ``"uq_target"``.
+    target_label:
+        Human-readable label for the target-level radio control,
+        e.g. ``"RMSE Target Level"`` or ``"UQ Target Level"``.
+    title:
+        Leaderboard section header text.
+    state_prefix:
+        Short string used to namespace ``st.session_state`` keys so multiple
+        leaderboard pages keep independent control state.  Use ``"opt"`` for
+        the Optimization leaderboard and ``"uq"`` for the UQ leaderboard.
+    raw_page:
+        Optional Streamlit page path for an "Open Raw Data" link shown at the
+        bottom.  Pass ``None`` to suppress the link.
+    show_failure_panel:
+        If ``True``, render a grouped-bar failure-rate chart below the main
+        performance chart.
+    canonical_target_levels:
+        If provided, the target-level selector always offers exactly these
+        values (as strings) regardless of what is present in the data.  Use
+        this to pin the UQ leaderboard to its fixed set of target-scaling
+        levels even when some have 100 % failure.
+    budget_store:
+        Optional DataFrame produced by ``load_uq_budget_store()``.  When
+        provided, an additional "Budget and Iterations for Coverage" section is
+        rendered below the main performance chart, showing mean budget
+        (N_ens·k_iter, solid lines) and mean iterations (k_iter, dashed lines)
+        vs ensemble size, followed by a coverage-failure-rate bar chart.
+    """
+    st.header(title)
+
+    if metric_store.empty:
+        st.warning("No metric data found. Expected NetCDF files in `data/` with a `metric` variable.")
+        return
+
+    # Derived column name for the string version of the target coordinate
+    target_str_col = f"{target_col}_str"
+
+    benchmark_values = sorted(metric_store["benchmark"].unique().tolist())
+    benchmark_options = ["All"] + benchmark_values
+    selected_benchmark = st.selectbox("Benchmark", options=benchmark_options, index=0)
+
+    filtered = (
+        metric_store.copy()
+        if selected_benchmark == "All"
+        else metric_store[metric_store["benchmark"] == selected_benchmark].copy()
+    )
+    filtered[target_str_col] = filtered[target_col].astype(str)
+
+    if canonical_target_levels is not None:
+        target_options = ["All targets"] + [str(float(t)) for t in canonical_target_levels]
+    else:
+        target_options = ["All targets"] + sorted(
+            metric_store[target_col].astype(str).unique().tolist()
+        )
+
+    scoring_options = [
+        "Mean Forward Model Runs",
+        "Minimum Forward Model Runs",
+        "Smallest Optimal Ensemble Size",
+        "Custom Blend",
+    ]
+
+    # Session-state keys namespaced by state_prefix so two leaderboard pages
+    # don't share control state.
+    k_target = f"{state_prefix}_selected_target"
+    k_scoring = f"{state_prefix}_scoring_mode"
+    k_weight = f"{state_prefix}_fwdruns_weight_percent"
+    k_methods = f"{state_prefix}_selected_methods"
+
+    current_target = st.session_state.get(k_target, "All targets")
+    if current_target not in target_options:
+        current_target = target_options[0]
+
+    current_scoring_mode = st.session_state.get(k_scoring, "Mean Forward Model Runs")
+    if current_scoring_mode not in scoring_options:
+        current_scoring_mode = scoring_options[0]
+
+    current_fwdruns_weight_percent = int(st.session_state.get(k_weight, 80))
+    current_fwdruns_weight_percent = max(0, min(100, current_fwdruns_weight_percent))
+
+    selected_target = current_target
+    scoring_mode = current_scoring_mode
+    fwdruns_weight = current_fwdruns_weight_percent / 100.0
+    ensemble_weight = 1.0 - fwdruns_weight
+
+    # Available methods for the current benchmark selection; used to populate the
+    # multiselect and to prune any stale saved selections when the benchmark changes.
+    available_methods = sorted(filtered["abbreviation"].dropna().unique().tolist())
+    saved_methods = st.session_state.get(k_methods, available_methods)
+    valid_saved = [m for m in saved_methods if m in available_methods]
+    st.session_state[k_methods] = valid_saved if valid_saved else available_methods
+
+    # Stable color scale: domain covers ALL methods so colors don't shift when a
+    # subset is displayed.
+    color_domain = available_methods
+    color_range = [_METHOD_PALETTE[i % len(_METHOD_PALETTE)] for i in range(len(available_methods))]
+    method_color = alt.Color(
+        "abbreviation:N",
+        title="Method",
+        scale=alt.Scale(domain=color_domain, range=color_range),
+    )
+
+    def build_scored_table(input_df: pd.DataFrame, add_rank: bool = True) -> pd.DataFrame:
+        ranking_source = (
+            input_df
+            if selected_target == "All targets"
+            else input_df[input_df[target_str_col] == selected_target]
+        )
+        if ranking_source.empty:
+            return ranking_source
+
+        # Failure rate from every row (NaN metric rows carry failure_rate=100)
+        failure_agg = ranking_source.groupby(
+            ["algorithm_type", "abbreviation", "Method", "family"], as_index=False
+        ).agg(**{"Mean Failure Rate (%)": ("failure_rate", "mean")})
+
+        # Metric stats only from runs that reached the target (non-NaN metric)
+        valid_rows = ranking_source.dropna(subset=["metric"])
+        if valid_rows.empty:
+            return pd.DataFrame()
+
+        scored_df = valid_rows.groupby(
+            ["algorithm_type", "abbreviation", "Method", "family"], as_index=False
+        ).agg(
+            **{"Mean Forward Model Runs": ("metric", "mean")},
+            **{"Minimum Forward Model Runs": ("metric", "min")},
+            **{"Targets Used": (target_str_col, "nunique")},
+            **{"Ensemble Sizes Used": ("ensemble_size", "nunique")},
+        )
+
+        best_per_target = (
+            valid_rows.sort_values(["algorithm_type", target_str_col, "metric", "ensemble_size"])
+            .groupby(["algorithm_type", "abbreviation", "Method", "family", target_str_col], as_index=False)
+            .first()[
+                [
+                    "algorithm_type",
+                    "abbreviation",
+                    "Method",
+                    "family",
+                    target_str_col,
+                    "ensemble_size",
+                ]
+            ]
+        )
+
+        optimal_ensemble = best_per_target.groupby(
+            ["algorithm_type", "abbreviation", "Method", "family"], as_index=False
+        ).agg(**{"Optimal Ensemble Size": ("ensemble_size", "mean")})
+
+        scored_df = scored_df.merge(
+            optimal_ensemble,
+            on=["algorithm_type", "abbreviation", "Method", "family"],
+            how="left",
+        )
+        scored_df = scored_df.merge(
+            failure_agg,
+            on=["algorithm_type", "abbreviation", "Method", "family"],
+            how="left",
+        )
+
+        scored_df["Optimal Ensemble Size"] = scored_df["Optimal Ensemble Size"].round(2)
+        scored_df["Mean Forward Model Runs"] = scored_df["Mean Forward Model Runs"].round(4)
+        scored_df["Minimum Forward Model Runs"] = scored_df["Minimum Forward Model Runs"].round(4)
+        scored_df["Mean Failure Rate (%)"] = scored_df["Mean Failure Rate (%)"].round(1)
+
+        mean_runs_min = scored_df["Mean Forward Model Runs"].min()
+        mean_runs_max = scored_df["Mean Forward Model Runs"].max()
+        if mean_runs_max > mean_runs_min:
+            scored_df["mean_runs_score"] = (
+                100.0 * (mean_runs_max - scored_df["Mean Forward Model Runs"]) / (mean_runs_max - mean_runs_min)
+            )
+        else:
+            scored_df["mean_runs_score"] = 100.0
+
+        minimum_runs_min = scored_df["Minimum Forward Model Runs"].min()
+        minimum_runs_max = scored_df["Minimum Forward Model Runs"].max()
+        if minimum_runs_max > minimum_runs_min:
+            scored_df["minimum_runs_score"] = (
+                100.0 * (minimum_runs_max - scored_df["Minimum Forward Model Runs"]) / (minimum_runs_max - minimum_runs_min)
+            )
+        else:
+            scored_df["minimum_runs_score"] = 100.0
+
+        ens_min = scored_df["Optimal Ensemble Size"].min()
+        ens_max = scored_df["Optimal Ensemble Size"].max()
+        if ens_max > ens_min:
+            scored_df["ensemble_score"] = (
+                100.0 * (ens_max - scored_df["Optimal Ensemble Size"]) / (ens_max - ens_min)
+            )
+        else:
+            scored_df["ensemble_score"] = 100.0
+
+        if scoring_mode == "Mean Forward Model Runs":
+            scored_df["Score"] = scored_df["mean_runs_score"]
+            sort_columns = ["Mean Forward Model Runs", "Optimal Ensemble Size", "abbreviation"]
+            ascending = [True, True, True]
+        elif scoring_mode == "Minimum Forward Model Runs":
+            scored_df["Score"] = scored_df["minimum_runs_score"]
+            sort_columns = ["Minimum Forward Model Runs", "Optimal Ensemble Size", "abbreviation"]
+            ascending = [True, True, True]
+        elif scoring_mode == "Smallest Optimal Ensemble Size":
+            scored_df["Score"] = scored_df["ensemble_score"]
+            sort_columns = ["Optimal Ensemble Size", "Mean Forward Model Runs", "abbreviation"]
+            ascending = [True, True, True]
+        else:
+            scored_df["Score"] = (
+                fwdruns_weight * scored_df["mean_runs_score"]
+                + ensemble_weight * scored_df["ensemble_score"]
+            )
+            sort_columns = ["Score", "Mean Forward Model Runs", "Optimal Ensemble Size", "abbreviation"]
+            ascending = [False, True, True, True]
+
+        scored_df = scored_df.sort_values(sort_columns, ascending=ascending).reset_index(drop=True)
+
+        if add_rank:
+            scored_df["Rank"] = scored_df.index + 1
+            scored_df["Placement"] = scored_df["Rank"].apply(
+                lambda rank: f"{ {1: '🥇', 2: '🥈', 3: '🥉'}.get(rank, '')} #{rank}".strip()
+            )
+
+        return scored_df
+
+    if selected_benchmark == "All":
+        benchmark_scores = []
+        for benchmark_name in benchmark_values:
+            benchmark_df = metric_store[metric_store["benchmark"] == benchmark_name].copy()
+            benchmark_df[target_str_col] = benchmark_df[target_col].astype(str)
+            scored = build_scored_table(benchmark_df, add_rank=False)
+            if scored.empty:
+                continue
+            scored["benchmark"] = benchmark_name
+            benchmark_scores.append(scored)
+
+        if benchmark_scores:
+            combined_scores = (
+                benchmark_scores[0].copy()
+                if len(benchmark_scores) == 1
+                else pd.concat(benchmark_scores, ignore_index=True)
+            )
+            leaderboard_df = combined_scores.groupby(
+                ["algorithm_type", "abbreviation", "Method", "family"], as_index=False
+            ).agg(
+                Score=("Score", "mean"),
+                **{"Mean Forward Model Runs": ("Mean Forward Model Runs", "mean")},
+                **{"Minimum Forward Model Runs": ("Minimum Forward Model Runs", "mean")},
+                **{"Optimal Ensemble Size": ("Optimal Ensemble Size", "mean")},
+                **{"Targets Used": ("Targets Used", "mean")},
+                **{"Ensemble Sizes Used": ("Ensemble Sizes Used", "mean")},
+                **{"Benchmarks Used": ("benchmark", "nunique")},
+                **{"Mean Failure Rate (%)": ("Mean Failure Rate (%)", "mean")},
+            )
+
+            leaderboard_df["Mean Forward Model Runs"] = leaderboard_df["Mean Forward Model Runs"].round(4)
+            leaderboard_df["Minimum Forward Model Runs"] = leaderboard_df["Minimum Forward Model Runs"].round(4)
+            leaderboard_df["Optimal Ensemble Size"] = leaderboard_df["Optimal Ensemble Size"].round(2)
+            leaderboard_df["Targets Used"] = leaderboard_df["Targets Used"].round().astype(int)
+            leaderboard_df["Ensemble Sizes Used"] = leaderboard_df["Ensemble Sizes Used"].round().astype(int)
+            leaderboard_df["Mean Failure Rate (%)"] = leaderboard_df["Mean Failure Rate (%)"].round(1)
+            leaderboard_df = leaderboard_df.sort_values(
+                ["Score", "Mean Forward Model Runs", "abbreviation"], ascending=[False, True, True]
+            ).reset_index(drop=True)
+            leaderboard_df["Rank"] = leaderboard_df.index + 1
+            leaderboard_df["Placement"] = leaderboard_df["Rank"].apply(
+                lambda rank: f"{ {1: '🥇', 2: '🥈', 3: '🥉'}.get(rank, '')} #{rank}".strip()
+            )
+        else:
+            leaderboard_df = pd.DataFrame(
+                columns=[
+                    "Placement",
+                    "abbreviation",
+                    "Method",
+                    "family",
+                    "Mean Forward Model Runs",
+                    "Minimum Forward Model Runs",
+                    "Score",
+                    "Optimal Ensemble Size",
+                    "Targets Used",
+                    "Ensemble Sizes Used",
+                    "Benchmarks Used",
+                    "Mean Failure Rate (%)",
+                ]
+            )
+    else:
+        leaderboard_df = build_scored_table(filtered, add_rank=True)
+        leaderboard_df["Benchmarks Used"] = 1
+
+    if scoring_mode == "Mean Forward Model Runs":
+        score_basis = "normalized mean of best forward model runs over selected target levels (lower is better)"
+    elif scoring_mode == "Minimum Forward Model Runs":
+        score_basis = "normalized minimum of forward model runs over selected targets and ensemble sizes (lower is better)"
+    elif scoring_mode == "Smallest Optimal Ensemble Size":
+        score_basis = "normalized mean optimal ensemble size over selected target levels (lower is better)"
+    else:
+        score_basis = (
+            "weighted blend of normalized forward-model-runs score and normalized ensemble-size score "
+            f"(forward-runs weight {fwdruns_weight:.0%}, ensemble-size weight {ensemble_weight:.0%})"
+        )
+
+    if selected_benchmark == "All":
+        score_basis = f"{score_basis}; in All mode, each method's final score is the mean of its per-benchmark scores"
+
+    # Controls expander — always shown so users can change target even when the
+    # current selection yields all failures.
+    with st.expander("Scoring & Target Controls", expanded=False):
+        st.radio(
+            target_label,
+            options=target_options,
+            horizontal=True,
+            key=k_target,
+        )
+
+        if show_scoring_modes:
+            st.radio(
+                "Scoring Method",
+                options=scoring_options,
+                horizontal=True,
+                key=k_scoring,
+            )
+
+            if st.session_state.get(k_scoring, "Mean Forward Model Runs") == "Custom Blend":
+                st.slider(
+                    "Blend Weight: Forward Runs vs Ensemble Size",
+                    min_value=0,
+                    max_value=100,
+                    step=5,
+                    key=k_weight,
+                    help=(
+                        "Higher forward-runs weight prioritizes fewer model evaluations; "
+                        "higher ensemble-size weight prioritizes smaller ensembles."
+                    ),
+                )
+
+        st.multiselect(
+            "Methods to display in charts",
+            options=available_methods,
+            key=k_methods,
+        )
+
+    selected_methods = st.session_state.get(k_methods, available_methods)
+    if not selected_methods:
+        selected_methods = available_methods
+
+    if leaderboard_df.empty:
+        st.warning(
+            "All runs failed to reach the target at this selection. "
+            "See the failure rate chart below."
+            if show_failure_panel
+            else "No rows available for the current benchmark/target selection."
+        )
+    else:
+        if selected_benchmark == "All":
+            table_column_order = [
+                "Placement",
+                "abbreviation",
+                "Method",
+                "family",
+                "Score",
+                "Mean Failure Rate (%)",
+                "Targets Used",
+                "Ensemble Sizes Used",
+                "Benchmarks Used",
+            ]
+        else:
+            table_column_order = [
+                "Placement",
+                "abbreviation",
+                "Method",
+                "family",
+                "Score",
+                "Mean Forward Model Runs",
+                "Minimum Forward Model Runs",
+                "Mean Failure Rate (%)",
+                "Optimal Ensemble Size",
+                "Targets Used",
+                "Ensemble Sizes Used",
+                "Benchmarks Used",
+            ]
+
+        st.subheader(f"Ranked Leaderboard — {selected_benchmark}")
+        st.dataframe(
+            leaderboard_df,
+            hide_index=True,
+            use_container_width=True,
+            column_config={
+                "Placement": st.column_config.TextColumn("Placement"),
+                "family": st.column_config.TextColumn("Family"),
+                "Method": st.column_config.TextColumn("Method"),
+                "abbreviation": st.column_config.TextColumn("Abbrev."),
+                "Mean Forward Model Runs": st.column_config.NumberColumn("Mean Forward Model Runs", format="%.4f"),
+                "Minimum Forward Model Runs": st.column_config.NumberColumn("Minimum Forward Model Runs", format="%.4f"),
+                "Score": st.column_config.ProgressColumn("Score (0-100)", min_value=0.0, max_value=100.0, format="%.1f"),
+                "Optimal Ensemble Size": st.column_config.NumberColumn("Mean Optimal Ensemble Size", format="%.2f"),
+                "Mean Failure Rate (%)": st.column_config.NumberColumn("Mean Failure Rate (%)", format="%.1f"),
+                "Targets Used": st.column_config.NumberColumn("Targets Used", format="%d"),
+                "Ensemble Sizes Used": st.column_config.NumberColumn("Ensemble Sizes Used", format="%d"),
+                "Benchmarks Used": st.column_config.NumberColumn("Benchmarks Used", format="%d"),
+            },
+            column_order=table_column_order,
+        )
+
+        st.info(
+            f"Score is a normalized 0–100 ranking based on **{score_basis}**. "
+            "For Mean/Minimum forward-runs scoring, values are computed from all selected metric target levels "
+            "and all ensemble sizes after averaging over random seeds."
+        )
+
+        if selected_benchmark != "All":
+            st.subheader("Mean Forward Model Runs vs Ensemble Size")
+            chart_source = (
+                filtered
+                if selected_target == "All targets"
+                else filtered[filtered[target_str_col] == selected_target]
+            )
+            chart_source = chart_source[chart_source["abbreviation"].isin(selected_methods)]
+            chart_df = chart_source.dropna(subset=["metric"]).groupby(
+                ["abbreviation", "ensemble_size"], as_index=False
+            ).agg(mean_forward_runs=("metric", "mean"))
+
+            all_ens_combos = chart_source[["abbreviation", "ensemble_size"]].drop_duplicates()
+            ens_ticks = sorted(all_ens_combos["ensemble_size"].unique().tolist()) if not all_ens_combos.empty else []
+
+            if not chart_df.empty:
+                _ok = chart_df[["abbreviation", "ensemble_size"]].assign(_ok=True)
+                fail_df = all_ens_combos.merge(_ok, on=["abbreviation", "ensemble_size"], how="left")
+                fail_df = fail_df[fail_df["_ok"].isna()].drop(columns="_ok").assign(mean_forward_runs=0.0)
+            else:
+                fail_df = all_ens_combos.assign(mean_forward_runs=0.0)
+
+            all_failed = chart_df.empty
+            chart_layers = []
+            if not chart_df.empty:
+                chart_layers.append(
+                    alt.Chart(chart_df)
+                    .mark_line(point=True)
+                    .encode(
+                        x=alt.X(
+                            "ensemble_size:Q",
+                            title="Ensemble Size",
+                            axis=alt.Axis(values=ens_ticks, format="d"),
+                        ),
+                        y=alt.Y("mean_forward_runs:Q", title="Mean Forward Model Runs"),
+                        color=method_color,
+                        tooltip=["abbreviation", "ensemble_size", alt.Tooltip("mean_forward_runs:Q", format=".4f")],
+                    )
+                )
+            if not fail_df.empty:
+                y_fwd = (
+                    alt.Y("mean_forward_runs:Q", title="Mean Forward Model Runs", scale=alt.Scale(domain=[0, 1]))
+                    if all_failed
+                    else alt.Y("mean_forward_runs:Q", title="Mean Forward Model Runs")
+                )
+                chart_layers.append(
+                    alt.Chart(fail_df)
+                    .mark_point(shape="cross", angle=45, size=200, filled=True, opacity=1.0)
+                    .encode(
+                        x=alt.X(
+                            "ensemble_size:Q",
+                            title="Ensemble Size",
+                            axis=alt.Axis(values=ens_ticks, format="d"),
+                        ),
+                        y=y_fwd,
+                        color=method_color,
+                        tooltip=[
+                            alt.Tooltip("abbreviation:N", title="Method"),
+                            alt.Tooltip("ensemble_size:Q", title="Ensemble Size"),
+                            alt.Tooltip("mean_forward_runs:Q", title="Value (all failed)"),
+                        ],
+                    )
+                )
+            if chart_layers:
+                st.altair_chart(alt.layer(*chart_layers), use_container_width=True)
+
+    # Mean-iterations-for-coverage section (UQ only, when budget_store provided)
+    if budget_store is not None and not budget_store.empty and selected_benchmark != "All":
+        bf = budget_store[budget_store["benchmark"] == selected_benchmark].copy()
+        bf[target_str_col] = bf[target_col].astype(str)
+        if selected_target != "All targets":
+            bf = bf[bf[target_str_col] == selected_target]
+        bf = bf[bf["abbreviation"].isin(selected_methods)]
+
+        iters_df = (
+            bf[["abbreviation", "ensemble_size", "mean_iters"]]
+            .dropna(subset=["mean_iters"])
+            .groupby(["abbreviation", "ensemble_size"], as_index=False)
+            .agg(mean_iters=("mean_iters", "mean"))
+        )
+        all_ens_combos_iters = bf[["abbreviation", "ensemble_size"]].drop_duplicates()
+        if not all_ens_combos_iters.empty:
+            iters_ticks = sorted(all_ens_combos_iters["ensemble_size"].unique().tolist())
+
+            if not iters_df.empty:
+                _ok_iters = iters_df[["abbreviation", "ensemble_size"]].assign(_ok=True)
+                fail_df_iters = all_ens_combos_iters.merge(_ok_iters, on=["abbreviation", "ensemble_size"], how="left")
+                fail_df_iters = fail_df_iters[fail_df_iters["_ok"].isna()].drop(columns="_ok").assign(mean_iters=0.0)
+            else:
+                fail_df_iters = all_ens_combos_iters.assign(mean_iters=0.0)
+
+            all_failed_iters = iters_df.empty
+            st.subheader("Mean Iterations for Coverage vs Ensemble Size")
+            iters_layers = []
+            if not iters_df.empty:
+                iters_layers.append(
+                    alt.Chart(iters_df)
+                    .mark_line(point=True)
+                    .encode(
+                        x=alt.X(
+                            "ensemble_size:Q",
+                            title="Ensemble Size",
+                            axis=alt.Axis(values=iters_ticks, format="d"),
+                        ),
+                        y=alt.Y("mean_iters:Q", title="Mean Iterations"),
+                        color=method_color,
+                        tooltip=[
+                            alt.Tooltip("abbreviation:N", title="Method"),
+                            alt.Tooltip("ensemble_size:Q", title="Ensemble Size"),
+                            alt.Tooltip("mean_iters:Q", format=".2f", title="Mean Iterations"),
+                        ],
+                    )
+                )
+            if not fail_df_iters.empty:
+                y_iters = (
+                    alt.Y("mean_iters:Q", title="Mean Iterations", scale=alt.Scale(domain=[0, 1]))
+                    if all_failed_iters
+                    else alt.Y("mean_iters:Q", title="Mean Iterations")
+                )
+                iters_layers.append(
+                    alt.Chart(fail_df_iters)
+                    .mark_point(shape="cross", angle=45, size=200, filled=True, opacity=1.0)
+                    .encode(
+                        x=alt.X(
+                            "ensemble_size:Q",
+                            title="Ensemble Size",
+                            axis=alt.Axis(values=iters_ticks, format="d"),
+                        ),
+                        y=y_iters,
+                        color=method_color,
+                        tooltip=[
+                            alt.Tooltip("abbreviation:N", title="Method"),
+                            alt.Tooltip("ensemble_size:Q", title="Ensemble Size"),
+                            alt.Tooltip("mean_iters:Q", title="Value (all failed)"),
+                        ],
+                    )
+                )
+            if iters_layers:
+                st.altair_chart(alt.layer(*iters_layers), use_container_width=True)
+
+    # Failure panel — rendered regardless of whether the scored table has rows
+    if show_failure_panel and selected_benchmark != "All":
+        failure_source = (
+            filtered
+            if selected_target == "All targets"
+            else filtered[filtered[target_str_col] == selected_target]
+        )
+        failure_source = failure_source[failure_source["abbreviation"].isin(selected_methods)]
+        if not failure_source.empty:
+            failure_df = failure_source.groupby(
+                ["abbreviation", "ensemble_size"], as_index=False
+            ).agg(mean_failure_rate=("failure_rate", "mean"))
+            failure_df = failure_df.sort_values("ensemble_size")
+
+            target_str = (
+                "All Targets"
+                if selected_target == "All targets"
+                else f"Target {selected_target}"
+            )
+            st.subheader(f"Failure Rate of Hitting {target_str}")
+            ens_ticks_fail = sorted(failure_df["ensemble_size"].unique().tolist())
+            failure_chart = (
+                alt.Chart(failure_df)
+                .mark_bar()
+                .encode(
+                    x=alt.X(
+                        "ensemble_size:O",
+                        title="Ensemble Size",
+                        sort=[str(e) for e in ens_ticks_fail],
+                        axis=alt.Axis(labelAngle=0),
+                    ),
+                    xOffset=alt.XOffset("abbreviation:N"),
+                    y=alt.Y(
+                        "mean_failure_rate:Q",
+                        title="Failure Rate (%)",
+                        scale=alt.Scale(domain=[0, 100]),
+                    ),
+                    color=method_color,
+                    tooltip=[
+                        alt.Tooltip("abbreviation:N", title="Method"),
+                        alt.Tooltip("ensemble_size:O", title="Ensemble Size"),
+                        alt.Tooltip("mean_failure_rate:Q", format=".1f", title="Failure Rate (%)"),
+                    ],
+                )
+            )
+            ceiling_line = (
+                alt.Chart(alt.Data(values=[{}]))
+                .mark_rule(color="grey", strokeDash=[4, 4])
+                .encode(y=alt.datum(100))
+            )
+            st.altair_chart(failure_chart + ceiling_line, use_container_width=True)
+
+    st.caption("Top 3 are shown as podium spots; remaining methods are directly comparable via normalized score.")
+
+    if raw_page is not None:
+        st.page_link(raw_page, label="Open Raw Data & CSV Export", icon="🧾")

src/common/method_registry.py

@@ -21,7 +21,7 @@ KNOWN_METHODS = {
         "abbreviation": "IEKF",
         "Method": "Iterative Ensemble Kalman Filter",
         "family": "Kalman",
-        "aliases": ["iekf", "gnsl"],
+        "aliases": ["iekf", "gnsl", "gnki"],
     },
     "uki": {
         "abbreviation": "UKI",
@@ -41,6 +41,12 @@ KNOWN_METHODS = {
         "family": "Bayesian",
         "aliases": ["hm"],
     },
+    "ces-eki-dmc": {
+        "abbreviation": "CES-EKI-DMC",
+        "Method": "Calibrate Emulate Sample (EKI-DataMisfitController)",
+        "family": "calibrate_then_emulate",
+        "aliases": ["ces-eki-dmc"]
+    }
 }
 
 

src/data_store.py

@@ -2,7 +2,8 @@ from __future__ import annotations
 
 import importlib
 from pathlib import Path
-import warnings 
+import warnings
+import numpy as np
 import pandas as pd
 import streamlit as st
 
@@ -22,6 +23,10 @@ except ModuleNotFoundError:
     )
 
 
+# ---------------------------------------------------------------------------
+# Optimization (RMSE) datasets
+# ---------------------------------------------------------------------------
+
 DATASET_FILES = {
     "L63": [
         "l63_ensemble_results.nc",
@@ -45,13 +50,66 @@ DATASET_FILES = {
     ],
 }
 
+EXPECTED_DIMS_OPT = ("algorithm_type", "rmse_target", "ensemble_size", "random_seed")
+EXPECTED_DIMS_UQ  = ("random_seed", "ensemble_size", "k_iter", "coverage_quantile")
 
 
-EXPECTED_DIMS = ("algorithm_type", "rmse_target", "ensemble_size", "random_seed")
+# ---------------------------------------------------------------------------
+# Uncertainty Quantification datasets
+# ---------------------------------------------------------------------------
+# Add entries here when UQ result NetCDF files are available.  Each file must
+# carry a ``metric`` variable and a ``uq_target`` coordinate (in place of the
+# ``rmse_target`` used by the optimization files).  The rest of the schema
+# (algorithm_type, ensemble_size, random_seed) is identical.
+#
+# Example:
+#   UQ_DATASET_FILES = {
+#       "L63": ["uq/l63_uq_results.nc"],
+#       "L96": ["uq/l96_uq_results.nc"],
+#   }
 
+UQ_DATASET_FILES: dict[str, list[str]] = {}
+
+# Canonical UQ target-scaling levels always offered in the UQ leaderboard selector.
+UQ_TARGET_LEVELS: list[float] = [1.0, 1.25, 1.5, 1.75, 2.0, 2.25, 2.5]
+
+# Budget-for-coverage source files: maps benchmark → list of (algorithm_type, nc_path) pairs.
+# Each NC must carry output_coverage (coverage_quantile, k_iter, ensemble_size, random_seed),
+# target_scaling, and output_dim.
+UQ_BUDGET_FILES: dict[str, list[tuple[str, str]]] = {
+    "L63":                [("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l63_ensemble_results_2026-06-15_minimal.nc")],
+    "L96":                [("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l96_ensemble_results_2026-06-15_minimal.nc")],
+    "L96_NN_FORCING":     [("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l96_nn_forcing_ensemble_results_2026-06-15_minimal.nc")],
+    "L96_SPATIAL_FORCING": [("ces-eki-dmc", "ces-eki-dmc_results/ces-eki-dmc_l96_spatial_forcing_ensemble_results_2026-06-15_minimal.nc")],
+}
+
+# Quantile levels used for the all-quantiles-satisfied budget-for-coverage condition.
+# Budget = N_ens · k_iter where first k s.t. |S(q)−q| ≤ c·√(q(1−q)/N_y) for ALL q below.
+UQ_COVERAGE_QUANTILES: list[float] = [0.15, 0.5, 0.85]
+
+
+
+# ---------------------------------------------------------------------------
+# Private loader — shared by both public load_* functions
+# ---------------------------------------------------------------------------
+
+def _load_store(dataset_files: dict[str, list[str]], target_col: str, *, drop_nan_metric: bool = True) -> pd.DataFrame:
+    """Load and merge NetCDF metric files into a long-format DataFrame.
+
+    Parameters
+    ----------
+    dataset_files:
+        Mapping of benchmark name → list of relative NetCDF paths under
+        ``data/``.
+    target_col:
+        Name of the target-coordinate dimension in the NetCDF files, e.g.
+        ``"rmse_target"`` or ``"uq_target"``.
+    """
+    if not dataset_files:
+        return pd.DataFrame(
+            columns=["benchmark", "algorithm_type", target_col, "ensemble_size", "metric", "failure_rate"]
+        )
 
-@st.cache_data(show_spinner=False)
-def load_metric_store() -> pd.DataFrame:
     xr = importlib.import_module("xarray")
 
     project_root = Path(__file__).resolve().parent.parent
@@ -59,7 +117,7 @@ def load_metric_store() -> pd.DataFrame:
 
     records: list[pd.DataFrame] = []
 
-    for benchmark_name, filenames in DATASET_FILES.items():
+    for benchmark_name, filenames in dataset_files.items():
         for filename in filenames:
             file_path = data_dir / filename
             if not file_path.exists():
@@ -72,44 +130,53 @@ def load_metric_store() -> pd.DataFrame:
                     continue
 
                 metric = dataset["metric"]
-                
+                if not set(metric.dims).issubset(EXPECTED_DIMS_OPT):
+                    warnings.warn(
+                        f"{file_path.name}: metric dims {metric.dims} contain unexpected "
+                        f"dimensions not in EXPECTED_DIMS_OPT {EXPECTED_DIMS_OPT} — skipping"
+                    )
+                    continue
+
                 # Track failures (metric == -1) as a percentage
                 failures_count = (metric == -1).sum(dim="random_seed")
 
                 failure_rate = (failures_count / dataset.sizes["random_seed"]) * 100
-                
-                # Filter out failed calibrations (metric == -1)
+
+                # Filter out failed runs (metric == -1)
                 metric = metric.where(metric != -1)
-                
+
                 metric_mean = metric.mean(dim="random_seed", skipna=True)
                 df = metric_mean.to_dataframe(name="metric").reset_index()
-                
+
                 df_failures = failure_rate.to_dataframe(name="failure_rate").reset_index()
                 df["failure_rate"] = df_failures["failure_rate"]
 
                 df["benchmark"] = benchmark_name
-                
+
                 if "algorithm_type" not in df.columns:
                     if "abc" in file_path.name:
                         df["algorithm_type"] = "abc"
                     elif "hm" in file_path.name:
                         df["algorithm_type"] = "hm"
-                        
+
                 if "ensemble_size" not in df.columns:
                     if "abc" in file_path.name:
                         df["ensemble_size"] = 1
-                        
-                missing_cols = [c for c in ["benchmark", "algorithm_type", "rmse_target", "ensemble_size", "metric", "failure_rate"] if c not in df.columns]
+
+                required_cols = ["benchmark", "algorithm_type", target_col, "ensemble_size", "metric", "failure_rate"]
+                missing_cols = [c for c in required_cols if c not in df.columns]
                 if missing_cols:
                     st.warning(
                         f"Skipping {file_path.name}: metric dataframe is missing expected columns {missing_cols}"
                     )
                     continue
 
-                records.append(df[["benchmark", "algorithm_type", "rmse_target", "ensemble_size", "metric", "failure_rate"]])
+                records.append(df[required_cols])
 
     if not records:
-        return pd.DataFrame(columns=["benchmark", "algorithm_type", "rmse_target", "ensemble_size", "metric", "failure_rate"])
+        return pd.DataFrame(
+            columns=["benchmark", "algorithm_type", target_col, "ensemble_size", "metric", "failure_rate"]
+        )
 
     merged = pd.concat(records, ignore_index=True)
     merged["algorithm_alias"] = merged["algorithm_type"].map(normalize_method_name)
@@ -117,10 +184,11 @@ def load_metric_store() -> pd.DataFrame:
     merged["ensemble_size"] = pd.to_numeric(merged["ensemble_size"], errors="coerce")
     merged["metric"] = pd.to_numeric(merged["metric"], errors="coerce")
     merged["failure_rate"] = pd.to_numeric(merged["failure_rate"], errors="coerce").fillna(0.0)
-    merged = merged.dropna(subset=["ensemble_size", "metric"])
+    if drop_nan_metric:
+        merged = merged.dropna(subset=["ensemble_size", "metric"])
+    else:
+        merged = merged.dropna(subset=["ensemble_size"])
     merged["ensemble_size"] = merged["ensemble_size"].astype(int)
-    merged["forward_model_runs"] = merged["metric"]
-
     merged["abbreviation"] = merged["algorithm_type"].map(lambda method: get_method_meta(method).get("abbreviation"))
     merged["Method"] = merged["algorithm_type"].map(lambda method: get_method_meta(method).get("Method"))
     merged["family"] = merged["algorithm_type"].map(lambda method: get_method_meta(method).get("family"))
@@ -136,23 +204,159 @@ def load_metric_store() -> pd.DataFrame:
     return merged
 
 
-def build_leaderboard(metric_store: pd.DataFrame) -> pd.DataFrame:
-    if metric_store.empty:
+# ---------------------------------------------------------------------------
+# Public cached loaders
+# ---------------------------------------------------------------------------
+
+@st.cache_data(show_spinner=False)
+def load_metric_store() -> pd.DataFrame:
+    """Load the optimization (RMSE) metric store from ``DATASET_FILES``."""
+    return _load_store(DATASET_FILES, "rmse_target")
+
+
+@st.cache_data(show_spinner=False)
+def load_uq_store() -> pd.DataFrame:
+    """Load the UQ metric store derived from budget-for-coverage computations.
+
+    Returns a DataFrame compatible with ``render_leaderboard`` where ``metric``
+    is the mean budget (N_ens·k_iter) to satisfy the coverage condition across
+    all three quantiles in ``UQ_COVERAGE_QUANTILES``.  ``uq_target`` is the
+    target-scaling level c.  Failure means the coverage target was never
+    reached within the k_iter range.
+    """
+    budget_df = load_uq_budget_store()
+    if budget_df.empty:
         return pd.DataFrame(
             columns=[
-                "family",
-                "Method",
-                "abbreviation",
-                "benchmark",
-                "Forward Model Runs",
-                "Optimal Ensemble Size",
-                "Target Level",
+                "benchmark", "algorithm_type", "algorithm_alias", "abbreviation",
+                "Method", "family", "uq_target", "ensemble_size", "metric", "failure_rate",
             ]
         )
-
-    best_idx = metric_store.groupby(["benchmark", "algorithm_type"])["metric"].idxmin()
-    best = metric_store.loc[best_idx].copy()
-    best = best.rename(columns={"metric": "Forward Model Runs", "rmse_target": "Target Level", "ensemble_size": "Optimal Ensemble Size"})
-    return best[["family", "Method", "abbreviation", "benchmark", "Forward Model Runs", "Optimal Ensemble Size", "Target Level"]].sort_values(
-        ["benchmark", "Forward Model Runs"], ascending=[True, True]
+    df = budget_df.copy()
+    df["metric"] = df["mean_budget"]
+    df["algorithm_alias"] = df["algorithm_type"].map(normalize_method_name)
+    df["Method"] = df["algorithm_type"].map(
+        lambda m: get_method_meta(canonicalize_method_name(normalize_method_name(m))).get(
+            "Method", m.upper()
+        )
     )
+    return df[
+        [
+            "benchmark", "algorithm_type", "algorithm_alias", "abbreviation",
+            "Method", "family", "uq_target", "ensemble_size", "metric", "failure_rate",
+        ]
+    ]
+
+
+@st.cache_data(show_spinner=False)
+def load_uq_budget_store() -> pd.DataFrame:
+    """Compute budget-for-coverage and iterations-for-coverage from ensemble result NC files.
+
+    For each (target_scaling c, ensemble_size N, random_seed):
+      Find the minimum k_iter s.t. |S(q) − q| ≤ c·√(q(1−q)/N_y) for ALL q in
+      UQ_COVERAGE_QUANTILES.  Budget = N · k_iter; NaN when target never reached.
+
+    Returns a DataFrame with columns:
+        benchmark, algorithm_type, abbreviation, family, uq_target,
+        ensemble_size, mean_budget, mean_iters, failure_count, failure_rate, n_seeds
+    """
+    xr = importlib.import_module("xarray")
+
+    project_root = Path(__file__).resolve().parent.parent
+    data_dir = project_root / "data"
+
+    bq_vals = np.array(UQ_COVERAGE_QUANTILES)
+    records: list[dict] = []
+
+    for benchmark_name, method_files in UQ_BUDGET_FILES.items():
+        for algorithm_type, filename in method_files:
+            file_path = data_dir / filename
+            if not file_path.exists():
+                warnings.warn(f"UQ budget file {file_path} does not exist")
+                continue
+
+            with xr.open_dataset(file_path) as ds:
+                if "output_coverage" not in ds:
+                    warnings.warn(f"{file_path.name}: missing output_coverage")
+                    continue
+
+                if set(ds["output_coverage"].dims) != set(EXPECTED_DIMS_UQ):
+                    warnings.warn(
+                        f"{file_path.name}: output_coverage dims {ds['output_coverage'].dims} "
+                        f"do not match EXPECTED_DIMS_UQ {EXPECTED_DIMS_UQ} — skipping"
+                    )
+                    continue
+
+                cov_q = ds["coverage_quantile"].values  # (n_cov_q,)
+                k_vals = ds["k_iter"].values            # (n_k,)  1-indexed
+                ens_vals = ds["ensemble_size"].values   # (n_ens,)
+                ts_vals = ds["target_scaling"].values   # (n_ts,)
+                # xarray drops dimensions that no variable uses; read output_dim
+                # directly from the underlying netCDF4 file to handle minimal files.
+                nc4 = importlib.import_module("netCDF4")
+                with nc4.Dataset(file_path) as _nc:
+                    if "output_dim" not in _nc.dimensions:
+                        warnings.warn(f"{file_path.name}: missing output_dim dimension")
+                        continue
+                    n_y = len(_nc.dimensions["output_dim"])
+                # dims: (coverage_quantile, k_iter, ensemble_size, random_seed)
+                cov_np = ds["output_coverage"].values
+
+                n_rng = cov_np.shape[3]
+
+                # Indices into coverage_quantile for [0.15, 0.5, 0.85]
+                bq_idx = [int(np.argmin(np.abs(cov_q - q))) for q in bq_vals]
+
+                for si, c in enumerate(ts_vals):
+                    tol = c * np.sqrt(bq_vals * (1.0 - bq_vals) / n_y)
+
+                    for ei, ens_f in enumerate(ens_vals):
+                        N_ens = int(ens_f)
+                        budgets = np.full(n_rng, np.nan)
+                        kiters = np.full(n_rng, np.nan)
+
+                        for ri in range(n_rng):
+                            for ki, kv in enumerate(k_vals):
+                                s_q = cov_np[bq_idx, ki, ei, ri]
+                                if np.any(np.isnan(s_q)):
+                                    continue
+                                if np.all(np.abs(s_q - bq_vals) <= tol):
+                                    budgets[ri] = N_ens * float(kv)
+                                    kiters[ri] = float(kv)
+                                    break
+
+                        valid_b = budgets[~np.isnan(budgets)]
+                        valid_k = kiters[~np.isnan(kiters)]
+                        n_fail = n_rng - len(valid_b)
+                        records.append(
+                            {
+                                "benchmark": benchmark_name,
+                                "algorithm_type": algorithm_type,
+                                "uq_target": float(c),
+                                "ensemble_size": N_ens,
+                                "mean_budget": float(np.mean(valid_b)) if len(valid_b) > 0 else np.nan,
+                                "mean_iters": float(np.mean(valid_k)) if len(valid_k) > 0 else np.nan,
+                                "failure_count": n_fail,
+                                "failure_rate": 100.0 * n_fail / n_rng,
+                                "n_seeds": n_rng,
+                            }
+                        )
+
+    if not records:
+        return pd.DataFrame(
+            columns=[
+                "benchmark", "algorithm_type", "abbreviation", "family",
+                "uq_target", "ensemble_size", "mean_budget", "mean_iters",
+                "failure_count", "failure_rate", "n_seeds",
+            ]
+        )
+
+    df = pd.DataFrame(records)
+
+    def _meta(m: str, key: str, fallback: str) -> str:
+        return get_method_meta(canonicalize_method_name(normalize_method_name(m))).get(key, fallback)
+
+    df["abbreviation"] = df["algorithm_type"].map(lambda m: _meta(m, "abbreviation", m.upper()))
+    df["family"] = df["algorithm_type"].map(lambda m: _meta(m, "family", "Kalman"))
+    df["Method"] = df["algorithm_type"].map(lambda m: _meta(m, "Method", m.upper()))
+    return df

src/pages/MethodDetails.py

@@ -15,6 +15,8 @@ st.set_page_config(page_title="Method Details", page_icon="📘", layout="wide")
 # Sidebar navigation
 st.sidebar.title("Navigation")
 st.sidebar.page_link("streamlit_app.py", label="Home", icon="🏠")
+st.sidebar.page_link("pages/OptimizationLeaderboard.py", label="Optimization Leaderboard", icon="📊")
+st.sidebar.page_link("pages/UQLeaderboard.py", label="UQ Leaderboard", icon="🎯")
 st.sidebar.page_link("pages/MethodDetails.py", label="Methods", icon="📘")
 st.sidebar.page_link("pages/RawData.py", label="Get Data", icon="🧾")
 
@@ -38,24 +40,24 @@ abbrs = methods_df["abbreviation"].tolist()
 # Metadata dictionary (can be moved to a JSON/YAML later)
 method_meta = {
     "TEKI": {
-        "citation": "Chada et al., SIAM/ASA J. UQ, 2018",
+        "citation": "Chada et al., SIAM/ASA J. UQ, 2020",
         "url": "https://doi.org/10.1137/17M114402X",
-        "summary": "EKI variant with Tikhonov regularization for stability.",
+        "summary": "EKI variant with Tikhonov regularization that adds a penalty term to prevent ensemble collapse and improve stability on nonlinear problems.",
     },
     "ETKI": {
-        "citation": "Ensemble transform variant (citation to be added)",
-        "url": "https://example.com/etki",
-        "summary": "Transform-based Kalman inversion using ensemble-space updates.",
+        "citation": "Schillings & Stuart, Numer. Math., 2017",
+        "url": "https://clima.github.io/EnsembleKalmanProcesses.jl/dev/",
+        "summary": "Ensemble Transform Kalman Inversion — applies an ensemble-space transform update that preserves the ensemble mean while reducing variance inflation.",
     },
     "IEKF": {
-        "citation": "Iterative ensemble Kalman filter (citation to be added)",
-        "url": "https://example.com/iekf",
-        "summary": "Iterative Kalman updates for nonlinear inverse problems.",
+        "citation": "Iglesias, Inverse Problems, 2016",
+        "url": "https://doi.org/10.1088/0266-5611/32/2/025002",
+        "summary": "Regularizing iterative ensemble Kalman method; repeatedly refines the ensemble around a regularized Gauss-Newton step for nonlinear inverse problems.",
     },
     "UKI": {
-        "citation": "Unscented transform-based inversion (placeholder)",
-        "url": "https://example.com/uki",
-        "summary": "Uses sigma points to propagate uncertainty without linearization.",
+        "citation": "Huang, Huang & Stuart, Physica D, 2022",
+        "url": "https://clima.github.io/EnsembleKalmanProcesses.jl/dev/",
+        "summary": "Unscented Kalman Inversion — propagates a deterministic set of sigma points through the forward model to estimate mean and covariance without linearization.",
     },
     "ABC": {
         "citation": "Approximate Bayesian Calibration",
@@ -63,10 +65,15 @@ method_meta = {
         "summary": "Sample without exact likelihoods until error falls below a target convergence.",
     },
     "HM": {
-        "citation": "History Matching",
+        "citation": "Williamson et al. 2013; King et al. 2025",
         "url": "https://example.com/hm",
         "summary": "Iterative constraint of parameter space using wave reductions.",
     },
+    "CES-EKI-DMC": {
+        "citation": "Cleary et al., J. Comput. Phys., 2021",
+        "url": "https://doi.org/10.1016/j.jcp.2020.109716",
+        "summary": "Calibrate-Emulate-Sample: uses EKI with a DataMisfitController to select training points, builds a GP emulator of the forward model, then samples the posterior via MCMC.",
+    },
 }
 
 # Selection UI (defaults to query param if valid)
@@ -102,7 +109,7 @@ st.markdown("**Benchmark Slice**")
 slice_df = metric_store[metric_store["abbreviation"] == sel].copy()
 slice_df = slice_df.sort_values(["benchmark", "rmse_target", "ensemble_size"])
 
-target_options = ["1.0", "1.1", "1.2"]
+target_options = sorted(slice_df["rmse_target"].astype(str).unique().tolist())
 selected_target = st.radio("RMSE Target Level", options=target_options, horizontal=True)
 
 best_table_view = slice_df[slice_df["rmse_target"].astype(str) == selected_target]
@@ -111,7 +118,7 @@ best_idx = best_table_view.groupby("benchmark")["metric"].idxmin()
 best_ensemble_df = best_table_view.loc[best_idx, ["benchmark", "rmse_target", "ensemble_size", "metric", "failure_rate"]].rename(
     columns={"metric": "Mean Forward Model Runs", "failure_rate": "Failure Rate (%)", "ensemble_size": "Optimal Ensemble Size", "rmse_target": "RMSE Target"}
 )
-st.dataframe(best_ensemble_df, use_container_width=True, hide_index=True)
+st.dataframe(best_ensemble_df, hide_index=True, use_container_width=True)
 
 if sel == "HM":
     st.markdown("### Failure Analysis")
@@ -145,7 +152,7 @@ st.markdown("**Best by Benchmark**")
 st.dataframe(
     best_by_benchmark,
     hide_index=True,
-    width="stretch",
+    use_container_width=True,
     column_config={
         "benchmark": st.column_config.TextColumn("Benchmark"),
         "Best Mean Forward Model Runs": st.column_config.NumberColumn("Best Mean Forward Model Runs", format="%.4f"),
@@ -169,13 +176,13 @@ chart = (
         tooltip=["benchmark", "ensemble_size", alt.Tooltip("mean_forward_runs:Q", format=".4f")],
     )
 )
-st.altair_chart(chart, width="stretch")
+st.altair_chart(chart, use_container_width=True)
 
 st.markdown("**All Averaged Configurations for Method**")
 st.dataframe(
     slice_df[["benchmark", "algorithm_alias", "rmse_target", "ensemble_size", "metric"]],
     hide_index=True,
-    width="stretch",
+    use_container_width=True,
     column_config={
         "benchmark": st.column_config.TextColumn("Benchmark"),
         "algorithm_alias": st.column_config.TextColumn("Source Alias"),
@@ -185,4 +192,4 @@ st.dataframe(
     },
 )
 
-st.page_link("streamlit_app.py", label="⬅ Back to Leaderboard", icon="↩️")
+st.page_link("pages/OptimizationLeaderboard.py", label="⬅ Back to Optimization Leaderboard", icon="↩️")

src/pages/OptimizationLeaderboard.py

@@ -0,0 +1,35 @@
+from pathlib import Path
+import sys
+
+import streamlit as st
+
+try:
+    from data_store import load_metric_store
+    from common.leaderboard import render_leaderboard
+except ModuleNotFoundError:
+    sys.path.append(str(Path(__file__).resolve().parents[1]))
+    from data_store import load_metric_store
+    from common.leaderboard import render_leaderboard
+
+st.set_page_config(page_title="Optimization Leaderboard", page_icon="📊", layout="wide")
+
+# Sidebar navigation
+st.sidebar.title("Navigation")
+st.sidebar.page_link("streamlit_app.py", label="Home", icon="🏠")
+st.sidebar.page_link("pages/OptimizationLeaderboard.py", label="Optimization Leaderboard", icon="📊")
+st.sidebar.page_link("pages/UQLeaderboard.py", label="UQ Leaderboard", icon="🎯")
+st.sidebar.page_link("pages/MethodDetails.py", label="Methods", icon="📘")
+st.sidebar.page_link("pages/RawData.py", label="Get Data", icon="🧾")
+
+st.markdown(r"**Optimization target.** A run succeeds when the normalized weighted distance from the ensemble-mean forward-model output to the true data falls at or below the (selected in the target controls) RMSE target level $\tau$:")
+st.latex(r"\frac{1}{N_y}\bigl(y - G(\bar{\theta})\bigr)^\top \Gamma^{-1}\bigl(y - G(\bar{\theta})\bigr) \leq \tau")
+st.markdown(r"where $y$ is the true observation, $G(\bar{\theta})$ is the forward-model output at the final ensemble mean $\bar{\theta}$, and $\Gamma$ is the observation noise covariance, and $N_y = \dim(y)$.")
+
+render_leaderboard(
+    load_metric_store(),
+    target_col="rmse_target",
+    target_label="RMSE Target Level",
+    title="Optimization Leaderboard",
+    state_prefix="opt",
+    raw_page="pages/RawData.py",
+)

src/pages/RawData.py

@@ -14,6 +14,8 @@ st.set_page_config(page_title="Raw Data", page_icon="🧾", layout="wide")
 # Sidebar navigation
 st.sidebar.title("Navigation")
 st.sidebar.page_link("streamlit_app.py", label="Home", icon="🏠")
+st.sidebar.page_link("pages/OptimizationLeaderboard.py", label="Optimization Leaderboard", icon="📊")
+st.sidebar.page_link("pages/UQLeaderboard.py", label="UQ Leaderboard", icon="🎯")
 st.sidebar.page_link("pages/MethodDetails.py", label="Methods", icon="📘")
 st.sidebar.page_link("pages/RawData.py", label="Get Data", icon="🧾")
 
@@ -47,7 +49,7 @@ st.dataframe(
         ]
     ],
     hide_index=True,
-    width="stretch",
+    use_container_width=True,
     column_config={
         "benchmark": st.column_config.TextColumn("Benchmark"),
         "abbreviation": st.column_config.TextColumn("Abbrev."),
@@ -69,4 +71,4 @@ st.download_button(
     mime="text/csv",
 )
 
-st.page_link("streamlit_app.py", label="⬅ Back to Leaderboard", icon="↩️")
+st.page_link("pages/OptimizationLeaderboard.py", label="⬅ Back to Optimization Leaderboard", icon="↩️")

src/pages/UQLeaderboard.py

@@ -0,0 +1,40 @@
+from pathlib import Path
+import sys
+
+import streamlit as st
+
+try:
+    from data_store import load_uq_store, load_uq_budget_store, UQ_TARGET_LEVELS
+    from common.leaderboard import render_leaderboard
+except ModuleNotFoundError:
+    sys.path.append(str(Path(__file__).resolve().parents[1]))
+    from data_store import load_uq_store, load_uq_budget_store, UQ_TARGET_LEVELS
+    from common.leaderboard import render_leaderboard
+
+st.set_page_config(page_title="UQ Leaderboard", page_icon="🎯", layout="wide")
+
+# Sidebar navigation
+st.sidebar.title("Navigation")
+st.sidebar.page_link("streamlit_app.py", label="Home", icon="🏠")
+st.sidebar.page_link("pages/OptimizationLeaderboard.py", label="Optimization Leaderboard", icon="📊")
+st.sidebar.page_link("pages/UQLeaderboard.py", label="UQ Leaderboard", icon="🎯")
+st.sidebar.page_link("pages/MethodDetails.py", label="Methods", icon="📘")
+st.sidebar.page_link("pages/RawData.py", label="Get Data", icon="🧾")
+
+st.markdown(r"**UQ target.** Marginal coverage, $\widetilde{S}(q)$ measures the fraction of output dimensions where the true observation $y$ falls below the $q$-quantile of the posterior predictive distribution. This effectively treats each output dimension as an independent trial, so we also whitened the outputs and data by $\Gamma$, Though not notated, we also truncate to retain 99% of the variance:")
+st.latex(r"\widetilde{S}(q) = \frac{1}{N_y}\sum_{n=1}^{N_y} \mathbb{I}\!\left(\mathrm{cdf}_{\,\Gamma^{-1/2}\mathrm{post}}\!\left(\Gamma^{-1/2}y\right) \leq q\right)")
+st.markdown(r"A run succeeds when **all three of the target quantiles** $q \in \{0.15, 0.50, 0.85\}$ simultaneously satisfy")
+st.latex(r"\left|\widetilde{S}(q) - q\right| \leq \alpha_c(q), \qquad \alpha_c(q) = c\sqrt{\frac{q(1-q)}{N_y}}")
+st.markdown(r"The boundary $\alpha_c(q)$ adapts the race-goal tolerance for different quantiles, as some are easier to estimate than others with finite samples. while $c$ is the target-scaling level that can be selected in the controls below, and $N_y = \dim(\Gamma^{-1/2}y)$ (and after truncation).")
+
+render_leaderboard(
+    load_uq_store(),
+    target_col="uq_target",
+    target_label="UQ Target (coverage tolerance scaling)",
+    title="Uncertainty Quantification Leaderboard",
+    state_prefix="uq",
+    show_failure_panel=True,
+    show_scoring_modes=False,
+    canonical_target_levels=UQ_TARGET_LEVELS,
+    budget_store=load_uq_budget_store(),
+)

src/streamlit_app.py

@@ -1,324 +1,103 @@
-import altair as alt
-import importlib
-import pandas as pd
 import streamlit as st
+from pathlib import Path
 
-st.set_page_config(layout="wide")
-
-if __package__:
-    data_store = importlib.import_module(f"{__package__}.data_store")
-else:
-    data_store = importlib.import_module("data_store")
-
-load_metric_store = data_store.load_metric_store
-
-st.title("Calibration Benchmark")
+st.set_page_config(page_title="Calibration Benchmark", page_icon="🏠", layout="wide")
 
 # Sidebar navigation
 st.sidebar.title("Navigation")
 st.sidebar.page_link("streamlit_app.py", label="Home", icon="🏠")
+st.sidebar.page_link("pages/OptimizationLeaderboard.py", label="Optimization Leaderboard", icon="📊")
+st.sidebar.page_link("pages/UQLeaderboard.py", label="UQ Leaderboard", icon="🎯")
 st.sidebar.page_link("pages/MethodDetails.py", label="Methods", icon="📘")
 st.sidebar.page_link("pages/RawData.py", label="Get Data", icon="🧾")
 
-
-show_home = st.session_state.get("show_home", True)
-if show_home:
-    st.header("Calibration Leaderboard")
-
-    metric_store = load_metric_store()
-    if metric_store.empty:
-        st.warning("No metric data found. Expected NetCDF files in `data/` with a `metric` variable.")
-        st.stop()
-
-    benchmark_values = sorted(metric_store["benchmark"].unique().tolist())
-    benchmark_options = ["All"] + benchmark_values
-    default_benchmark_index = 0 
-    selected_benchmark = st.selectbox("Benchmark", options=benchmark_options, index=default_benchmark_index)
-
-    filtered = metric_store.copy() if selected_benchmark == "All" else metric_store[metric_store["benchmark"] == selected_benchmark].copy()
-    filtered["rmse_target_str"] = filtered["rmse_target"].astype(str)
-
-    target_options = ["All targets"] + sorted(metric_store["rmse_target"].astype(str).unique().tolist())
-    scoring_options = [
-        "Mean Forward Model Runs",
-        "Minimum Forward Model Runs",
-        "Smallest Optimal Ensemble Size",
-        "Custom Blend",
-    ]
-
-    current_target = st.session_state.get("selected_target", "All targets")
-    if current_target not in target_options:
-        current_target = target_options[0]
-
-    current_scoring_mode = st.session_state.get("scoring_mode", "Mean Forward Model Runs")
-    if current_scoring_mode not in scoring_options:
-        current_scoring_mode = scoring_options[0]
-
-    current_fwdruns_weight_percent = int(st.session_state.get("fwdruns_weight_percent", 80))
-    current_fwdruns_weight_percent = max(0, min(100, current_fwdruns_weight_percent))
-
-    selected_target = current_target
-    scoring_mode = current_scoring_mode
-    fwdruns_weight = current_fwdruns_weight_percent / 100.0
-    ensemble_weight = 1.0 - fwdruns_weight
-
-    def build_scored_table(input_df, add_rank=True):
-        ranking_source = input_df if selected_target == "All targets" else input_df[input_df["rmse_target_str"] == selected_target]
-        if ranking_source.empty:
-            return ranking_source
-
-        scored_df = ranking_source.groupby(["algorithm_type", "abbreviation", "Method", "family"], as_index=False).agg(
-            **{"Mean Forward Model Runs": ("metric", "mean")},
-            **{"Minimum Forward Model Runs": ("metric", "min")},
-            **{"Targets Used": ("rmse_target_str", "nunique")},
-            **{"Ensemble Sizes Used": ("ensemble_size", "nunique")},
-        )
-
-        best_per_target = (
-            ranking_source.sort_values(["algorithm_type", "rmse_target_str", "metric", "ensemble_size"])
-            .groupby(["algorithm_type", "abbreviation", "Method", "family", "rmse_target_str"], as_index=False)
-            .first()[
-                [
-                    "algorithm_type",
-                    "abbreviation",
-                    "Method",
-                    "family",
-                    "rmse_target_str",
-                    "ensemble_size",
-                ]
-            ]
-        )
-
-        optimal_ensemble = best_per_target.groupby(
-            ["algorithm_type", "abbreviation", "Method", "family"], as_index=False
-        ).agg(**{"Optimal Ensemble Size": ("ensemble_size", "mean")})
-
-        scored_df = scored_df.merge(
-            optimal_ensemble,
-            on=["algorithm_type", "abbreviation", "Method", "family"],
-            how="left",
-        )
-
-        scored_df["Optimal Ensemble Size"] = scored_df["Optimal Ensemble Size"].round(2)
-        scored_df["Mean Forward Model Runs"] = scored_df["Mean Forward Model Runs"].round(4)
-        scored_df["Minimum Forward Model Runs"] = scored_df["Minimum Forward Model Runs"].round(4)
-
-        mean_runs_min = scored_df["Mean Forward Model Runs"].min()
-        mean_runs_max = scored_df["Mean Forward Model Runs"].max()
-        if mean_runs_max > mean_runs_min:
-            scored_df["mean_runs_score"] = 100.0 * (mean_runs_max - scored_df["Mean Forward Model Runs"]) / (mean_runs_max - mean_runs_min)
-        else:
-            scored_df["mean_runs_score"] = 100.0
-
-        minimum_runs_min = scored_df["Minimum Forward Model Runs"].min()
-        minimum_runs_max = scored_df["Minimum Forward Model Runs"].max()
-        if minimum_runs_max > minimum_runs_min:
-            scored_df["minimum_runs_score"] = 100.0 * (minimum_runs_max - scored_df["Minimum Forward Model Runs"]) / (minimum_runs_max - minimum_runs_min)
-        else:
-            scored_df["minimum_runs_score"] = 100.0
-
-        ens_min = scored_df["Optimal Ensemble Size"].min()
-        ens_max = scored_df["Optimal Ensemble Size"].max()
-        if ens_max > ens_min:
-            scored_df["ensemble_score"] = 100.0 * (ens_max - scored_df["Optimal Ensemble Size"]) / (ens_max - ens_min)
-        else:
-            scored_df["ensemble_score"] = 100.0
-
-        if scoring_mode == "Mean Forward Model Runs":
-            scored_df["Score"] = scored_df["mean_runs_score"]
-            sort_columns = ["Mean Forward Model Runs", "Optimal Ensemble Size", "abbreviation"]
-            ascending = [True, True, True]
-        elif scoring_mode == "Minimum Forward Model Runs":
-            scored_df["Score"] = scored_df["minimum_runs_score"]
-            sort_columns = ["Minimum Forward Model Runs", "Optimal Ensemble Size", "abbreviation"]
-            ascending = [True, True, True]
-        elif scoring_mode == "Smallest Optimal Ensemble Size":
-            scored_df["Score"] = scored_df["ensemble_score"]
-            sort_columns = ["Optimal Ensemble Size", "Mean Forward Model Runs", "abbreviation"]
-            ascending = [True, True, True]
-        else:
-            scored_df["Score"] = (
-                fwdruns_weight * scored_df["mean_runs_score"]
-                + ensemble_weight * scored_df["ensemble_score"]
-            )
-            sort_columns = ["Score", "Mean Forward Model Runs", "Optimal Ensemble Size", "abbreviation"]
-            ascending = [False, True, True, True]
-
-        scored_df = scored_df.sort_values(sort_columns, ascending=ascending).reset_index(drop=True)
-
-        if add_rank:
-            scored_df["Rank"] = scored_df.index + 1
-            scored_df["Placement"] = scored_df["Rank"].apply(
-                lambda rank: f"{ {1: '🥇', 2: '🥈', 3: '🥉'}.get(rank, '')} #{rank}".strip()
-            )
-
-        return scored_df
-
-    if selected_benchmark == "All":
-        benchmark_scores = []
-        for benchmark_name in benchmark_values:
-            benchmark_df = metric_store[metric_store["benchmark"] == benchmark_name].copy()
-            benchmark_df["rmse_target_str"] = benchmark_df["rmse_target"].astype(str)
-            scored = build_scored_table(benchmark_df, add_rank=False)
-            if scored.empty:
-                continue
-            scored["benchmark"] = benchmark_name
-            benchmark_scores.append(scored)
-
-        if benchmark_scores:
-            combined_scores = benchmark_scores[0].copy() if len(benchmark_scores) == 1 else pd.concat(benchmark_scores, ignore_index=True)
-            leaderboard_df = combined_scores.groupby(["algorithm_type", "abbreviation", "Method", "family"], as_index=False).agg(
-                Score=("Score", "mean"),
-                **{"Mean Forward Model Runs": ("Mean Forward Model Runs", "mean")},
-                **{"Minimum Forward Model Runs": ("Minimum Forward Model Runs", "mean")},
-                **{"Optimal Ensemble Size": ("Optimal Ensemble Size", "mean")},
-                **{"Targets Used": ("Targets Used", "mean")},
-                **{"Ensemble Sizes Used": ("Ensemble Sizes Used", "mean")},
-                **{"Benchmarks Used": ("benchmark", "nunique")},
-            )
-
-            leaderboard_df["Mean Forward Model Runs"] = leaderboard_df["Mean Forward Model Runs"].round(4)
-            leaderboard_df["Minimum Forward Model Runs"] = leaderboard_df["Minimum Forward Model Runs"].round(4)
-            leaderboard_df["Optimal Ensemble Size"] = leaderboard_df["Optimal Ensemble Size"].round(2)
-            leaderboard_df["Targets Used"] = leaderboard_df["Targets Used"].round().astype(int)
-            leaderboard_df["Ensemble Sizes Used"] = leaderboard_df["Ensemble Sizes Used"].round().astype(int)
-            leaderboard_df = leaderboard_df.sort_values(["Score", "Mean Forward Model Runs", "abbreviation"], ascending=[False, True, True]).reset_index(drop=True)
-            leaderboard_df["Rank"] = leaderboard_df.index + 1
-            leaderboard_df["Placement"] = leaderboard_df["Rank"].apply(
-                lambda rank: f"{ {1: '🥇', 2: '🥈', 3: '🥉'}.get(rank, '')} #{rank}".strip()
-            )
-        else:
-            leaderboard_df = pd.DataFrame(
-                columns=[
-                    "Placement",
-                    "abbreviation",
-                    "Method",
-                    "family",
-                    "Mean Forward Model Runs",
-                    "Minimum Forward Model Runs",
-                    "Score",
-                    "Optimal Ensemble Size",
-                    "Targets Used",
-                    "Ensemble Sizes Used",
-                    "Benchmarks Used",
-                ]
-            )
-    else:
-        leaderboard_df = build_scored_table(filtered, add_rank=True)
-        leaderboard_df["Benchmarks Used"] = 1
-
-    if scoring_mode == "Mean Forward Model Runs":
-        score_basis = "normalized mean of best forward model runs over selected target levels (lower is better)"
-    elif scoring_mode == "Minimum Forward Model Runs":
-        score_basis = "normalized minimum of forward model runs over selected targets and ensemble sizes (lower is better)"
-    elif scoring_mode == "Smallest Optimal Ensemble Size":
-        score_basis = "normalized mean optimal ensemble size over selected target levels (lower is better)"
-    else:
-        score_basis = (
-            "weighted blend of normalized forward-model-runs score and normalized ensemble-size score "
-            f"(forward-runs weight {fwdruns_weight:.0%}, ensemble-size weight {ensemble_weight:.0%})"
-        )
-
-    if selected_benchmark == "All":
-        score_basis = f"{score_basis}; in All mode, each method's final score is the mean of its per-benchmark scores"
-
-    if leaderboard_df.empty:
-        st.warning("No rows available for the current benchmark/target selection.")
-        st.stop()
-
-    if selected_benchmark == "All":
-        table_column_order = [
-            "Placement",
-            "abbreviation",
-            "Method",
-            "family",
-            "Score",
-            "Targets Used",
-            "Ensemble Sizes Used",
-            "Benchmarks Used",
-        ]
-    else:
-        table_column_order = [
-            "Placement",
-            "abbreviation",
-            "Method",
-            "family",
-            "Score",
-            "Mean Forward Model Runs",
-            "Minimum Forward Model Runs",
-            "Optimal Ensemble Size",
-            "Targets Used",
-            "Ensemble Sizes Used",
-            "Benchmarks Used",
-        ]
-
-    st.subheader(f"Ranked Leaderboard — {selected_benchmark}")
-    st.dataframe(
-        leaderboard_df,
-        hide_index=True,
-        width="stretch",
-        column_config={
-            "Placement": st.column_config.TextColumn("Placement"),
-            "family": st.column_config.TextColumn("Family"),
-            "Method": st.column_config.TextColumn("Method"),
-            "abbreviation": st.column_config.TextColumn("Abbrev."),
-            "Mean Forward Model Runs": st.column_config.NumberColumn("Mean Forward Model Runs", format="%.4f"),
-            "Minimum Forward Model Runs": st.column_config.NumberColumn("Minimum Forward Model Runs", format="%.4f"),
-            "Score": st.column_config.ProgressColumn("Score (0-100)", min_value=0.0, max_value=100.0, format="%.1f"),
-            "Optimal Ensemble Size": st.column_config.NumberColumn("Mean Optimal Ensemble Size", format="%.2f"),
-            "Targets Used": st.column_config.NumberColumn("Targets Used", format="%d"),
-            "Ensemble Sizes Used": st.column_config.NumberColumn("Ensemble Sizes Used", format="%d"),
-            "Benchmarks Used": st.column_config.NumberColumn("Benchmarks Used", format="%d"),
-        },
-        column_order=table_column_order,
+st.title("Calibration Benchmark")
+st.markdown(
+    "A benchmark comparing parameter-calibration methods on chaotic dynamical systems. "
+    "Methods are ranked by **forward-model run efficiency** — how many forward-model "
+    "evaluations are needed, on average across random seeds, to reach a target accuracy."
+)
+
+st.divider()
+
+col1, col2 = st.columns(2, gap="large")
+
+with col1:
+    st.subheader("📊 Optimization Leaderboard")
+    st.markdown(
+        "Ranks methods by mean forward-model runs to reach an **RMSE target** on "
+        "Lorenz-63 and Lorenz-96 benchmarks. "
+        "Lower is better; failed runs are tracked separately as a failure rate."
     )
-
-    with st.expander("Scoring & Target Controls", expanded=False):
-        st.radio(
-            "RMSE Target Level",
-            options=target_options,
-            horizontal=True,
-            key="selected_target",
-        )
-
-        st.radio(
-            "Scoring Method",
-            options=scoring_options,
-            horizontal=True,
-            key="scoring_mode",
-        )
-
-        if st.session_state.get("scoring_mode", "Mean Forward Model Runs") == "Custom Blend":
-            st.slider(
-                "Blend Weight: Forward Runs vs Ensemble Size",
-                min_value=0,
-                max_value=100,
-                step=5,
-                key="fwdruns_weight_percent",
-                help="Higher forward-runs weight prioritizes fewer model evaluations; higher ensemble-size weight prioritizes smaller ensembles.",
-            )
-
-    st.info(
-        f"Score is a normalized 0–100 ranking based on **{score_basis}**. "
-        "For Mean/Minimum forward-runs scoring, values are computed from all selected metric target levels and all ensemble sizes "
-        "after averaging over random seeds."
+    st.page_link("pages/OptimizationLeaderboard.py", label="Go to Optimization Leaderboard →")
+
+with col2:
+    st.subheader("🎯 UQ Leaderboard")
+    st.markdown(
+        "Ranks methods by mean forward-model runs to reach an **uncertainty quantification "
+        "target**. Same metric and benchmarks as the Optimization Leaderboard, evaluated "
+        "at a UQ-specific convergence criterion."
     )
-
-    if selected_benchmark != "All":
-        st.subheader("Mean Forward Model Runs vs Ensemble Size")
-        chart_source = filtered if selected_target == "All targets" else filtered[filtered["rmse_target_str"] == selected_target]
-        chart_df = chart_source.groupby(["abbreviation", "ensemble_size"], as_index=False).agg(mean_forward_runs=("metric", "mean"))
-        chart = (
-            alt.Chart(chart_df)
-            .mark_line(point=True)
-            .encode(
-                x=alt.X("ensemble_size:Q", title="Ensemble Size"),
-                y=alt.Y("mean_forward_runs:Q", title="Mean Forward Model Runs"),
-                color=alt.Color("abbreviation:N", title="Method"),
-                tooltip=["abbreviation", "ensemble_size", alt.Tooltip("mean_forward_runs:Q", format=".4f")],
-            )
-        )
-        st.altair_chart(chart, width="stretch")
-    st.caption("Top 3 are shown as podium spots; remaining methods are directly comparable via normalized score.")
-    st.page_link("pages/RawData.py", label="Open Raw Data & CSV Export", icon="🧾")
-
+    st.page_link("pages/UQLeaderboard.py", label="Go to UQ Leaderboard →")
+
+st.divider()
+
+st.subheader("Benchmarks")
+st.markdown(
+    "Results are reported on four benchmark configurations of the [Lorenz system]"
+    "(https://en.wikipedia.org/wiki/Lorenz_system), a standard testbed for "
+    "data-assimilation and calibration algorithms:"
+)
+
+_media = Path(__file__).parent / "media"
+
+st.markdown("- **L63** — Lorenz-63 — 3-variable chaotic attractor; learn 2 parameters, strongly nonlinear.")
+st.image(
+    str(_media / "posterior_ribbons_20_13_k5.png"),
+    caption="Example prior-posterior & truth. L: Difference to true parameter. R: data-sample/output predictived distribution (state-mean [1:3],  state-covariance (diag [4:6] and off-diag [7:9]))",
+    width=800,
+)
+
+
+st.markdown("- **L96** — Lorenz-96 (40-variable); learn 1-parameter constant forcing.")
+st.image(
+    str(_media / "posterior_ribbons_const-force_12_1_k3.png"),
+    caption="Example prior-posterior & truth. L: Difference to true parameter. C: parameter-induced forcing of the L96 system. R: data-sample/output predictived distribution ([1:40] state-mean [41:80] state-std)",
+    width=1200,
+)
+
+st.markdown("- **L96_SPATIAL_FORCING** — Lorenz-96 (40-variable) with spatially-varying forcing; learn 40 parameters; moderately correlated prior.")
+st.image(
+    str(_media / "posterior_ribbons_vec-force_65_1_k3.png"),
+    caption="Example prior-posterior & truth. L: Difference to true parameters. C: parameter-induced forcing of the L96 system. R: data-sample/output predictived distribution ([1:40] state-mean [41:80] state-std)",
+    width=1200,
+)
+
+
+st.markdown("- **L96_NN_FORCING** — Lorenz-96 (100-variable) with a neural-network forcing; Learn 61 parameters (weights and biases of the network). Reasonable prior given.")
+st.image(
+    str(_media / "posterior_ribbons_flux-force_80_1_k3.png"),
+    caption="Example prior-posterior & truth. L: Difference to true parameters (weights). C: parameter-induced forcing of the L96 system. R: data-sample/output predictived distribution ([1:100] state-mean [101:200] state-std)",
+    width=1200,
+)
+
+st.subheader("Method families")
+st.markdown(
+    "Methods are grouped into three families. See the **📘 Methods** page for "
+    "citations and per-method performance charts."
+)
+families = {
+    "Kalman": "Ensemble Kalman variants (TEKI, ETKI, IEKF, UKI) — update an ensemble of parameter guesses via a linearized observation operator.",
+    "Bayesian": "Sampling-based approaches (ABC, HM) — explore parameter space without requiring gradient information.",
+    "Calibrate-then-emulate": "Two-stage pipelines (CES-EKI-DMC) — use an initial calibration phase to build a cheap emulator, then sample the posterior via MCMC.",
+}
+for family, desc in families.items():
+    st.markdown(f"- **{family}** — {desc}")
+
+st.subheader("Key metric")
+st.markdown(
+    "The reported metric is the **mean number of forward-model evaluations** required "
+    "to reach the target, averaged over random seeds. "
+    "A value of **−1** indicates a failed run (did not reach the target); "
+    "the **failure rate** shows the fraction of seeds that failed."
+)