diff --git a/.gitattributes b/.gitattributes index a6344aac8c09253b3b630fb776ae94478aa0275b..3204b8f28c8b0aca3ce494d43d638356c02e0652 100644 --- a/.gitattributes +++ b/.gitattributes @@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *.zip filter=lfs diff=lfs merge=lfs -text *.zst filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text +assets/primary_rank_change_preview.png filter=lfs diff=lfs merge=lfs -text +assets/selection_regret_preview.png filter=lfs diff=lfs merge=lfs -text +assets/task_rank_map_preview.png filter=lfs diff=lfs merge=lfs -text +paper/manuscript_final.pdf filter=lfs diff=lfs merge=lfs -text +paper_outputs/figures/fig_selection_regret_scatter.pdf filter=lfs diff=lfs merge=lfs -text +paper_outputs/figures/overview_wildfire.pdf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md index 3f05c384c9f16973a8bab5bb5ffeeb81de06d84a..98e0f14d6b749a99c41922f419fca7f6220fdbbe 100644 --- a/README.md +++ b/README.md @@ -3,144 +3,146 @@ license: mit tags: - wildfire - geospatial + - weather - earth-observation - foundation-models - evaluation - - reproducibility - - paper-artifact -pretty_name: FireWx-FM and Wildfire Evaluation Contracts + - pytorch +pipeline_tag: image-segmentation +library_name: pytorch +pretty_name: WildFIRE-FM --- -# FireWx-FM and Wildfire Evaluation Contracts +# WildFIRE-FM -This repository contains the public code and lightweight artifacts for the -paper *Does Your Wildfire Prediction Model Actually Work, or Just Score Well?* +**WildFIRE-FM** is a wildfire-specialized regional reference backbone for gridded wildfire prediction and fixed-contract evaluation. It is trained on California regional weather, active-fire supervision, static fuel/canopy/exposure layers, and event-level wildfire resources used by supporting tasks in the paper. -The release has two parts: +This repository is organized as a model release first, with paper artifacts included for reproducibility. It contains the released WildFIRE-FM checkpoints, model configuration metadata, final manuscript artifacts, and scripts used to rebuild the public paper figures/tables from compact summaries. Raw data are **not** redistributed here. -- **FireWx-FM reference backbone artifacts.** FireWx-FM is a wildfire-specialized - reference model used as an in-domain comparator in the paper. -- **Fixed-contract evaluation artifacts.** The paper evaluates wildfire transfer - under fixed task, metric, matching-rule, scope, and head-family choices. +## What Is In This Release -This is a paper-artifact repository. It includes scripts, compact summary files, -paper tables, and paper figures needed to inspect and reproduce reported -outputs. It does **not** redistribute raw weather, fire, fuel, exposure, -incident, perimeter, feature-cache, or private model files. +**Model weights.** Five seeded WildFIRE-FM occupancy checkpoints are provided under `models/wildfire_fm/checkpoints/seed_*/best_firms_prauc.pt`. The checkpoints are compact U-Net PyTorch state files trained for 12-hour wildfire occupancy on a 5 km EPSG:5070 grid. -## Key Results in the Release +**Model metadata.** Training configs, run summaries, tile summaries, and checkpoint hashes are under `models/wildfire_fm/`. -The bundled paper outputs reproduce the main results reported in the manuscript. +**Evaluation artifacts.** Final paper figures, tables, and compact result summaries are under `paper_outputs/` and `artifacts/results/`. -| Check | Paper artifact | What it shows | -|---|---|---| -| Matching-rule sensitivity (RQ1) | `paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf` | The same occupancy outputs can move sharply from exact to tolerated to union \(F_1\), especially under fire-prone scopes. | -| Fixed-feature head selection (RQ2) | `paper_outputs/figures/fig_selection_regret_rq2.tikz` | Selecting a head by ranking evidence can lose decision performance relative to direct decision selection. | -| Same-contract transfer matrix (RQ3) | `paper_outputs/tables/tab_primary_results.tex` | FireWx-FM, Prithvi-WxC, Aurora, ClimaX, StormCast, DLWP, FCN, FengWu, FuXi, Pangu-Weather, and AlphaEarth are compared under fixed occupancy and spread contracts. | -| Supporting task forms (RQ4) | `paper_outputs/tables/tab_supporting_results.tex` and `paper_outputs/figures/fig_task_rank_map.pdf` | Backbone ranking changes across burned area, analog retrieval, smoke PM2.5, and extreme heat task forms. | +**Data-source notes.** The data used by the study are public or provider-hosted, but must be obtained separately. See `data_sources/DATA_SOURCES.md`. -Selected displayed values from the paper: +## Model At A Glance -- FireWx-FM reference occupancy union \(F_1\): `59.0656 ± 2.7372`. -- ClimaX occupancy union \(F_1\): `60.1506 ± 7.5865`. -- FireWx-FM fire-spread AP: `30.0900 ± 1.2500`. -- FireWx-FM smoke PM2.5 RMSE: `4.4646 ± 0.0060`. -- AlphaEarth smoke PM2.5 RMSE: `4.4403 ± 0.0488`. +| Item | WildFIRE-FM release detail | +|---|---| +| Task | 12-hour gridded wildfire occupancy prediction | +| Region/grid | California regional grid, 5 km, EPSG:5070 | +| Inputs | 16 channels: weather fields, validity masks, static fuel/canopy/exposure layers | +| Backbone | Compact U-Net with occupancy and auxiliary spatial-support heads | +| Training split | June-August 2024 train, September 2024 validation, October 2024 test | +| Seeds | 1, 7, 42, 99, 123 | +| Raw data | Not bundled; source list and access notes are documented | -Values are stored as TeX table cells under `paper_outputs/tables/`. The -corresponding compact CSV/JSON summaries are under `artifacts/results/`. +## Main Evaluation Results -## Repository Layout +The paper evaluates WildFIRE-FM and ten Earth-FM baselines under fixed task contracts. Scores below are final-paper mean ± std values. -```text -artifacts/ - manifests/ table/figure provenance metadata and SHA-256 hashes - results/ compact CSV/JSON summaries used by paper outputs -data_sources/ source list and download notes for raw data -docs/ artifact map and Hugging Face release notes -experiments/ sanitized raw-rerun scripts and Slurm templates -paper_outputs/ - figures/ PDF/TikZ figures used by the manuscript - tables/ TeX table blocks used by the manuscript -scripts/ release rebuild and audit scripts -``` +| Contract | WildFIRE-FM | Best comparator | Reading | +|---|---:|---:|---| +| Occupancy union F1 (%) | 59.0656 ± 2.7372 | 60.1506 ± 7.5865, ClimaX | Matching rule changes the apparent winner. | +| Fire-spread AP (%) | 30.0900 ± 1.2500 | 16.6221 ± 1.6965, Aurora | WildFIRE-FM leads the spread AP contract. | +| Final burned-area log-RMSE | 1.1657 ± 0.0126 | 1.3070 ± 0.0980, DLWP | Lower is better; task form matters. | +| Analog nDCG@10 | 0.5099 ± 0.0336 | 0.5086 ± 0.0440, AlphaEarth | Retrieval behaves differently from spatial decision tasks. | +| Smoke PM2.5 RMSE | 4.4646 ± 0.0060 | 4.4403 ± 0.0488, AlphaEarth | Supporting tasks do not preserve one universal ranking. | +| Extreme-heat RMSE-C | 0.2179 ± 0.0043 | 0.2194 ± 0.0039, AlphaEarth | Error metrics use a separate contract. | -## Quick Reproduction +## Visual Summary -The paper-output path uses only the Python standard library. +**Head-selection sensitivity under fixed features.** The plot shows selection regret when a head chosen by PR-AUC differs from the head chosen by the decision metric. -```bash -python3 scripts/reproduce_paper_outputs.py -``` +![Selection-regret preview](assets/selection_regret_preview.png) -This command: +**Task-form rank changes.** The supporting-task rank map shows that backbone ordering changes when the task form and metric family change. -- rebuilds the RQ1 fire-prone progression figure from summary JSON; -- rebuilds the RQ2 selection-regret TikZ figure from CSV; -- rebuilds the RQ4 rank-map PDF from the released main tables; -- checks SHA-256 hashes for all final paper outputs; -- audits that stale labels, local paths, and incomplete placeholders are absent. +![Task-rank map preview](assets/task_rank_map_preview.png) -Expected terminal tail: +**Primary-task rank changes.** The primary-task rank-change map summarizes how occupancy and spread rankings change across fixed matching rules. -```text -Paper-output checksum check passed. -Release audit passed. -Rebuilt reproducible outputs and passed release audit. +![Primary-rank-change preview](assets/primary_rank_change_preview.png) + +## Loading A Checkpoint + +The released weights are PyTorch checkpoints. The model architecture used by the checkpoints is documented in `models/wildfire_fm/modeling_unet.py` and mirrors the compact U-Net used in the paper. + +```python +import torch +from models.wildfire_fm.modeling_unet import UNetSmallFlex + +model = UNetSmallFlex( + in_ch=16, + base=32, + dropout=0.1, + norm_type="group", + norm_groups=8, + use_aux_spatial_head=True, +) +checkpoint = torch.load( + "models/wildfire_fm/checkpoints/seed_1/best_firms_prauc.pt", + map_location="cpu", +) +state = checkpoint.get("model", checkpoint) +model.load_state_dict(state) +model.eval() ``` -## What Is Included +The raw gridded inputs are not included. To run inference on new data, construct the same 16-channel grid described in `data_sources/DATA_SOURCES.md` and the paper's Section 2. + +## Data Used By The Study -- Final paper table TeX files under `paper_outputs/tables/`. -- Final paper figures under `paper_outputs/figures/`. -- Small released CSV/JSON summary artifacts under `artifacts/results/`. -- Builder scripts for reproducible paper-output figures under `scripts/`. -- Sanitized raw-rerun reference scripts under `experiments/raw_reference/`. -- Data-source documentation under `data_sources/DATA_SOURCES.md`. -- A table/figure provenance map under `docs/artifact_map.md`. +The study uses public or provider-hosted resources but does not redistribute the raw data: -## What Is Not Included +- NOAA HRRR fields for regional weather inputs. +- NASA FIRMS active-fire detections for occupancy supervision. +- LANDFIRE fuel and canopy layers for static landscape context. +- Wildfire Risk to Communities housing density and LandScan population for exposure context. +- WFIGS and MTBS event-level resources for supporting burned-area and analog tasks. +- External Earth-FM/backbone assets for comparator features. -Raw data are not bundled. The paper uses public or provider-hosted resources, -including NOAA HRRR, NASA FIRMS, LANDFIRE, Wildfire Risk to Communities, -LandScan, WFIGS, MTBS, and external Earth-FM/backbone sources. +See `data_sources/DATA_SOURCES.md` for source roles and access entry points. -See `data_sources/DATA_SOURCES.md` for the role of each source and public access -entry points. Full raw-data reruns require users to obtain those sources -independently and rebuild local feature caches. +## Repository Layout + +```text +models/wildfire_fm/ WildFIRE-FM configs, metrics, checkpoint manifest, and model code +paper/manuscript_final.pdf final paper PDF from the current release bundle +paper_outputs/ final paper figures and TeX table blocks +artifacts/results/ compact CSV/JSON summaries used by paper outputs +experiments/ sanitized rerun scripts and Slurm template +data_sources/ data-source notes; no raw data included +scripts/ paper-output rebuild and audit utilities +``` -## Reproducibility Scope +## Reproducing Released Paper Outputs -There are two levels of reproducibility: +The lightweight paper-output check uses compact released artifacts and does not require raw data or GPUs. -1. **Paper-output reproduction from bundled artifacts.** This is lightweight and - does not require raw data, GPUs, or Slurm. It verifies the exact files used by - the manuscript. -2. **Raw-data reruns.** These require separately downloaded source data, local - preprocessing, model dependencies, and compute resources. The repository - provides sanitized scripts and Slurm templates, but not the raw inputs. +```bash +python3 scripts/reproduce_paper_outputs.py +``` -## Intended Use +This rebuilds selected figures, checks hashes, and audits the release for stale labels or placeholders. Full raw-data reruns require separately downloaded source data and local feature caches. -Use this repository to inspect paper values, reproduce released figures from -summary artifacts, audit table/figure provenance, or adapt the fixed-contract -evaluation workflow for wildfire transfer studies. +## Limitations -Do not use this repository as a raw dataset mirror. Do not treat the included -summary artifacts as a substitute for the original data sources. +This release is not a raw-data mirror and is not a general global wildfire forecasting product. WildFIRE-FM is a regional reference backbone trained for the study's fixed-contract comparisons. Use outside the California regional grid requires new preprocessing, validation, and contract-specific evaluation. ## Citation -If you use this release, please cite: - ```bibtex @misc{wildfire_fm_evaluation_contracts_2026, title = {Does Your Wildfire Prediction Model Actually Work, or Just Score Well?}, - author = {Anonymous}, + author = {Yangshuang Xu and Yuyang Dai and Liling Chang and Qi Wang and Yushun Dong}, year = {2026}, - note = {FireWx-FM and fixed-contract wildfire evaluation code and artifacts} + note = {WildFIRE-FM model and fixed-contract wildfire evaluation artifacts} } ``` -The BibTeX entry will be updated with arXiv metadata after the preprint is -public. +The citation will be updated with arXiv metadata after the preprint is public. diff --git a/artifacts/manifests/paper_outputs.sha256 b/artifacts/manifests/paper_outputs.sha256 index a3813a18a1c4d96a811bd2663bcbdc61381cc95e..cc4a484f3f7d58d557c1dc742f03fe2b75cfe93b 100644 --- a/artifacts/manifests/paper_outputs.sha256 +++ b/artifacts/manifests/paper_outputs.sha256 @@ -1,8 +1,16 @@ +d0f724c709d39f9655e6e0c74b2ef51e6d7fadec6fb9d272fa605660d9416ff5 assets/primary_rank_change_preview.png +44bd6782e56031510ebb2401f42983d625294ccc3573659277876be7c158a97a assets/selection_regret_preview.png +a209caf6d8fa1d2c3be45086e397c534dbefe79176f87a8fa9b12c82b770e475 assets/task_rank_map_preview.png +c342978b2f0f25cf6e430b860702895bbb3b512145c8c6e38aa2233b416d835e paper/manuscript_final.pdf b369d13e0419fa8272ccdc994b6642f3b141248a879c030218e387c583537eb2 paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf +e3110c70c3cf8ecb8671163a401a155920266e3f907f9c6baf08e27ec6e6c410 paper_outputs/figures/fig_primary_rank_change_map.pdf +4e5b791ba4d136f722bd75a61097203836819ce9411def1caac4cc1e6d881275 paper_outputs/figures/fig_rank_heatmap1.pdf b2e56403e2774c457dd12c4685e2dc7492e22e32df46fcc5c37b3087110f2439 paper_outputs/figures/fig_selection_regret_rq2.tikz +fabb8b55aac901199cc03773741a26685becffd074f52568c93bee517c2c42c0 paper_outputs/figures/fig_selection_regret_scatter.pdf bc4d35ad9cb4c1f9ba8f31c7c340d9684c9dd2d55f5a2e60604a2b58b90cbe40 paper_outputs/figures/fig_task_contract_tiles.pdf c382f5d69f25cc2f5db174601a33d0fd0928b44910a2a4b1c131954bd42113d9 paper_outputs/figures/fig_task_rank_map.pdf 015ab951b0af5c130e4894092a5dd0bb0fd62e710467163a9df8246d8cf369f4 paper_outputs/figures/matching.pdf +7dca6ae4a9b179693802f47d24dd66734c0f332b372a7976832a0d429333b755 paper_outputs/figures/overview_wildfire.pdf e8abbd2668517f5cae14933ed943fe103e74132886b0ff48ecd1685978549504 paper_outputs/tables/tab_app_analog_rank_depth.tex 81db28aace3366625f1cfd5935892eb5af672d5ecd8327e6dcba00b7b04e2b3c paper_outputs/tables/tab_app_burned_area_median_acre.tex 4a93401ef355c02eb0cc6b2e9a1506f9ed9d912301ec6829581247e40991bdfb paper_outputs/tables/tab_app_contract_params_full.tex @@ -14,7 +22,10 @@ f740b8f076490e852efa88fa8180ca08bb6b12901ff3ec3687c7e5c0b236da4e paper_outputs/ 6850ee131e203f66392c79f17f59214672b362274f42285b252b83ac0ede1eb3 paper_outputs/tables/tab_app_seed_robustness.tex 1ca91ca451f846e59cb62ea64a616780c698b9dee80918a05467bd6c40df2dd5 paper_outputs/tables/tab_app_smoke_high_event.tex cd65372622e8dd388adb1122a3e93b22d2090fba836405b08a078d5159b182de paper_outputs/tables/tab_app_spread_ap_by_scope.tex -a31d4a4e0f2f1c7f90a5610acea77aef5a48e63c754ab2159a42473dce2c3b94 paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex -22614e90568cc562c023c540bdfdec14c0923ecf55d432fffa2619625b856092 paper_outputs/tables/tab_fireprone_contract_progression.tex +2b168c92af29ae40c324e9660d48177ea0c79e4559a3c2aa571d53043ee83b53 paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex +c822daa85e29dde4ac92b4be34f4d41040fa04da3a2674bdc4d0494dbaaceb69 paper_outputs/tables/tab_fireprone_contract_progression.tex 6672c62a150d83a351f4fa23ac04537d9aaae01af6056f689437d9b7d8bcee40 paper_outputs/tables/tab_primary_results.tex +d11d82273acb389b46c8fc1d15c1e37f1f90332ae9d1fb7b8eb5ff0f8847dc2d paper_outputs/tables/tab_selection_regret_full_head.tex +11f230e0462ded2821f3d5d45421d8b8278b61695d76799246d2e8bf873e2789 paper_outputs/tables/tab_selection_regret_scope.tex +3b1277700ececdbb4107667a5d4166a75224a84282810ec5d21bbf2ebc7fa163 paper_outputs/tables/tab_selection_regret_scope_sweep.tex 717555b2584658c936aa8fc27b63f1068dc5f796a297bcef0576cf020b3ddaf8 paper_outputs/tables/tab_supporting_results.tex diff --git a/artifacts/results/fireprone_contract_progression_summary.json b/artifacts/results/fireprone_contract_progression_summary.json index c02b01cf0b9ef58124777217fc6d8d8190cd3c0a..4c2e4ca556e789e0ef75a1386058cee7895fa068 100644 --- a/artifacts/results/fireprone_contract_progression_summary.json +++ b/artifacts/results/fireprone_contract_progression_summary.json @@ -1,7509 +1,7524 @@ { - "min_runs": 5, - "missing_or_incomplete": [], - "nondegenerate_prediction_issues": [], + "run_roots": [ + "", + "" + ], "repair_run_roots": [], + "repair_sources": [ + "", + "", + "" + ], "repair_scopes": [ "train_fire_top02pct" ], - "repair_sources": [ - "REDACTED_LOCAL_RESULT_PATH" - ], - "repair_threshold_max_ppr": 0.999999, - "repair_threshold_min_ppr": 1e-06, + "tolerance_combo": "t0_s8", + "union_combo": "t3_s8", + "threshold_selector": "strict_f1", + "threshold_min_ppr": 1e-06, + "threshold_max_ppr": 0.999999, "repair_threshold_selector": "strict_f1_nondegenerate", + "repair_threshold_min_ppr": 1e-06, + "repair_threshold_max_ppr": 0.999999, + "min_runs": 5, + "missing_or_incomplete": [], + "seed_variation_issues": [], + "nondegenerate_prediction_issues": [], "rows": [ { - "difference": 0.38916402243638726, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0006305504019789734, - "scope": "full_domain", + "label": "AlphaEarth", "seed": 1, - "source_priority": 0, - "strict_f1": 0.01841620626151013, - "threshold": 0.005, + "scope": "full_domain", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.01841620626151013, "tolerance_f1": 0.3233671389763082, + "union_f1": 0.4075802286978974, + "difference": 0.38916402243638726, + "predicted_positive_rate": 0.0006305504019789734, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4075802286978974 + "path": "", + "source_priority": 0 }, { - "difference": 0.4907581884518499, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0024215889977243494, - "scope": "train_fire_top05pct", + "label": "AlphaEarth", "seed": 1, - "source_priority": 0, - "strict_f1": 0.05801376597836775, - "threshold": 0.005, + "scope": "train_fire_top05pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.05801376597836775, "tolerance_f1": 0.45453465315678554, + "union_f1": 0.5487719544302176, + "difference": 0.4907581884518499, + "predicted_positive_rate": 0.0024215889977243494, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5487719544302176 + "path": "", + "source_priority": 0 }, { - "difference": 0.48639654658611775, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0012293459978232908, - "scope": "train_fire_top10pct", + "label": "AlphaEarth", "seed": 1, - "source_priority": 0, - "strict_f1": 0.05730937348227295, - "threshold": 0.005, + "scope": "train_fire_top10pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.05730937348227295, "tolerance_f1": 0.4503813713224908, + "union_f1": 0.5437059200683907, + "difference": 0.48639654658611775, + "predicted_positive_rate": 0.0012293459978232908, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5437059200683907 + "path": "", + "source_priority": 0 }, { - "difference": 0.4576561443425643, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0006233766233766234, - "scope": "train_fire_top20pct", + "label": "AlphaEarth", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0544280442804428, - "threshold": 0.005, + "scope": "train_fire_top20pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.0544280442804428, "tolerance_f1": 0.42554776574916503, + "union_f1": 0.5120841886230071, + "difference": 0.4576561443425643, + "predicted_positive_rate": 0.0006233766233766234, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5120841886230071 + "path": "", + "source_priority": 0 }, { - "difference": 0.4724120147971174, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0015512677798392085, - "scope": "full_domain", + "label": "AlphaEarth", "seed": 123, - "source_priority": 0, - "strict_f1": 0.015582982831138383, - "threshold": 0.005, + "scope": "full_domain", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.015582982831138383, "tolerance_f1": 0.3412019506417814, + "union_f1": 0.4879949976282558, + "difference": 0.4724120147971174, + "predicted_positive_rate": 0.0015512677798392085, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4879949976282558 + "path": "", + "source_priority": 0 }, { - "difference": 0.572037609102215, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005397249431087365, - "scope": "train_fire_top05pct", + "label": "AlphaEarth", "seed": 123, - "source_priority": 0, - "strict_f1": 0.06425702811244981, - "threshold": 0.005, + "scope": "train_fire_top05pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.06425702811244981, "tolerance_f1": 0.44689699140777084, + "union_f1": 0.6362946372146648, + "difference": 0.572037609102215, + "predicted_positive_rate": 0.005397249431087365, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6362946372146648 + "path": "", + "source_priority": 0 }, { - "difference": 0.5674882177936724, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.002728307113881468, - "scope": "train_fire_top10pct", + "label": "AlphaEarth", "seed": 123, - "source_priority": 0, - "strict_f1": 0.06358911647814125, - "threshold": 0.005, + "scope": "train_fire_top10pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.06358911647814125, "tolerance_f1": 0.4435187312985661, + "union_f1": 0.6310773342718137, + "difference": 0.5674882177936724, + "predicted_positive_rate": 0.002728307113881468, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6310773342718137 + "path": "", + "source_priority": 0 }, { - "difference": 0.5375844952591261, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0013673469387755102, - "scope": "train_fire_top20pct", + "label": "AlphaEarth", "seed": 123, - "source_priority": 0, - "strict_f1": 0.06170275882527441, - "threshold": 0.005, + "scope": "train_fire_top20pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.06170275882527441, "tolerance_f1": 0.4259185655133553, + "union_f1": 0.5992872540844005, + "difference": 0.5375844952591261, + "predicted_positive_rate": 0.0013673469387755102, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5992872540844005 + "path": "", + "source_priority": 0 }, { - "difference": 0.3681553641584426, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0005820655534941249, - "scope": "full_domain", + "label": "AlphaEarth", "seed": 42, - "source_priority": 0, - "strict_f1": 0.024598770061496925, - "threshold": 0.005, + "scope": "full_domain", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.024598770061496925, "tolerance_f1": 0.3224566312114399, + "union_f1": 0.3927541342199395, + "difference": 0.3681553641584426, + "predicted_positive_rate": 0.0005820655534941249, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3927541342199395 + "path": "", + "source_priority": 0 }, { - "difference": 0.42865099938480467, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.002226179875333927, - "scope": "train_fire_top05pct", + "label": "AlphaEarth", "seed": 42, - "source_priority": 0, - "strict_f1": 0.07774936061381074, - "threshold": 0.005, + "scope": "train_fire_top05pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.07774936061381074, "tolerance_f1": 0.46103410735073025, + "union_f1": 0.5064003599986154, + "difference": 0.42865099938480467, + "predicted_positive_rate": 0.002226179875333927, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5064003599986154 + "path": "", + "source_priority": 0 }, { - "difference": 0.42494003459095464, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00112051053725141, - "scope": "train_fire_top10pct", + "label": "AlphaEarth", "seed": 42, - "source_priority": 0, - "strict_f1": 0.07711821410451548, - "threshold": 0.005, + "scope": "train_fire_top10pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.07711821410451548, "tolerance_f1": 0.4570756272994282, + "union_f1": 0.5020582486954701, + "difference": 0.42494003459095464, + "predicted_positive_rate": 0.00112051053725141, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5020582486954701 + "path": "", + "source_priority": 0 }, { - "difference": 0.39876530350195205, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0005602968460111318, - "scope": "train_fire_top20pct", + "label": "AlphaEarth", "seed": 42, - "source_priority": 0, - "strict_f1": 0.07357212003872217, - "threshold": 0.005, + "scope": "train_fire_top20pct", "threshold_key": "0.0050", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.005, + "strict_f1": 0.07357212003872217, "tolerance_f1": 0.42986611450839224, + "union_f1": 0.4723374235406742, + "difference": 0.39876530350195205, + "predicted_positive_rate": 0.0005602968460111318, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4723374235406742 + "path": "", + "source_priority": 0 }, { - "difference": 0.1971458843464264, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00030327767470624613, - "scope": "full_domain", + "label": "AlphaEarth", "seed": 7, - "source_priority": 0, - "strict_f1": 0.018581463856786766, - "threshold": 0.02, + "scope": "full_domain", "threshold_key": "0.0200", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.02, + "strict_f1": 0.018581463856786766, "tolerance_f1": 0.19137568834619276, + "union_f1": 0.21572734820321318, + "difference": 0.1971458843464264, + "predicted_positive_rate": 0.00030327767470624613, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21572734820321318 + "path": "", + "source_priority": 0 }, { - "difference": 0.32645966955454164, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0025873157217769865, - "scope": "train_fire_top05pct", + "label": "AlphaEarth", "seed": 7, - "source_priority": 0, - "strict_f1": 0.06853879105188007, - "threshold": 0.01, + "scope": "train_fire_top05pct", "threshold_key": "0.0100", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.01, + "strict_f1": 0.06853879105188007, "tolerance_f1": 0.348489928787613, + "union_f1": 0.39499846060642174, + "difference": 0.32645966955454164, + "predicted_positive_rate": 0.0025873157217769865, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39499846060642174 + "path": "", + "source_priority": 0 }, { - "difference": 0.29747106554895125, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.001793311566241219, - "scope": "train_fire_top10pct", + "label": "AlphaEarth", "seed": 7, - "source_priority": 0, - "strict_f1": 0.05725646123260437, - "threshold": 0.01, + "scope": "train_fire_top10pct", "threshold_key": "0.0100", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.01, + "strict_f1": 0.05725646123260437, "tolerance_f1": 0.31417824074074074, + "union_f1": 0.3547275267815556, + "difference": 0.29747106554895125, + "predicted_positive_rate": 0.001793311566241219, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3547275267815556 + "path": "", + "source_priority": 0 }, { - "difference": 0.22027935343108077, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00035250463821892396, - "scope": "train_fire_top20pct", + "label": "AlphaEarth", "seed": 7, - "source_priority": 0, - "strict_f1": 0.047398843930635835, - "threshold": 0.02, + "scope": "train_fire_top20pct", "threshold_key": "0.0200", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.02, + "strict_f1": 0.047398843930635835, "tolerance_f1": 0.25536195152859176, + "union_f1": 0.2676781973617166, + "difference": 0.22027935343108077, + "predicted_positive_rate": 0.00035250463821892396, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2676781973617166 + "path": "", + "source_priority": 0 }, { - "difference": 0.3415199178431181, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00038899196042053183, - "scope": "full_domain", + "label": "AlphaEarth", "seed": 99, - "source_priority": 0, - "strict_f1": 0.02585193889541716, - "threshold": 0.01, + "scope": "full_domain", "threshold_key": "0.0100", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.01, + "strict_f1": 0.02585193889541716, "tolerance_f1": 0.29397699288019324, + "union_f1": 0.36737185673853523, + "difference": 0.3415199178431181, + "predicted_positive_rate": 0.00038899196042053183, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.36737185673853523 + "path": "", + "source_priority": 0 }, { - "difference": 0.42366919991467084, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.001496487582863362, - "scope": "train_fire_top05pct", + "label": "AlphaEarth", "seed": 99, - "source_priority": 0, - "strict_f1": 0.07710843373493977, - "threshold": 0.01, + "scope": "train_fire_top05pct", "threshold_key": "0.0100", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.01, + "strict_f1": 0.07710843373493977, "tolerance_f1": 0.4329934518241347, + "union_f1": 0.5007776336496106, + "difference": 0.42366919991467084, + "predicted_positive_rate": 0.001496487582863362, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5007776336496106 + "path": "", + "source_priority": 0 }, { - "difference": 0.42043572680093133, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0007507173246264965, - "scope": "train_fire_top10pct", + "label": "AlphaEarth", "seed": 99, - "source_priority": 0, - "strict_f1": 0.07655502392344497, - "threshold": 0.01, + "scope": "train_fire_top10pct", "threshold_key": "0.0100", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.01, + "strict_f1": 0.07655502392344497, "tolerance_f1": 0.4297532170909699, + "union_f1": 0.4969907507243763, + "difference": 0.42043572680093133, + "predicted_positive_rate": 0.0007507173246264965, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4969907507243763 + "path": "", + "source_priority": 0 }, { - "difference": 0.3953377323997304, - "label": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0003753865182436611, - "scope": "train_fire_top20pct", + "label": "AlphaEarth", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0724391624221845, - "threshold": 0.01, + "scope": "train_fire_top20pct", "threshold_key": "0.0100", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.01, + "strict_f1": 0.0724391624221845, "tolerance_f1": 0.40493051697817745, + "union_f1": 0.4677768948219149, + "difference": 0.3953377323997304, + "predicted_positive_rate": 0.0003753865182436611, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4677768948219149 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "Aurora", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 1, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 1, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 1, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "Aurora", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 123, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 123, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 123, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.1941569792865045, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9922968460111318, - "scope": "full_domain", + "label": "Aurora", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0004887421693148924, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004887421693148924, "tolerance_f1": 0.06979673814819128, + "union_f1": 0.19464572145581938, + "difference": 0.1941569792865045, + "predicted_positive_rate": 0.9922968460111318, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19464572145581938 + "path": "", + "source_priority": 0 }, { - "difference": 0.38784279977334685, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.92520035618878, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 42, - "source_priority": 0, - "strict_f1": 0.00562524160545995, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.00562524160545995, "tolerance_f1": 0.15949652359184557, + "union_f1": 0.3934680413788068, + "difference": 0.38784279977334685, + "predicted_positive_rate": 0.92520035618878, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3934680413788068 + "path": "", + "source_priority": 0 }, { - "difference": 0.2275932584637248, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.960522410210745, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0027388277045923585, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0027388277045923585, "tolerance_f1": 0.0868744819295806, + "union_f1": 0.23033208616831716, + "difference": 0.2275932584637248, + "predicted_positive_rate": 0.960522410210745, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23033208616831716 + "path": "", + "source_priority": 0 }, { - "difference": 0.16810681009431644, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9781818181818182, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0014656828060244616, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014656828060244616, "tolerance_f1": 0.06224100702898884, + "union_f1": 0.1695724929003409, + "difference": 0.16810681009431644, + "predicted_positive_rate": 0.9781818181818182, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1695724929003409 + "path": "", + "source_priority": 0 }, { - "difference": 0.1941569792865045, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9922968460111318, - "scope": "full_domain", + "label": "Aurora", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0004887421693148924, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", + "threshold": 0.0002, + "strict_f1": 0.0004887421693148924, + "tolerance_f1": 0.06979673814819128, + "union_f1": 0.19464572145581938, + "difference": 0.1941569792865045, + "predicted_positive_rate": 0.9922968460111318, "tolerance_combo": "t0_s8", - "tolerance_f1": 0.06979673814819128, "union_combo": "t3_s8", - "union_f1": 0.19464572145581938 + "path": "", + "source_priority": 0 }, { - "difference": 0.38784279977334685, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.92520035618878, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00562524160545995, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.00562524160545995, "tolerance_f1": 0.15949652359184557, + "union_f1": 0.3934680413788068, + "difference": 0.38784279977334685, + "predicted_positive_rate": 0.92520035618878, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3934680413788068 + "path": "", + "source_priority": 0 }, { - "difference": 0.2275932584637248, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.960522410210745, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0027388277045923585, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0027388277045923585, "tolerance_f1": 0.0868744819295806, + "union_f1": 0.23033208616831716, + "difference": 0.2275932584637248, + "predicted_positive_rate": 0.960522410210745, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23033208616831716 + "path": "", + "source_priority": 0 }, { - "difference": 0.16810681009431644, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9781818181818182, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0014656828060244616, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014656828060244616, "tolerance_f1": 0.06224100702898884, + "union_f1": 0.1695724929003409, + "difference": 0.16810681009431644, + "predicted_positive_rate": 0.9781818181818182, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1695724929003409 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "Aurora", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 99, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 99, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 99, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.1928199932355324, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "full_domain", + "label": "ClimaX", "seed": 1, - "source_priority": 0, - "strict_f1": 0.00048497822606044473, - "threshold": 2e-05, + "scope": "full_domain", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.00048497822606044473, "tolerance_f1": 0.06928162590128548, + "union_f1": 0.19330497146159284, + "difference": 0.1928199932355324, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19330497146159284 + "path": "", + "source_priority": 0 }, { - "difference": 0.3644918022569579, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 1, - "source_priority": 0, - "strict_f1": 0.005205570700778368, - "threshold": 2e-05, + "scope": "train_fire_top05pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.005205570700778368, "tolerance_f1": 0.14852385215852132, + "union_f1": 0.36969737295773625, + "difference": 0.3644918022569579, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.36969737295773625 + "path": "", + "source_priority": 0 }, { - "difference": 0.21975162068302262, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0026308476146364055, - "threshold": 2e-05, + "scope": "train_fire_top10pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.0026308476146364055, "tolerance_f1": 0.08362683149966108, + "union_f1": 0.22238246829765904, + "difference": 0.21975162068302262, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22238246829765904 + "path": "", + "source_priority": 0 }, { - "difference": 0.16481712333667892, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0014337271963217482, - "threshold": 2e-05, + "scope": "train_fire_top20pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.0014337271963217482, "tolerance_f1": 0.06094415351278495, + "union_f1": 0.16625085053300068, + "difference": 0.16481712333667892, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16625085053300068 + "path": "", + "source_priority": 0 }, { - "difference": 0.19494871620879223, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9878311688311688, - "scope": "full_domain", + "label": "ClimaX", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0004909510781395833, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004909510781395833, "tolerance_f1": 0.07010120797866884, + "union_f1": 0.19543966728693182, + "difference": 0.19494871620879223, + "predicted_positive_rate": 0.9878311688311688, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19543966728693182 + "path": "", + "source_priority": 0 }, { - "difference": 0.3879673384615635, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9248293262095577, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0056274920321647175, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0056274920321647175, "tolerance_f1": 0.15955540678900526, + "union_f1": 0.39359483049372823, + "difference": 0.3879673384615635, + "predicted_positive_rate": 0.9248293262095577, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39359483049372823 + "path": "", + "source_priority": 0 }, { - "difference": 0.22772278113809574, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9599040269120411, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 123, - "source_priority": 0, - "strict_f1": 0.002740589677112216, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002740589677112216, "tolerance_f1": 0.08692801516378341, + "union_f1": 0.23046337081520796, + "difference": 0.22772278113809574, + "predicted_positive_rate": 0.9599040269120411, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23046337081520796 + "path": "", + "source_priority": 0 }, { - "difference": 0.16822352141468047, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9774397031539889, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 123, - "source_priority": 0, - "strict_f1": 0.001466794800465328, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001466794800465328, "tolerance_f1": 0.06228679141188911, + "union_f1": 0.1696903162151458, + "difference": 0.16822352141468047, + "predicted_positive_rate": 0.9774397031539889, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1696903162151458 + "path": "", + "source_priority": 0 }, { - "difference": 0.19417262666147908, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9922864564007421, - "scope": "full_domain", + "label": "ClimaX", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0004887472853776956, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004887472853776956, "tolerance_f1": 0.06980139080211295, + "union_f1": 0.19466137394685679, + "difference": 0.19417262666147908, + "predicted_positive_rate": 0.9922864564007421, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19466137394685679 + "path": "", + "source_priority": 0 }, { - "difference": 0.36525822235524846, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9974176313446127, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0052190130376387305, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0052190130376387305, "tolerance_f1": 0.14887976367271333, + "union_f1": 0.3704772353928872, + "difference": 0.36525822235524846, + "predicted_positive_rate": 0.9974176313446127, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3704772353928872 + "path": "", + "source_priority": 0 }, { - "difference": 0.22041063105557585, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9966310477886613, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0026397290373912042, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0026397290373912042, "tolerance_f1": 0.08389766026250713, + "union_f1": 0.22305036009296705, + "difference": 0.22041063105557585, + "predicted_positive_rate": 0.9966310477886613, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22305036009296705 + "path": "", + "source_priority": 0 }, { - "difference": 0.1653872236463206, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9962374768089054, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0014391381051513698, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014391381051513698, "tolerance_f1": 0.061167283891925196, + "union_f1": 0.16682636175147197, + "difference": 0.1653872236463206, + "predicted_positive_rate": 0.9962374768089054, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16682636175147197 + "path": "", + "source_priority": 0 }, { - "difference": 0.19283033214709877, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9999406307977736, - "scope": "full_domain", + "label": "ClimaX", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00048500701355756607, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.00048500701355756607, "tolerance_f1": 0.06928559683931526, + "union_f1": 0.19331533916065632, + "difference": 0.19283033214709877, + "predicted_positive_rate": 0.9999406307977736, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19331533916065632 + "path": "", + "source_priority": 0 }, { - "difference": 0.3646677061475394, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9994063520332442, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 7, - "source_priority": 0, - "strict_f1": 0.005208654759994569, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005208654759994569, "tolerance_f1": 0.14860552018347153, + "union_f1": 0.36987636090753395, + "difference": 0.3646677061475394, + "predicted_positive_rate": 0.9994063520332442, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.36987636090753395 + "path": "", + "source_priority": 0 }, { - "difference": 0.2198095249514603, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9997031760166222, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0026316277173409446, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0026316277173409446, "tolerance_f1": 0.08365062280406448, + "union_f1": 0.22244115266880124, + "difference": 0.2198095249514603, + "predicted_positive_rate": 0.9997031760166222, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22244115266880124 + "path": "", + "source_priority": 0 }, { - "difference": 0.16483953803239848, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9998515769944342, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0014339398734177216, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014339398734177216, "tolerance_f1": 0.06095292465378015, + "union_f1": 0.1662734779058162, + "difference": 0.16483953803239848, + "predicted_positive_rate": 0.9998515769944342, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1662734779058162 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "ClimaX", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 99, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 99, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 99, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.1934553039285264, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9963636363636363, - "scope": "full_domain", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0004867477888159953, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004867477888159953, "tolerance_f1": 0.06952568867327341, + "union_f1": 0.19394205171734238, + "difference": 0.1934553039285264, + "predicted_positive_rate": 0.9963636363636363, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19394205171734238 + "path": "", + "source_priority": 0 }, { - "difference": 0.3644918022569579, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top05pct", "seed": 1, - "source_priority": 0, - "strict_f1": 0.005205570700778368, - "threshold": 2e-05, + "scope": "train_fire_top05pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.005205570700778368, "tolerance_f1": 0.14852385215852132, + "union_f1": 0.36969737295773625, + "difference": 0.3644918022569579, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.36969737295773625 + "path": "", + "source_priority": 0 }, { - "difference": 0.21975162068302262, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top10pct", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0026308476146364055, - "threshold": 2e-05, + "scope": "train_fire_top10pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.0026308476146364055, "tolerance_f1": 0.08362683149966108, + "union_f1": 0.22238246829765904, + "difference": 0.21975162068302262, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22238246829765904 + "path": "", + "source_priority": 0 }, { - "difference": 0.16481712333667892, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top20pct", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0014337271963217482, - "threshold": 2e-05, + "scope": "train_fire_top20pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.0014337271963217482, "tolerance_f1": 0.06094415351278495, + "union_f1": 0.16625085053300068, + "difference": 0.16481712333667892, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16625085053300068 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", "seed": 123, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", "seed": 123, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", "seed": 123, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", "seed": 42, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", "seed": 42, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", "seed": 42, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.19345790561199683, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9963487940630797, - "scope": "full_domain", "seed": 7, - "source_priority": 0, - "strict_f1": 0.000486755037982903, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.000486755037982903, "tolerance_f1": 0.06952668837151453, + "union_f1": 0.19394466064997973, + "difference": 0.19345790561199683, + "predicted_positive_rate": 0.9963487940630797, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19394466064997973 + "path": "", + "source_priority": 0 }, { - "difference": 0.36457973300001806, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9997031760166222, - "scope": "train_fire_top05pct", "seed": 7, - "source_priority": 0, - "strict_f1": 0.005207112273731229, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005207112273731229, "tolerance_f1": 0.1485646749474909, + "union_f1": 0.3697868452737493, + "difference": 0.36457973300001806, + "predicted_positive_rate": 0.9997031760166222, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3697868452737493 + "path": "", + "source_priority": 0 }, { - "difference": 0.21978056900402743, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9998515880083111, - "scope": "train_fire_top10pct", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0026312376081679546, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0026312376081679546, "tolerance_f1": 0.08363872545998453, + "union_f1": 0.22241180661219537, + "difference": 0.21978056900402743, + "predicted_positive_rate": 0.9998515880083111, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22241180661219537 + "path": "", + "source_priority": 0 }, { - "difference": 0.1648283299225597, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.999925788497217, - "scope": "train_fire_top20pct", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0014338335269832637, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014338335269832637, "tolerance_f1": 0.06094853876771749, + "union_f1": 0.16626216344954295, + "difference": 0.1648283299225597, + "predicted_positive_rate": 0.999925788497217, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16626216344954295 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", "seed": 99, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", "seed": 99, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", "seed": 99, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.1928199932355324, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 0, - "strict_f1": 0.00048497822606044473, - "threshold": 2e-05, + "scope": "full_domain", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.00048497822606044473, "tolerance_f1": 0.06928162590128548, + "union_f1": 0.19330497146159284, + "difference": 0.1928199932355324, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19330497146159284 + "path": "", + "source_priority": 0 }, { - "difference": 0.3644918022569579, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 0, - "strict_f1": 0.005205570700778368, - "threshold": 2e-05, + "scope": "train_fire_top05pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.005205570700778368, "tolerance_f1": 0.14852385215852132, + "union_f1": 0.36969737295773625, + "difference": 0.3644918022569579, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.36969737295773625 + "path": "", + "source_priority": 0 }, { - "difference": 0.21975162068302262, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0026308476146364055, - "threshold": 2e-05, + "scope": "train_fire_top10pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.0026308476146364055, "tolerance_f1": 0.08362683149966108, + "union_f1": 0.22238246829765904, + "difference": 0.21975162068302262, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22238246829765904 + "path": "", + "source_priority": 0 }, { - "difference": 0.16481712333667892, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 1.0, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0014337271963217482, - "threshold": 2e-05, + "scope": "train_fire_top20pct", "threshold_key": "0.0000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 2e-05, + "strict_f1": 0.0014337271963217482, "tolerance_f1": 0.06094415351278495, + "union_f1": 0.16625085053300068, + "difference": 0.16481712333667892, + "predicted_positive_rate": 1.0, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16625085053300068 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.19415173790531431, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9923265306122449, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0004887275525827606, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004887275525827606, "tolerance_f1": 0.06979472310089276, + "union_f1": 0.19464046545789707, + "difference": 0.19415173790531431, + "predicted_positive_rate": 0.9923265306122449, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19464046545789707 + "path": "", + "source_priority": 0 }, { - "difference": 0.38764370394898157, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9257940041555358, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 0, - "strict_f1": 0.005621644664100071, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005621644664100071, "tolerance_f1": 0.15940240082223983, + "union_f1": 0.39326534861308166, + "difference": 0.38764370394898157, + "predicted_positive_rate": 0.9257940041555358, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39326534861308166 + "path": "", + "source_priority": 0 }, { - "difference": 0.22753113988967633, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9608192341941229, - "scope": "train_fire_top10pct", - "seed": 42, - "source_priority": 0, - "strict_f1": 0.002737982762277539, - "threshold": 0.0002, + "label": "Prithvi-WxC", + "seed": 42, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002737982762277539, "tolerance_f1": 0.08684880939041824, + "union_f1": 0.23026912265195387, + "difference": 0.22753113988967633, + "predicted_positive_rate": 0.9608192341941229, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23026912265195387 + "path": "", + "source_priority": 0 }, { - "difference": 0.16808348725958006, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9783302411873841, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0014654606094294809, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014654606094294809, "tolerance_f1": 0.062231858228220235, + "union_f1": 0.16954894786900954, + "difference": 0.16808348725958006, + "predicted_positive_rate": 0.9783302411873841, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16954894786900954 + "path": "", + "source_priority": 0 }, { - "difference": 0.19479328198743862, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9887050092764379, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 0, - "strict_f1": 0.000490517270597817, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.000490517270597817, "tolerance_f1": 0.07004142068407385, + "union_f1": 0.19528379925803643, + "difference": 0.19479328198743862, + "predicted_positive_rate": 0.9887050092764379, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19528379925803643 + "path": "", + "source_priority": 0 }, { - "difference": 0.38764370394898157, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9257940041555358, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 0, - "strict_f1": 0.005621644664100071, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005621644664100071, "tolerance_f1": 0.15940240082223983, + "union_f1": 0.39326534861308166, + "difference": 0.38764370394898157, + "predicted_positive_rate": 0.9257940041555358, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39326534861308166 + "path": "", + "source_priority": 0 }, { - "difference": 0.22753113988967633, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9608192341941229, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 0, - "strict_f1": 0.002737982762277539, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002737982762277539, "tolerance_f1": 0.08684880939041824, + "union_f1": 0.23026912265195387, + "difference": 0.22753113988967633, + "predicted_positive_rate": 0.9608192341941229, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23026912265195387 + "path": "", + "source_priority": 0 }, { - "difference": 0.16808348725958006, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9783302411873841, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0014654606094294809, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014654606094294809, "tolerance_f1": 0.062231858228220235, + "union_f1": 0.16954894786900954, + "difference": 0.16808348725958006, + "predicted_positive_rate": 0.9783302411873841, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16954894786900954 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0001, + "scope": "train_fire_top05pct", "threshold_key": "0.0001", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0001, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0001, + "scope": "train_fire_top10pct", "threshold_key": "0.0001", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0001, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0001, + "scope": "train_fire_top20pct", "threshold_key": "0.0001", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0001, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.601348973225457, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.014596289424860853, - "scope": "full_domain", + "label": "Reference", "seed": 0, - "source_priority": 0, - "strict_f1": 0.006568199246490847, - "threshold": 0.95, + "scope": "full_domain", "threshold_key": "0.9500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.006568199246490847, "tolerance_f1": 0.29937890424270136, + "union_f1": 0.6079171724719479, + "difference": 0.601348973225457, + "predicted_positive_rate": 0.014596289424860853, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6079171724719479 + "path": "", + "source_priority": 0 }, { - "difference": 0.7136230759527924, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02449292569506283, - "scope": "train_fire_top05pct", + "label": "Reference", "seed": 0, - "source_priority": 0, - "strict_f1": 0.04873596787441818, - "threshold": 0.95, + "scope": "train_fire_top05pct", "threshold_key": "0.9500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.04873596787441818, "tolerance_f1": 0.40145293653054454, + "union_f1": 0.7623590438272105, + "difference": 0.7136230759527924, + "predicted_positive_rate": 0.02449292569506283, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7623590438272105 + "path": "", + "source_priority": 0 }, { - "difference": 0.7104053704635315, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.012246462847531414, - "scope": "train_fire_top10pct", + "label": "Reference", "seed": 0, - "source_priority": 0, - "strict_f1": 0.0486915291328531, - "threshold": 0.95, + "scope": "train_fire_top10pct", "threshold_key": "0.9500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.0486915291328531, "tolerance_f1": 0.4004617740449798, + "union_f1": 0.7590968995963846, + "difference": 0.7104053704635315, + "predicted_positive_rate": 0.012246462847531414, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7590968995963846 + "path": "", + "source_priority": 0 }, { - "difference": 0.6811711568125168, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.006123685837971553, - "scope": "train_fire_top20pct", + "label": "Reference", "seed": 0, - "source_priority": 0, - "strict_f1": 0.04827336828783222, - "threshold": 0.95, + "scope": "train_fire_top20pct", "threshold_key": "0.9500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.04827336828783222, "tolerance_f1": 0.39128423640663973, + "union_f1": 0.7294445251003491, + "difference": 0.6811711568125168, + "predicted_positive_rate": 0.006123685837971553, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7294445251003491 + "path": "", + "source_priority": 0 }, { - "difference": 0.1941734129578403, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9922820037105752, - "scope": "full_domain", + "label": "StormCast", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0004887494780088282, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004887494780088282, "tolerance_f1": 0.06980169309182878, + "union_f1": 0.19466216243584913, + "difference": 0.1941734129578403, + "predicted_positive_rate": 0.9922820037105752, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19466216243584913 + "path": "", + "source_priority": 0 }, { - "difference": 0.36528470808376384, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9973285841495992, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 1, - "source_priority": 0, - "strict_f1": 0.005219477804850899, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005219477804850899, "tolerance_f1": 0.14889206690983883, + "union_f1": 0.3705041858886147, + "difference": 0.36528470808376384, + "predicted_positive_rate": 0.9973285841495992, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3705041858886147 + "path": "", + "source_priority": 0 }, { - "difference": 0.2204193669035129, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9965865241911547, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 1, - "source_priority": 0, - "strict_f1": 0.0026398468145228757, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0026398468145228757, "tolerance_f1": 0.08390125124244428, + "union_f1": 0.22305921371803578, + "difference": 0.2204193669035129, + "predicted_positive_rate": 0.9965865241911547, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22305921371803578 + "path": "", + "source_priority": 0 }, { - "difference": 0.16539060875783795, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9962152133580705, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 1, - "source_priority": 0, - "strict_f1": 0.001439170243914543, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001439170243914543, "tolerance_f1": 0.06116860905264603, + "union_f1": 0.1668297790017525, + "difference": 0.16539060875783795, + "predicted_positive_rate": 0.9962152133580705, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1668297790017525 + "path": "", + "source_priority": 0 }, { - "difference": 0.19479855806395832, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9886753246753247, - "scope": "full_domain", + "label": "StormCast", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0004905319945803844, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004905319945803844, "tolerance_f1": 0.07004345000159352, + "union_f1": 0.1952890900585387, + "difference": 0.19479855806395832, + "predicted_positive_rate": 0.9886753246753247, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1952890900585387 + "path": "", + "source_priority": 0 }, { - "difference": 0.3877432263256208, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9254971801721579, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 123, - "source_priority": 0, - "strict_f1": 0.005623442559599164, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005623442559599164, "tolerance_f1": 0.15944944831691155, + "union_f1": 0.39336666888522, + "difference": 0.3877432263256208, + "predicted_positive_rate": 0.9254971801721579, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39336666888522 + "path": "", + "source_priority": 0 }, { - "difference": 0.22756219493836724, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9606708222024339, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 123, - "source_priority": 0, - "strict_f1": 0.002738405168257641, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002738405168257641, "tolerance_f1": 0.08686164376307684, + "union_f1": 0.23030060010662487, + "difference": 0.22756219493836724, + "predicted_positive_rate": 0.9606708222024339, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23030060010662487 + "path": "", + "source_priority": 0 }, { - "difference": 0.16809514786801025, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9782560296846011, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 123, - "source_priority": 0, - "strict_f1": 0.0014655716993051168, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014655716993051168, "tolerance_f1": 0.06223643229238444, + "union_f1": 0.16956071956731536, + "difference": 0.16809514786801025, + "predicted_positive_rate": 0.9782560296846011, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16956071956731536 + "path": "", + "source_priority": 0 }, { - "difference": 0.1955214345352177, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9846233766233766, - "scope": "full_domain", + "label": "StormCast", "seed": 42, - "source_priority": 0, - "strict_f1": 0.0004925501476206198, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004925501476206198, "tolerance_f1": 0.07032155974511028, + "union_f1": 0.19601398468283832, + "difference": 0.1955214345352177, + "predicted_positive_rate": 0.9846233766233766, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19601398468283832 + "path": "", + "source_priority": 0 }, { - "difference": 0.38844131515497954, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 42, - "source_priority": 0, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 0 }, { - "difference": 0.2282163144290469, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 42, - "source_priority": 0, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 0 }, { - "difference": 0.16866850620027352, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 42, - "source_priority": 0, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 0 }, { - "difference": 0.19479855806395832, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9886753246753247, - "scope": "full_domain", + "label": "StormCast", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0004905319945803844, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004905319945803844, "tolerance_f1": 0.07004345000159352, + "union_f1": 0.1952890900585387, + "difference": 0.19479855806395832, + "predicted_positive_rate": 0.9886753246753247, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1952890900585387 + "path": "", + "source_priority": 0 }, { - "difference": 0.38784279977334685, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.92520035618878, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00562524160545995, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.00562524160545995, "tolerance_f1": 0.15949652359184557, + "union_f1": 0.3934680413788068, + "difference": 0.38784279977334685, + "predicted_positive_rate": 0.92520035618878, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3934680413788068 + "path": "", + "source_priority": 0 }, { - "difference": 0.2275932584637248, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.960522410210745, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0027388277045923585, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0027388277045923585, "tolerance_f1": 0.0868744819295806, + "union_f1": 0.23033208616831716, + "difference": 0.2275932584637248, + "predicted_positive_rate": 0.960522410210745, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23033208616831716 + "path": "", + "source_priority": 0 }, { - "difference": 0.16810681009431644, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9781818181818182, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0014656828060244616, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014656828060244616, "tolerance_f1": 0.06224100702898884, + "union_f1": 0.1695724929003409, + "difference": 0.16810681009431644, + "predicted_positive_rate": 0.9781818181818182, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1695724929003409 + "path": "", + "source_priority": 0 }, { - "difference": 0.19415960008322136, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9922820037105752, - "scope": "full_domain", + "label": "StormCast", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0004887494780088282, - "threshold": 0.0002, + "scope": "full_domain", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0004887494780088282, "tolerance_f1": 0.06979774571547358, + "union_f1": 0.1946483495612302, + "difference": 0.19415960008322136, + "predicted_positive_rate": 0.9922820037105752, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1946483495612302 + "path": "", + "source_priority": 0 }, { - "difference": 0.3877432263256208, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9254971801721579, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 99, - "source_priority": 0, - "strict_f1": 0.005623442559599164, - "threshold": 0.0002, + "scope": "train_fire_top05pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.005623442559599164, "tolerance_f1": 0.15944944831691155, + "union_f1": 0.39336666888522, + "difference": 0.3877432263256208, + "predicted_positive_rate": 0.9254971801721579, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39336666888522 + "path": "", + "source_priority": 0 }, { - "difference": 0.22756219493836724, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9606708222024339, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 99, - "source_priority": 0, - "strict_f1": 0.002738405168257641, - "threshold": 0.0002, + "scope": "train_fire_top10pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.002738405168257641, "tolerance_f1": 0.08686164376307684, + "union_f1": 0.23030060010662487, + "difference": 0.22756219493836724, + "predicted_positive_rate": 0.9606708222024339, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23030060010662487 + "path": "", + "source_priority": 0 }, { - "difference": 0.16809514786801025, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9782560296846011, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 99, - "source_priority": 0, - "strict_f1": 0.0014655716993051168, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014655716993051168, "tolerance_f1": 0.06223643229238444, + "union_f1": 0.16956071956731536, + "difference": 0.16809514786801025, + "predicted_positive_rate": 0.9782560296846011, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16956071956731536 + "path": "", + "source_priority": 0 }, { - "difference": 0.3044358568583716, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.057648732220160794, - "scope": "full_domain", + "label": "DLWP", "seed": 17, - "source_priority": 0, - "strict_f1": 0.001786130143937306, - "threshold": 0.08, + "scope": "full_domain", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.001786130143937306, "tolerance_f1": 0.16383974771641738, + "union_f1": 0.3062219870023089, + "difference": 0.3044358568583716, + "predicted_positive_rate": 0.057648732220160794, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3062219870023089 + "path": "", + "source_priority": 0 }, { - "difference": 0.5668021940668209, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.05733155238943306, - "scope": "train_fire_top05pct", + "label": "DLWP", "seed": 17, - "source_priority": 0, - "strict_f1": 0.01972516815912186, - "threshold": 0.08, + "scope": "train_fire_top05pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.01972516815912186, "tolerance_f1": 0.3500819147736832, + "union_f1": 0.5865273622259427, + "difference": 0.5668021940668209, + "predicted_positive_rate": 0.05733155238943306, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5865273622259427 + "path": "", + "source_priority": 0 }, { - "difference": 0.5537110636994692, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02979989116453943, - "scope": "train_fire_top10pct", + "label": "DLWP", "seed": 17, - "source_priority": 0, - "strict_f1": 0.018998410174880763, - "threshold": 0.08, + "scope": "train_fire_top10pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.018998410174880763, "tolerance_f1": 0.3409571337826908, + "union_f1": 0.57270947387435, + "difference": 0.5537110636994692, + "predicted_positive_rate": 0.02979989116453943, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.57270947387435 + "path": "", + "source_priority": 0 }, { - "difference": 0.44734760227295295, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.010166975881261595, - "scope": "train_fire_top20pct", + "label": "DLWP", "seed": 17, - "source_priority": 0, - "strict_f1": 0.01931818181818182, - "threshold": 0.1, + "scope": "train_fire_top20pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.01931818181818182, "tolerance_f1": 0.28953047944835925, + "union_f1": 0.4666657840911348, + "difference": 0.44734760227295295, + "predicted_positive_rate": 0.010166975881261595, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4666657840911348 + "path": "", + "source_priority": 0 }, { - "difference": 0.2924864697969017, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04519196042053185, - "scope": "full_domain", + "label": "DLWP", "seed": 37, - "source_priority": 0, - "strict_f1": 0.0019600476944939, - "threshold": 0.1, + "scope": "full_domain", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.0019600476944939, "tolerance_f1": 0.17299721297318826, + "union_f1": 0.2944465174913956, + "difference": 0.2924864697969017, + "predicted_positive_rate": 0.04519196042053185, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2944465174913956 + "path": "", + "source_priority": 0 }, { - "difference": 0.5053802602935323, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04792965271593945, - "scope": "train_fire_top05pct", + "label": "DLWP", "seed": 37, - "source_priority": 0, - "strict_f1": 0.024960845732184808, - "threshold": 0.1, + "scope": "train_fire_top05pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.024960845732184808, "tolerance_f1": 0.35323298220195054, + "union_f1": 0.5303411060257172, + "difference": 0.5053802602935323, + "predicted_positive_rate": 0.04792965271593945, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5303411060257172 + "path": "", + "source_priority": 0 }, { - "difference": 0.486215877804802, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.025880577817354308, - "scope": "train_fire_top10pct", + "label": "DLWP", "seed": 37, - "source_priority": 0, - "strict_f1": 0.023191305534082125, - "threshold": 0.1, + "scope": "train_fire_top10pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.023191305534082125, "tolerance_f1": 0.33689092573275314, + "union_f1": 0.5094071833388841, + "difference": 0.486215877804802, + "predicted_positive_rate": 0.025880577817354308, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5094071833388841 + "path": "", + "source_priority": 0 }, { - "difference": 0.26320327785336994, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0027860235003092145, - "scope": "train_fire_top20pct", + "label": "DLWP", "seed": 37, - "source_priority": 0, - "strict_f1": 0.028949691085613415, - "threshold": 0.2, + "scope": "train_fire_top20pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.028949691085613415, "tolerance_f1": 0.20397124965652888, + "union_f1": 0.29215296893898335, + "difference": 0.26320327785336994, + "predicted_positive_rate": 0.0027860235003092145, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.29215296893898335 + "path": "", + "source_priority": 0 }, { - "difference": 0.17556822794728488, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02625751391465677, - "scope": "full_domain", + "label": "DLWP", "seed": 57, - "source_priority": 0, - "strict_f1": 0.001008153911497155, - "threshold": 0.1, + "scope": "full_domain", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.001008153911497155, "tolerance_f1": 0.09460750931776095, + "union_f1": 0.17657638185878205, + "difference": 0.17556822794728488, + "predicted_positive_rate": 0.02625751391465677, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17657638185878205 + "path": "", + "source_priority": 0 }, { - "difference": 0.5636022889410879, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.1306866528148808, - "scope": "train_fire_top05pct", + "label": "DLWP", "seed": 57, - "source_priority": 0, - "strict_f1": 0.017035016422646553, - "threshold": 0.05, + "scope": "train_fire_top05pct", "threshold_key": "0.0500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.05, + "strict_f1": 0.017035016422646553, "tolerance_f1": 0.3101459291213085, + "union_f1": 0.5806373053637345, + "difference": 0.5636022889410879, + "predicted_positive_rate": 0.1306866528148808, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5806373053637345 + "path": "", + "source_priority": 0 }, { - "difference": 0.3946007462885974, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02746487582863362, - "scope": "train_fire_top10pct", + "label": "DLWP", "seed": 57, - "source_priority": 0, - "strict_f1": 0.018477139910622206, - "threshold": 0.08, + "scope": "train_fire_top10pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.018477139910622206, "tolerance_f1": 0.2606560686639405, + "union_f1": 0.41307788619921965, + "difference": 0.3946007462885974, + "predicted_positive_rate": 0.02746487582863362, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.41307788619921965 + "path": "", + "source_priority": 0 }, { - "difference": 0.30932859219900066, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.021262213976499692, - "scope": "train_fire_top20pct", + "label": "DLWP", "seed": 57, - "source_priority": 0, - "strict_f1": 0.01226752201682564, - "threshold": 0.08, + "scope": "train_fire_top20pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.01226752201682564, "tolerance_f1": 0.19661692001462533, + "union_f1": 0.3215961142158263, + "difference": 0.30932859219900066, + "predicted_positive_rate": 0.021262213976499692, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3215961142158263 + "path": "", + "source_priority": 0 }, { - "difference": 0.26339192421292845, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0589282622139765, - "scope": "full_domain", + "label": "DLWP", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0016304484151205278, - "threshold": 0.08, + "scope": "full_domain", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.0016304484151205278, "tolerance_f1": 0.14442504878529014, + "union_f1": 0.265022372628049, + "difference": 0.26339192421292845, + "predicted_positive_rate": 0.0589282622139765, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.265022372628049 + "path": "", + "source_priority": 0 }, { - "difference": 0.45626799482364494, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.06984763035519936, - "scope": "train_fire_top05pct", + "label": "DLWP", "seed": 7, - "source_priority": 0, - "strict_f1": 0.016864097224592905, - "threshold": 0.08, + "scope": "train_fire_top05pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.016864097224592905, "tolerance_f1": 0.28520211660517514, + "union_f1": 0.47313209204823786, + "difference": 0.45626799482364494, + "predicted_positive_rate": 0.06984763035519936, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.47313209204823786 + "path": "", + "source_priority": 0 }, { - "difference": 0.3591438068453628, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.052691204115959234, - "scope": "train_fire_top10pct", + "label": "DLWP", "seed": 7, - "source_priority": 0, - "strict_f1": 0.011312372621310312, - "threshold": 0.08, + "scope": "train_fire_top10pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.011312372621310312, "tolerance_f1": 0.2160234001333052, + "union_f1": 0.3704561794666731, + "difference": 0.3591438068453628, + "predicted_positive_rate": 0.052691204115959234, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3704561794666731 + "path": "", + "source_priority": 0 }, { - "difference": 0.2695667418377301, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04168893011750155, - "scope": "train_fire_top20pct", + "label": "DLWP", "seed": 7, - "source_priority": 0, - "strict_f1": 0.007379212786746584, - "threshold": 0.08, + "scope": "train_fire_top20pct", "threshold_key": "0.0800", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.08, + "strict_f1": 0.007379212786746584, "tolerance_f1": 0.1600767886426284, + "union_f1": 0.2769459546244767, + "difference": 0.2695667418377301, + "predicted_positive_rate": 0.04168893011750155, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2769459546244767 + "path": "", + "source_priority": 0 }, { - "difference": 0.36515720594022893, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.028077427334570193, - "scope": "full_domain", + "label": "DLWP", "seed": 97, - "source_priority": 0, - "strict_f1": 0.002078902199880332, - "threshold": 0.1, + "scope": "full_domain", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.002078902199880332, "tolerance_f1": 0.16987085515333694, + "union_f1": 0.36723610814010926, + "difference": 0.36515720594022893, + "predicted_positive_rate": 0.028077427334570193, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.36723610814010926 + "path": "", + "source_priority": 0 }, { - "difference": 0.5906566149800531, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.16420797467102008, - "scope": "train_fire_top05pct", + "label": "DLWP", "seed": 97, - "source_priority": 0, - "strict_f1": 0.011684287006420426, - "threshold": 0.05, + "scope": "train_fire_top05pct", "threshold_key": "0.0500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.05, + "strict_f1": 0.011684287006420426, "tolerance_f1": 0.2874938355126855, + "union_f1": 0.6023409019864735, + "difference": 0.5906566149800531, + "predicted_positive_rate": 0.16420797467102008, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6023409019864735 + "path": "", + "source_priority": 0 }, { - "difference": 0.48212021492719026, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.11266572672405263, - "scope": "train_fire_top10pct", + "label": "DLWP", "seed": 97, - "source_priority": 0, - "strict_f1": 0.008571862589787548, - "threshold": 0.05, + "scope": "train_fire_top10pct", "threshold_key": "0.0500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.05, + "strict_f1": 0.008571862589787548, "tolerance_f1": 0.22837595071441363, + "union_f1": 0.4906920775169778, + "difference": 0.48212021492719026, + "predicted_positive_rate": 0.11266572672405263, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4906920775169778 + "path": "", + "source_priority": 0 }, { - "difference": 0.38082140073143794, - "label": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02008596165739023, - "scope": "train_fire_top20pct", + "label": "DLWP", "seed": 97, - "source_priority": 0, - "strict_f1": 0.008323671928416423, - "threshold": 0.1, + "scope": "train_fire_top20pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.008323671928416423, "tolerance_f1": 0.19681923783547378, + "union_f1": 0.3891450726598544, + "difference": 0.38082140073143794, + "predicted_positive_rate": 0.02008596165739023, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3891450726598544 + "path": "", + "source_priority": 0 }, { - "difference": 0.3840989936961055, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.03192430426716141, - "scope": "full_domain", + "label": "FCN", "seed": 17, - "source_priority": 0, - "strict_f1": 0.002068681772914111, - "threshold": 0.15, + "scope": "full_domain", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.002068681772914111, "tolerance_f1": 0.19094441931639336, + "union_f1": 0.38616767546901964, + "difference": 0.3840989936961055, + "predicted_positive_rate": 0.03192430426716141, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.38616767546901964 + "path": "", + "source_priority": 0 }, { - "difference": 0.46051807014713964, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.06528643514395963, - "scope": "train_fire_top05pct", + "label": "FCN", "seed": 17, - "source_priority": 0, - "strict_f1": 0.012095158293562608, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.012095158293562608, "tolerance_f1": 0.25572856051368525, + "union_f1": 0.47261322844070225, + "difference": 0.46051807014713964, + "predicted_positive_rate": 0.06528643514395963, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.47261322844070225 + "path": "", + "source_priority": 0 }, { - "difference": 0.30442339402679647, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.03344711586029484, - "scope": "train_fire_top10pct", + "label": "FCN", "seed": 17, - "source_priority": 0, - "strict_f1": 0.006403642961329112, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.006403642961329112, "tolerance_f1": 0.16176542623012913, + "union_f1": 0.3108270369881256, + "difference": 0.30442339402679647, + "predicted_positive_rate": 0.03344711586029484, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3108270369881256 + "path": "", + "source_priority": 0 }, { - "difference": 0.2121721179401987, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.01035930735930736, - "scope": "train_fire_top20pct", + "label": "FCN", "seed": 17, - "source_priority": 0, - "strict_f1": 0.004578192172408018, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.004578192172408018, "tolerance_f1": 0.10554731176319206, + "union_f1": 0.2167503101126067, + "difference": 0.2121721179401987, + "predicted_positive_rate": 0.01035930735930736, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2167503101126067 + "path": "", + "source_priority": 0 }, { - "difference": 0.33775307067336774, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.009495980210265924, - "scope": "full_domain", + "label": "FCN", "seed": 37, - "source_priority": 0, - "strict_f1": 0.00208290997764682, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.00208290997764682, "tolerance_f1": 0.16485748333864078, + "union_f1": 0.33983598065101456, + "difference": 0.33775307067336774, + "predicted_positive_rate": 0.009495980210265924, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.33983598065101456 + "path": "", + "source_priority": 0 }, { - "difference": 0.48713449736377723, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.01251113089937667, - "scope": "train_fire_top05pct", + "label": "FCN", "seed": 37, - "source_priority": 0, - "strict_f1": 0.024537870112874206, - "threshold": 0.2, + "scope": "train_fire_top05pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.024537870112874206, "tolerance_f1": 0.27999632336313063, + "union_f1": 0.5116723674766515, + "difference": 0.48713449736377723, + "predicted_positive_rate": 0.01251113089937667, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5116723674766515 + "path": "", + "source_priority": 0 }, { - "difference": 0.4238672329193606, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00802908875037103, - "scope": "train_fire_top10pct", + "label": "FCN", "seed": 37, - "source_priority": 0, - "strict_f1": 0.019849146486701073, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.019849146486701073, "tolerance_f1": 0.23420382391023928, + "union_f1": 0.4437163794060617, + "difference": 0.4238672329193606, + "predicted_positive_rate": 0.00802908875037103, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4437163794060617 + "path": "", + "source_priority": 0 }, { - "difference": 0.3647698181559645, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005085961657390229, - "scope": "train_fire_top20pct", + "label": "FCN", "seed": 37, - "source_priority": 0, - "strict_f1": 0.016197783461210574, - "threshold": 0.2, + "scope": "train_fire_top20pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.016197783461210574, "tolerance_f1": 0.19512358967789506, + "union_f1": 0.3809676016171751, + "difference": 0.3647698181559645, + "predicted_positive_rate": 0.005085961657390229, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3809676016171751 + "path": "", + "source_priority": 0 }, { - "difference": 0.2830040701913329, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0016064316635745208, - "scope": "full_domain", + "label": "FCN", "seed": 57, - "source_priority": 0, - "strict_f1": 0.0033447053314602988, - "threshold": 0.5, + "scope": "full_domain", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.0033447053314602988, "tolerance_f1": 0.16083569292964747, + "union_f1": 0.2863487755227932, + "difference": 0.2830040701913329, + "predicted_positive_rate": 0.0016064316635745208, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2863487755227932 + "path": "", + "source_priority": 0 }, { - "difference": 0.4820839133891601, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0197734243593549, - "scope": "train_fire_top05pct", + "label": "FCN", "seed": 57, - "source_priority": 0, - "strict_f1": 0.01569234169521494, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.01569234169521494, "tolerance_f1": 0.300679830289768, + "union_f1": 0.49777625508437506, + "difference": 0.4820839133891601, + "predicted_positive_rate": 0.0197734243593549, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.49777625508437506 + "path": "", + "source_priority": 0 }, { - "difference": 0.387627119265477, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.015055159790244386, - "scope": "train_fire_top10pct", + "label": "FCN", "seed": 57, - "source_priority": 0, - "strict_f1": 0.010726695875509895, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.010726695875509895, "tolerance_f1": 0.2256264421482596, + "union_f1": 0.3983538151409869, + "difference": 0.387627119265477, + "predicted_positive_rate": 0.015055159790244386, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3983538151409869 + "path": "", + "source_priority": 0 }, { - "difference": 0.3250188323968997, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.010080395794681509, - "scope": "train_fire_top20pct", + "label": "FCN", "seed": 57, - "source_priority": 0, - "strict_f1": 0.00859106529209622, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.00859106529209622, "tolerance_f1": 0.18647162974553239, + "union_f1": 0.33360989768899596, + "difference": 0.3250188323968997, + "predicted_positive_rate": 0.010080395794681509, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.33360989768899596 + "path": "", + "source_priority": 0 }, { - "difference": 0.4822930267328773, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.027048113790970933, - "scope": "full_domain", + "label": "FCN", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0026467855605157606, - "threshold": 0.15, + "scope": "full_domain", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.0026467855605157606, "tolerance_f1": 0.23211789308126832, + "union_f1": 0.48493981229339306, + "difference": 0.4822930267328773, + "predicted_positive_rate": 0.027048113790970933, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.48493981229339306 + "path": "", + "source_priority": 0 }, { - "difference": 0.5645980289087711, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.06537300880577818, - "scope": "train_fire_top05pct", + "label": "FCN", "seed": 7, - "source_priority": 0, - "strict_f1": 0.01222529471692621, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.01222529471692621, "tolerance_f1": 0.3028035236186743, + "union_f1": 0.5768233236256973, + "difference": 0.5645980289087711, + "predicted_positive_rate": 0.06537300880577818, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5768233236256973 + "path": "", + "source_priority": 0 }, { - "difference": 0.4452328295737776, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.030024982685267636, - "scope": "train_fire_top10pct", + "label": "FCN", "seed": 7, - "source_priority": 0, - "strict_f1": 0.008918001736248123, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.008918001736248123, "tolerance_f1": 0.2270292295163525, + "union_f1": 0.4541508313100257, + "difference": 0.4452328295737776, + "predicted_positive_rate": 0.030024982685267636, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4541508313100257 + "path": "", + "source_priority": 0 }, { - "difference": 0.3226698082083355, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.009654916512059368, - "scope": "train_fire_top20pct", + "label": "FCN", "seed": 7, - "source_priority": 0, - "strict_f1": 0.008704984497972811, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.008704984497972811, "tolerance_f1": 0.1596269732904185, + "union_f1": 0.3313747927063083, + "difference": 0.3226698082083355, + "predicted_positive_rate": 0.009654916512059368, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3313747927063083 + "path": "", + "source_priority": 0 }, { - "difference": 0.5017259192233794, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005936920222634508, - "scope": "full_domain", + "label": "FCN", "seed": 97, - "source_priority": 0, - "strict_f1": 0.00400312243549969, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.00400312243549969, "tolerance_f1": 0.22655113161542761, + "union_f1": 0.5057290416588791, + "difference": 0.5017259192233794, + "predicted_positive_rate": 0.005936920222634508, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5057290416588791 + "path": "", + "source_priority": 0 }, { - "difference": 0.6396713821034217, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.07633570792520035, - "scope": "train_fire_top05pct", + "label": "FCN", "seed": 97, - "source_priority": 0, - "strict_f1": 0.016606090988845718, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.016606090988845718, "tolerance_f1": 0.3296389749733903, + "union_f1": 0.6562774730922674, + "difference": 0.6396713821034217, + "predicted_positive_rate": 0.07633570792520035, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6562774730922674 + "path": "", + "source_priority": 0 }, { - "difference": 0.5525178187961205, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.049730384881765115, - "scope": "train_fire_top10pct", + "label": "FCN", "seed": 97, - "source_priority": 0, - "strict_f1": 0.012986069049061175, - "threshold": 0.15, + "scope": "train_fire_top10pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.012986069049061175, "tolerance_f1": 0.2724598040570218, + "union_f1": 0.5655038878451817, + "difference": 0.5525178187961205, + "predicted_positive_rate": 0.049730384881765115, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5655038878451817 + "path": "", + "source_priority": 0 }, { - "difference": 0.4298567373898511, - "label": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.006658627087198516, - "scope": "train_fire_top20pct", + "label": "FCN", "seed": 97, - "source_priority": 0, - "strict_f1": 0.011738073279114613, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.011738073279114613, "tolerance_f1": 0.20219147661108805, + "union_f1": 0.4415948106689657, + "difference": 0.4298567373898511, + "predicted_positive_rate": 0.006658627087198516, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4415948106689657 + "path": "", + "source_priority": 0 }, { - "difference": 0.2798111779709097, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004433395176252319, - "scope": "full_domain", + "label": "FengWu", "seed": 17, - "source_priority": 0, - "strict_f1": 0.003438698584843275, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.003438698584843275, "tolerance_f1": 0.1569315132056949, + "union_f1": 0.283249876555753, + "difference": 0.2798111779709097, + "predicted_positive_rate": 0.004433395176252319, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.283249876555753 + "path": "", + "source_priority": 0 }, { - "difference": 0.31898912374476807, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.008098347679825863, - "scope": "train_fire_top05pct", + "label": "FengWu", "seed": 17, - "source_priority": 0, - "strict_f1": 0.01848001848001848, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.01848001848001848, "tolerance_f1": 0.21207047330588913, + "union_f1": 0.3374691422247865, + "difference": 0.31898912374476807, + "predicted_positive_rate": 0.008098347679825863, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3374691422247865 + "path": "", + "source_priority": 0 }, { - "difference": 0.28149494747245746, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005720045513010785, - "scope": "train_fire_top10pct", + "label": "FengWu", "seed": 17, - "source_priority": 0, - "strict_f1": 0.014059753954305799, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.014059753954305799, "tolerance_f1": 0.17969557482138315, + "union_f1": 0.29555470142676327, + "difference": 0.28149494747245746, + "predicted_positive_rate": 0.005720045513010785, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.29555470142676327 + "path": "", + "source_priority": 0 }, { - "difference": 0.2536326799641239, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0035126777983920843, - "scope": "train_fire_top20pct", + "label": "FengWu", "seed": 17, - "source_priority": 0, - "strict_f1": 0.011695906432748539, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.011695906432748539, "tolerance_f1": 0.1579992352547558, + "union_f1": 0.26532858639687246, + "difference": 0.2536326799641239, + "predicted_positive_rate": 0.0035126777983920843, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.26532858639687246 + "path": "", + "source_priority": 0 }, { - "difference": 0.2236424973966951, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0023098330241187385, - "scope": "full_domain", + "label": "FengWu", "seed": 37, - "source_priority": 0, - "strict_f1": 0.0018414421399496025, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.0018414421399496025, "tolerance_f1": 0.10327893388535589, + "union_f1": 0.2254839395366447, + "difference": 0.2236424973966951, + "predicted_positive_rate": 0.0023098330241187385, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2254839395366447 + "path": "", + "source_priority": 0 }, { - "difference": 0.26699212025153246, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0032452755515979025, - "scope": "train_fire_top05pct", + "label": "FengWu", "seed": 37, - "source_priority": 0, - "strict_f1": 0.01267427122940431, - "threshold": 0.2, + "scope": "train_fire_top05pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.01267427122940431, "tolerance_f1": 0.13288915578187116, + "union_f1": 0.27966639148093675, + "difference": 0.26699212025153246, + "predicted_positive_rate": 0.0032452755515979025, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.27966639148093675 + "path": "", + "source_priority": 0 }, { - "difference": 0.24813584863600618, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0019738794894627485, - "scope": "train_fire_top10pct", + "label": "FengWu", "seed": 37, - "source_priority": 0, - "strict_f1": 0.011273957158962795, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.011273957158962795, "tolerance_f1": 0.1191832198916516, + "union_f1": 0.259409805794969, + "difference": 0.24813584863600618, + "predicted_positive_rate": 0.0019738794894627485, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.259409805794969 + "path": "", + "source_priority": 0 }, { - "difference": 0.2344501305987902, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0010810142238713667, - "scope": "train_fire_top20pct", + "label": "FengWu", "seed": 37, - "source_priority": 0, - "strict_f1": 0.010316368638239341, - "threshold": 0.2, + "scope": "train_fire_top20pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.010316368638239341, "tolerance_f1": 0.11041562343515275, + "union_f1": 0.24476649923702953, + "difference": 0.2344501305987902, + "predicted_positive_rate": 0.0010810142238713667, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.24476649923702953 + "path": "", + "source_priority": 0 }, { - "difference": 0.06735555938074993, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00042881880024737165, - "scope": "full_domain", + "label": "FengWu", "seed": 57, - "source_priority": 0, - "strict_f1": 0.001842299189388357, - "threshold": 0.5, + "scope": "full_domain", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.001842299189388357, "tolerance_f1": 0.042906126190853267, + "union_f1": 0.06919785857013828, + "difference": 0.06735555938074993, + "predicted_positive_rate": 0.00042881880024737165, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.06919785857013828 + "path": "", + "source_priority": 0 }, { - "difference": 0.3168732218355133, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005567923221529633, - "scope": "train_fire_top05pct", + "label": "FengWu", "seed": 57, - "source_priority": 0, - "strict_f1": 0.0205686630369026, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.0205686630369026, "tolerance_f1": 0.1866312847963942, + "union_f1": 0.3374418848724159, + "difference": 0.3168732218355133, + "predicted_positive_rate": 0.005567923221529633, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3374418848724159 + "path": "", + "source_priority": 0 }, { - "difference": 0.3141659158267793, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0028556940734144652, - "scope": "train_fire_top10pct", + "label": "FengWu", "seed": 57, - "source_priority": 0, - "strict_f1": 0.02015411973918198, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.02015411973918198, "tolerance_f1": 0.1840127014362662, + "union_f1": 0.33432003556596124, + "difference": 0.3141659158267793, + "predicted_positive_rate": 0.0028556940734144652, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.33432003556596124 + "path": "", + "source_priority": 0 }, { - "difference": 0.2991917676543021, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0015157699443413728, - "scope": "train_fire_top20pct", + "label": "FengWu", "seed": 57, - "source_priority": 0, - "strict_f1": 0.018831348656881748, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.018831348656881748, "tolerance_f1": 0.17388386719173746, + "union_f1": 0.31802311631118385, + "difference": 0.2991917676543021, + "predicted_positive_rate": 0.0015157699443413728, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.31802311631118385 + "path": "", + "source_priority": 0 }, { - "difference": 0.18397565646547948, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0016931354359925788, - "scope": "full_domain", + "label": "FengWu", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00268370607028754, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.00268370607028754, "tolerance_f1": 0.09710793321083327, + "union_f1": 0.186659362535767, + "difference": 0.18397565646547948, + "predicted_positive_rate": 0.0016931354359925788, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.186659362535767 + "path": "", + "source_priority": 0 }, { - "difference": 0.20932175095916816, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0030746017611556348, - "scope": "train_fire_top05pct", + "label": "FengWu", "seed": 7, - "source_priority": 0, - "strict_f1": 0.013054830287206266, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.013054830287206266, "tolerance_f1": 0.12305221580797777, + "union_f1": 0.22237658124637444, + "difference": 0.20932175095916816, + "predicted_positive_rate": 0.0030746017611556348, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22237658124637444 + "path": "", + "source_priority": 0 }, { - "difference": 0.19895488431057093, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0019998515880083113, - "scope": "train_fire_top10pct", + "label": "FengWu", "seed": 7, - "source_priority": 0, - "strict_f1": 0.011185682326621925, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.011185682326621925, "tolerance_f1": 0.11778287307219655, + "union_f1": 0.21014056663719285, + "difference": 0.19895488431057093, + "predicted_positive_rate": 0.0019998515880083113, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21014056663719285 + "path": "", + "source_priority": 0 }, { - "difference": 0.19412591498868878, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0010822510822510823, - "scope": "train_fire_top20pct", + "label": "FengWu", "seed": 7, - "source_priority": 0, - "strict_f1": 0.010309278350515464, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.010309278350515464, "tolerance_f1": 0.11130870584431331, + "union_f1": 0.20443519333920424, + "difference": 0.19412591498868878, + "predicted_positive_rate": 0.0010822510822510823, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.20443519333920424 + "path": "", + "source_priority": 0 }, { - "difference": 0.43726313068468925, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.012518243661100804, - "scope": "full_domain", + "label": "FengWu", "seed": 97, - "source_priority": 0, - "strict_f1": 0.0032567291196169464, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.0032567291196169464, "tolerance_f1": 0.20002472134774485, + "union_f1": 0.4405198598043062, + "difference": 0.43726313068468925, + "predicted_positive_rate": 0.012518243661100804, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4405198598043062 + "path": "", + "source_priority": 0 }, { - "difference": 0.3146235231202003, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.010032650638171565, - "scope": "train_fire_top05pct", + "label": "FengWu", "seed": 97, - "source_priority": 0, - "strict_f1": 0.013695949911954608, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", + "threshold": 0.3, + "strict_f1": 0.013695949911954608, + "tolerance_f1": 0.15917085028122405, + "union_f1": 0.3283194730321549, + "difference": 0.3146235231202003, + "predicted_positive_rate": 0.010032650638171565, "tolerance_combo": "t0_s8", - "tolerance_f1": 0.15917085028122405, "union_combo": "t3_s8", - "union_f1": 0.3283194730321549 + "path": "", + "source_priority": 0 }, { - "difference": 0.10438312130672354, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0004947066389630949, - "scope": "train_fire_top10pct", + "label": "FengWu", "seed": 97, - "source_priority": 0, - "strict_f1": 0.005460750853242321, - "threshold": 0.5, + "scope": "train_fire_top10pct", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.005460750853242321, "tolerance_f1": 0.04684227229450968, + "union_f1": 0.10984387215996587, + "difference": 0.10438312130672354, + "predicted_positive_rate": 0.0004947066389630949, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.10984387215996587 + "path": "", + "source_priority": 0 }, { - "difference": 0.1019380452757028, - "label": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0003116883116883117, - "scope": "train_fire_top20pct", + "label": "FengWu", "seed": 97, - "source_priority": 0, - "strict_f1": 0.004807692307692307, - "threshold": 0.5, + "scope": "train_fire_top20pct", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.004807692307692307, "tolerance_f1": 0.04393081279657125, + "union_f1": 0.1067457375833951, + "difference": 0.1019380452757028, + "predicted_positive_rate": 0.0003116883116883117, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1067457375833951 + "path": "", + "source_priority": 0 }, { - "difference": 0.3742624367300788, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004230179344465059, - "scope": "full_domain", + "label": "FuXi", "seed": 17, - "source_priority": 0, - "strict_f1": 0.003097173828881146, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.003097173828881146, "tolerance_f1": 0.20441214417386547, + "union_f1": 0.37735961055895995, + "difference": 0.3742624367300788, + "predicted_positive_rate": 0.004230179344465059, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.37735961055895995 + "path": "", + "source_priority": 0 }, { - "difference": 0.530575061359281, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.024220837043633127, - "scope": "train_fire_top05pct", + "label": "FuXi", "seed": 17, - "source_priority": 0, - "strict_f1": 0.02138840232322301, - "threshold": 0.2, + "scope": "train_fire_top05pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.02138840232322301, "tolerance_f1": 0.30180702036802187, + "union_f1": 0.551963463682504, + "difference": 0.530575061359281, + "predicted_positive_rate": 0.024220837043633127, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.551963463682504 + "path": "", + "source_priority": 0 }, { - "difference": 0.43990346446961154, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0040170179083803305, - "scope": "train_fire_top10pct", + "label": "FuXi", "seed": 17, - "source_priority": 0, - "strict_f1": 0.019476002782286114, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.019476002782286114, "tolerance_f1": 0.26166119072060445, + "union_f1": 0.45937946725189766, + "difference": 0.43990346446961154, + "predicted_positive_rate": 0.0040170179083803305, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.45937946725189766 + "path": "", + "source_priority": 0 }, { - "difference": 0.40169955480281794, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0031719233147804574, - "scope": "train_fire_top20pct", + "label": "FuXi", "seed": 17, - "source_priority": 0, - "strict_f1": 0.013674670058832883, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.013674670058832883, "tolerance_f1": 0.25519274478475146, + "union_f1": 0.4153742248616508, + "difference": 0.40169955480281794, + "predicted_positive_rate": 0.0031719233147804574, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4153742248616508 + "path": "", + "source_priority": 0 }, { - "difference": 0.217453250561222, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.001522943722943723, - "scope": "full_domain", + "label": "FuXi", "seed": 37, - "source_priority": 0, - "strict_f1": 0.0029424127784783522, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.0029424127784783522, "tolerance_f1": 0.13539446199130012, + "union_f1": 0.22039566333970034, + "difference": 0.217453250561222, + "predicted_positive_rate": 0.001522943722943723, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22039566333970034 + "path": "", + "source_priority": 0 }, { - "difference": 0.3862603850081952, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.007007519540912239, - "scope": "train_fire_top05pct", + "label": "FuXi", "seed": 37, - "source_priority": 0, - "strict_f1": 0.02109053497942387, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.02109053497942387, "tolerance_f1": 0.26086649919553234, + "union_f1": 0.4073509199876191, + "difference": 0.3862603850081952, + "predicted_positive_rate": 0.007007519540912239, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4073509199876191 + "path": "", + "source_priority": 0 }, { - "difference": 0.23030507895417707, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0016448995745522905, - "scope": "train_fire_top10pct", + "label": "FuXi", "seed": 37, - "source_priority": 0, - "strict_f1": 0.016701461377870565, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.016701461377870565, "tolerance_f1": 0.1656142112861716, + "union_f1": 0.24700654033204764, + "difference": 0.23030507895417707, + "predicted_positive_rate": 0.0016448995745522905, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.24700654033204764 + "path": "", + "source_priority": 0 }, { - "difference": 0.21606998197866278, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0009029066171923315, - "scope": "train_fire_top20pct", + "label": "FuXi", "seed": 37, - "source_priority": 0, - "strict_f1": 0.015267175572519083, - "threshold": 0.2, + "scope": "train_fire_top20pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.015267175572519083, "tolerance_f1": 0.15340459969134643, + "union_f1": 0.23133715755118187, + "difference": 0.21606998197866278, + "predicted_positive_rate": 0.0009029066171923315, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23133715755118187 + "path": "", + "source_priority": 0 }, { - "difference": 0.35798567074450705, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0026514533085961658, - "scope": "full_domain", + "label": "FuXi", "seed": 57, - "source_priority": 0, - "strict_f1": 0.005897939994871356, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.005897939994871356, "tolerance_f1": 0.22566599998043468, + "union_f1": 0.3638836107393784, + "difference": 0.35798567074450705, + "predicted_positive_rate": 0.0026514533085961658, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3638836107393784 + "path": "", + "source_priority": 0 }, { - "difference": 0.49281035455029043, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004068962105471455, - "scope": "train_fire_top05pct", + "label": "FuXi", "seed": 57, - "source_priority": 0, - "strict_f1": 0.030370370370370364, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.030370370370370364, "tolerance_f1": 0.3499638832422191, + "union_f1": 0.5231807249206608, + "difference": 0.49281035455029043, + "predicted_positive_rate": 0.004068962105471455, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5231807249206608 + "path": "", + "source_priority": 0 }, { - "difference": 0.45700386933894555, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.002442614029880281, - "scope": "train_fire_top10pct", + "label": "FuXi", "seed": 57, - "source_priority": 0, - "strict_f1": 0.026973684210526316, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.026973684210526316, "tolerance_f1": 0.3231296638237823, + "union_f1": 0.48397755354947186, + "difference": 0.45700386933894555, + "predicted_positive_rate": 0.002442614029880281, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.48397755354947186 + "path": "", + "source_priority": 0 }, { - "difference": 0.4314537012527686, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0013982683982683983, - "scope": "train_fire_top20pct", + "label": "FuXi", "seed": 57, - "source_priority": 0, - "strict_f1": 0.0239695995323005, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.0239695995323005, "tolerance_f1": 0.30046102485294895, + "union_f1": 0.4554233007850691, + "difference": 0.4314537012527686, + "predicted_positive_rate": 0.0013982683982683983, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4554233007850691 + "path": "", + "source_priority": 0 }, { - "difference": 0.44532955766011967, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005430797773654916, - "scope": "full_domain", + "label": "FuXi", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0033573873422136956, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.0033573873422136956, "tolerance_f1": 0.2682493391647262, + "union_f1": 0.44868694500233336, + "difference": 0.44532955766011967, + "predicted_positive_rate": 0.005430797773654916, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.44868694500233336 + "path": "", + "source_priority": 0 }, { - "difference": 0.6138219771674631, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0665751459384585, - "scope": "train_fire_top05pct", + "label": "FuXi", "seed": 7, - "source_priority": 0, - "strict_f1": 0.016660707901322847, - "threshold": 0.1, + "scope": "train_fire_top05pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.016660707901322847, "tolerance_f1": 0.38200179460101663, + "union_f1": 0.6304826850687859, + "difference": 0.6138219771674631, + "predicted_positive_rate": 0.0665751459384585, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6304826850687859 + "path": "", + "source_priority": 0 }, { - "difference": 0.34740284065122157, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0019404867913327397, - "scope": "train_fire_top10pct", + "label": "FuXi", "seed": 7, - "source_priority": 0, - "strict_f1": 0.011389521640091117, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.011389521640091117, "tolerance_f1": 0.2251054852320675, + "union_f1": 0.35879236229131267, + "difference": 0.34740284065122157, + "predicted_positive_rate": 0.0019404867913327397, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.35879236229131267 + "path": "", + "source_priority": 0 }, { - "difference": 0.2820322009484037, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0013902288188002473, - "scope": "train_fire_top20pct", + "label": "FuXi", "seed": 7, - "source_priority": 0, - "strict_f1": 0.008802816901408451, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.008802816901408451, "tolerance_f1": 0.18265891116039673, + "union_f1": 0.29083501784981214, + "difference": 0.2820322009484037, + "predicted_positive_rate": 0.0013902288188002473, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.29083501784981214 + "path": "", + "source_priority": 0 }, { - "difference": 0.45054053460571325, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.008687322201607915, - "scope": "full_domain", + "label": "FuXi", "seed": 97, - "source_priority": 0, - "strict_f1": 0.003573506191307238, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.003573506191307238, "tolerance_f1": 0.21789178988640112, + "union_f1": 0.4541140407970205, + "difference": 0.45054053460571325, + "predicted_positive_rate": 0.008687322201607915, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4541140407970205 + "path": "", + "source_priority": 0 }, { - "difference": 0.5715381697222122, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04470416542990007, - "scope": "train_fire_top05pct", + "label": "FuXi", "seed": 97, - "source_priority": 0, - "strict_f1": 0.012024257632789628, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.012024257632789628, "tolerance_f1": 0.30008183753413636, + "union_f1": 0.5835624273550017, + "difference": 0.5715381697222122, + "predicted_positive_rate": 0.04470416542990007, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5835624273550017 + "path": "", + "source_priority": 0 }, { - "difference": 0.45337080969477445, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.01593821114079351, - "scope": "train_fire_top10pct", + "label": "FuXi", "seed": 97, - "source_priority": 0, - "strict_f1": 0.008170871559633027, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.008170871559633027, "tolerance_f1": 0.22512986809908905, + "union_f1": 0.4615416812544075, + "difference": 0.45337080969477445, + "predicted_positive_rate": 0.01593821114079351, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4615416812544075 + "path": "", + "source_priority": 0 }, { - "difference": 0.43708558281057147, - "label": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.011240568954854669, - "scope": "train_fire_top20pct", + "label": "FuXi", "seed": 97, - "source_priority": 0, - "strict_f1": 0.006516342573438146, - "threshold": 0.2, + "scope": "train_fire_top20pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.006516342573438146, "tolerance_f1": 0.20602494796886814, + "union_f1": 0.4436019253840096, + "difference": 0.43708558281057147, + "predicted_positive_rate": 0.011240568954854669, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4436019253840096 + "path": "", + "source_priority": 0 }, { - "difference": 0.2698875125828892, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0034029684601113172, - "scope": "full_domain", "seed": 17, - "source_priority": 0, - "strict_f1": 0.0018999796430752525, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.0018999796430752525, "tolerance_f1": 0.11199020227536667, + "union_f1": 0.27178749222596443, + "difference": 0.2698875125828892, + "predicted_positive_rate": 0.0034029684601113172, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.27178749222596443 + "path": "", + "source_priority": 0 }, { - "difference": 0.2627312378871571, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00599584446423271, - "scope": "train_fire_top05pct", "seed": 17, - "source_priority": 0, - "strict_f1": 0.006323656223052602, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.006323656223052602, "tolerance_f1": 0.1236361475439318, + "union_f1": 0.26905489411020966, + "difference": 0.2627312378871571, + "predicted_positive_rate": 0.00599584446423271, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.26905489411020966 + "path": "", + "source_priority": 0 }, { - "difference": 0.2112720683625158, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004856782428020184, - "scope": "train_fire_top10pct", "seed": 17, - "source_priority": 0, - "strict_f1": 0.004407051282051282, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.004407051282051282, "tolerance_f1": 0.09804613935969869, + "union_f1": 0.2156791196445671, + "difference": 0.2112720683625158, + "predicted_positive_rate": 0.004856782428020184, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2156791196445671 + "path": "", + "source_priority": 0 }, { - "difference": 0.19289957921827447, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0036982065553494126, - "scope": "train_fire_top20pct", "seed": 17, - "source_priority": 0, - "strict_f1": 0.003081232492997199, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.003081232492997199, "tolerance_f1": 0.0918614020950846, + "union_f1": 0.19598081171127166, + "difference": 0.19289957921827447, + "predicted_positive_rate": 0.0036982065553494126, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19598081171127166 + "path": "", + "source_priority": 0 }, { - "difference": 0.44278198559528875, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.015876932591218307, - "scope": "full_domain", "seed": 37, - "source_priority": 0, - "strict_f1": 0.002025689424980434, - "threshold": 0.1, + "scope": "full_domain", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.002025689424980434, "tolerance_f1": 0.17658327178527805, + "union_f1": 0.4448076750202692, + "difference": 0.44278198559528875, + "predicted_positive_rate": 0.015876932591218307, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4448076750202692 + "path": "", + "source_priority": 0 }, { - "difference": 0.5419792852745725, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02707034728406055, - "scope": "train_fire_top05pct", "seed": 37, - "source_priority": 0, - "strict_f1": 0.014334527877323108, - "threshold": 0.1, + "scope": "train_fire_top05pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.014334527877323108, "tolerance_f1": 0.23786880092608761, + "union_f1": 0.5563138131518957, + "difference": 0.5419792852745725, + "predicted_positive_rate": 0.02707034728406055, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5563138131518957 + "path": "", + "source_priority": 0 }, { - "difference": 0.467280448823381, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.018105026219451866, - "scope": "train_fire_top10pct", "seed": 37, - "source_priority": 0, - "strict_f1": 0.010952623535404991, - "threshold": 0.1, + "scope": "train_fire_top10pct", "threshold_key": "0.1000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.1, + "strict_f1": 0.010952623535404991, "tolerance_f1": 0.19299696406724362, + "union_f1": 0.478233072358786, + "difference": 0.467280448823381, + "predicted_positive_rate": 0.018105026219451866, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.478233072358786 + "path": "", + "source_priority": 0 }, { - "difference": 0.31356600176953114, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004520098948670378, - "scope": "train_fire_top20pct", "seed": 37, - "source_priority": 0, - "strict_f1": 0.003306175463454953, - "threshold": 0.15, + "scope": "train_fire_top20pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.003306175463454953, "tolerance_f1": 0.10046960132658891, + "union_f1": 0.3168721772329861, + "difference": 0.31356600176953114, + "predicted_positive_rate": 0.004520098948670378, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3168721772329861 + "path": "", + "source_priority": 0 }, { - "difference": 0.3065577101854246, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0028299319727891157, - "scope": "full_domain", "seed": 57, - "source_priority": 0, - "strict_f1": 0.00458918723078781, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.00458918723078781, "tolerance_f1": 0.14940756501428915, + "union_f1": 0.31114689741621243, + "difference": 0.3065577101854246, + "predicted_positive_rate": 0.0028299319727891157, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.31114689741621243 + "path": "", + "source_priority": 0 }, { - "difference": 0.3107866198278387, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.006668645493222519, - "scope": "train_fire_top05pct", "seed": 57, - "source_priority": 0, - "strict_f1": 0.012796587576646227, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.012796587576646227, "tolerance_f1": 0.15839502058327895, + "union_f1": 0.3235832074044849, + "difference": 0.3107866198278387, + "predicted_positive_rate": 0.006668645493222519, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3235832074044849 + "path": "", + "source_priority": 0 }, { - "difference": 0.285638850087505, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004374443455031167, - "scope": "train_fire_top10pct", "seed": 57, - "source_priority": 0, - "strict_f1": 0.010430247718383311, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.010430247718383311, "tolerance_f1": 0.14381976498868418, + "union_f1": 0.2960690978058883, + "difference": 0.285638850087505, + "predicted_positive_rate": 0.004374443455031167, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2960690978058883 + "path": "", + "source_priority": 0 }, { - "difference": 0.2659461938342217, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.002423005565862709, - "scope": "train_fire_top20pct", "seed": 57, - "source_priority": 0, - "strict_f1": 0.0094525403702245, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.0094525403702245, "tolerance_f1": 0.1330139941503748, + "union_f1": 0.2753987342044462, + "difference": 0.2659461938342217, + "predicted_positive_rate": 0.002423005565862709, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2753987342044462 + "path": "", + "source_priority": 0 }, { - "difference": 0.1610876335708797, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0013768707482993197, - "scope": "full_domain", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00412434125105018, - "threshold": 0.5, + "scope": "full_domain", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.00412434125105018, "tolerance_f1": 0.08503531719948608, + "union_f1": 0.1652119748219299, + "difference": 0.1610876335708797, + "predicted_positive_rate": 0.0013768707482993197, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1652119748219299 + "path": "", + "source_priority": 0 }, { - "difference": 0.27422409940357173, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.019518650440288908, - "scope": "train_fire_top05pct", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00961323496534764, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.00961323496534764, "tolerance_f1": 0.13752896018113672, + "union_f1": 0.2838373343689194, + "difference": 0.27422409940357173, + "predicted_positive_rate": 0.019518650440288908, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2838373343689194 + "path": "", + "source_priority": 0 }, { - "difference": 0.14003318233886966, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0035594142673394677, - "scope": "train_fire_top10pct", "seed": 7, - "source_priority": 0, - "strict_f1": 0.005579507988840983, - "threshold": 0.5, + "scope": "train_fire_top10pct", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.005579507988840983, "tolerance_f1": 0.07891517659613233, + "union_f1": 0.14561269032771065, + "difference": 0.14003318233886966, + "predicted_positive_rate": 0.0035594142673394677, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.14561269032771065 + "path": "", + "source_priority": 0 }, { - "difference": 0.13031562548755435, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0019332096474953618, - "scope": "train_fire_top20pct", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0051329911339244045, - "threshold": 0.5, + "scope": "train_fire_top20pct", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.0051329911339244045, "tolerance_f1": 0.07335569213391166, + "union_f1": 0.13544861662147875, + "difference": 0.13031562548755435, + "predicted_positive_rate": 0.0019332096474953618, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.13544861662147875 + "path": "", + "source_priority": 0 }, { - "difference": 0.13447416377324056, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0013461966604823748, - "scope": "full_domain", "seed": 97, - "source_priority": 0, - "strict_f1": 0.0015570260801868433, - "threshold": 0.5, + "scope": "full_domain", "threshold_key": "0.5000", "threshold_selector": "strict_f1", + "threshold": 0.5, + "strict_f1": 0.0015570260801868433, + "tolerance_f1": 0.04422040571422221, + "union_f1": 0.1360311898534274, + "difference": 0.13447416377324056, + "predicted_positive_rate": 0.0013461966604823748, "tolerance_combo": "t0_s8", - "tolerance_f1": 0.04422040571422221, "union_combo": "t3_s8", - "union_f1": 0.1360311898534274 + "path": "", + "source_priority": 0 }, { - "difference": 0.35676759703557515, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.014237657069357871, - "scope": "train_fire_top05pct", "seed": 97, - "source_priority": 0, - "strict_f1": 0.010571134928791661, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.010571134928791661, "tolerance_f1": 0.16460271440438795, + "union_f1": 0.3673387319643668, + "difference": 0.35676759703557515, + "predicted_positive_rate": 0.014237657069357871, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3673387319643668 + "path": "", + "source_priority": 0 }, { - "difference": 0.11744148944724705, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.002389433066191748, - "scope": "train_fire_top10pct", "seed": 97, - "source_priority": 0, - "strict_f1": 0.002669336002669336, - "threshold": 0.5, + "scope": "train_fire_top10pct", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.002669336002669336, "tolerance_f1": 0.03347640061525648, + "union_f1": 0.12011082544991639, + "difference": 0.11744148944724705, + "predicted_positive_rate": 0.002389433066191748, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.12011082544991639 + "path": "", + "source_priority": 0 }, { - "difference": 0.11926427213402073, + "model_tag": "pangu24_h24", "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0014211502782931354, - "scope": "train_fire_top20pct", "seed": 97, - "source_priority": 0, - "strict_f1": 0.002313475997686524, - "threshold": 0.5, + "scope": "train_fire_top20pct", "threshold_key": "0.5000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.5, + "strict_f1": 0.002313475997686524, "tolerance_f1": 0.036969834429575875, + "union_f1": 0.12157774813170726, + "difference": 0.11926427213402073, + "predicted_positive_rate": 0.0014211502782931354, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.12157774813170726 + "path": "", + "source_priority": 0 }, { - "difference": 0.3981469865854522, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.01413716759431045, - "scope": "full_domain", + "label": "Pangu-Weather", "seed": 17, - "source_priority": 0, - "strict_f1": 0.0019783244452090142, - "threshold": 0.2, + "scope": "full_domain", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.0019783244452090142, "tolerance_f1": 0.1809729686535962, + "union_f1": 0.40012531103066123, + "difference": 0.3981469865854522, + "predicted_positive_rate": 0.01413716759431045, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.40012531103066123 + "path": "", + "source_priority": 0 }, { - "difference": 0.4579044245182327, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.027451271396062134, - "scope": "train_fire_top05pct", + "label": "Pangu-Weather", "seed": 17, - "source_priority": 0, - "strict_f1": 0.011848926191063935, - "threshold": 0.2, + "scope": "train_fire_top05pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.011848926191063935, "tolerance_f1": 0.23514322359982898, + "union_f1": 0.46975335070929664, + "difference": 0.4579044245182327, + "predicted_positive_rate": 0.027451271396062134, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.46975335070929664 + "path": "", + "source_priority": 0 }, { - "difference": 0.37162889777394226, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.021221678044919364, - "scope": "train_fire_top10pct", + "label": "Pangu-Weather", "seed": 17, - "source_priority": 0, - "strict_f1": 0.008011413520632135, - "threshold": 0.2, + "scope": "train_fire_top10pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.008011413520632135, "tolerance_f1": 0.18336249012390426, + "union_f1": 0.3796403112945744, + "difference": 0.37162889777394226, + "predicted_positive_rate": 0.021221678044919364, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3796403112945744 + "path": "", + "source_priority": 0 }, { - "difference": 0.3180949408491859, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.017168212739641312, - "scope": "train_fire_top20pct", + "label": "Pangu-Weather", "seed": 17, - "source_priority": 0, - "strict_f1": 0.005117388748660143, - "threshold": 0.2, + "scope": "train_fire_top20pct", "threshold_key": "0.2000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.2, + "strict_f1": 0.005117388748660143, "tolerance_f1": 0.14938342628742876, + "union_f1": 0.323212329597846, + "difference": 0.3180949408491859, + "predicted_positive_rate": 0.017168212739641312, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.323212329597846 + "path": "", + "source_priority": 0 }, { - "difference": 0.3350135537819443, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.00479208410636982, - "scope": "full_domain", + "label": "Pangu-Weather", "seed": 37, - "source_priority": 0, - "strict_f1": 0.0014740203906154037, - "threshold": 0.15, + "scope": "full_domain", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.0014740203906154037, "tolerance_f1": 0.14188194761227685, + "union_f1": 0.3364875741725597, + "difference": 0.3350135537819443, + "predicted_positive_rate": 0.00479208410636982, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3364875741725597 + "path": "", + "source_priority": 0 }, { - "difference": 0.42661251656646776, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.006297615514000198, - "scope": "train_fire_top05pct", + "label": "Pangu-Weather", "seed": 37, - "source_priority": 0, - "strict_f1": 0.012218828103304637, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.012218828103304637, "tolerance_f1": 0.1893747086942956, + "union_f1": 0.4388313446697724, + "difference": 0.42661251656646776, + "predicted_positive_rate": 0.006297615514000198, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4388313446697724 + "path": "", + "source_priority": 0 }, { - "difference": 0.4013551577246312, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004044226773523301, - "scope": "train_fire_top10pct", + "label": "Pangu-Weather", "seed": 37, - "source_priority": 0, - "strict_f1": 0.010149942329873126, - "threshold": 0.15, + "scope": "train_fire_top10pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.010149942329873126, "tolerance_f1": 0.1623425297604306, + "union_f1": 0.41150510005450436, + "difference": 0.4013551577246312, + "predicted_positive_rate": 0.004044226773523301, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.41150510005450436 + "path": "", + "source_priority": 0 }, { - "difference": 0.36502027762951006, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0025015460729746444, - "scope": "train_fire_top20pct", + "label": "Pangu-Weather", "seed": 37, - "source_priority": 0, - "strict_f1": 0.008453410182516812, - "threshold": 0.15, + "scope": "train_fire_top20pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.008453410182516812, "tolerance_f1": 0.15039592221466835, + "union_f1": 0.3734736878120269, + "difference": 0.36502027762951006, + "predicted_positive_rate": 0.0025015460729746444, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3734736878120269 + "path": "", + "source_priority": 0 }, { - "difference": 0.35070363810152716, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004669140383426098, - "scope": "full_domain", + "label": "Pangu-Weather", "seed": 57, - "source_priority": 0, - "strict_f1": 0.003978746443051044, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.003978746443051044, "tolerance_f1": 0.18301623358471802, + "union_f1": 0.3546823845445782, + "difference": 0.35070363810152716, + "predicted_positive_rate": 0.004669140383426098, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3546823845445782 + "path": "", + "source_priority": 0 }, { - "difference": 0.4222173656123986, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.008479271791827446, - "scope": "train_fire_top05pct", + "label": "Pangu-Weather", "seed": 57, - "source_priority": 0, - "strict_f1": 0.017845192951148784, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.017845192951148784, "tolerance_f1": 0.2501388218123639, + "union_f1": 0.44006255856354737, + "difference": 0.4222173656123986, + "predicted_positive_rate": 0.008479271791827446, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.44006255856354737 + "path": "", + "source_priority": 0 }, { - "difference": 0.40500926707190377, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004714554269318294, - "scope": "train_fire_top10pct", + "label": "Pangu-Weather", "seed": 57, - "source_priority": 0, - "strict_f1": 0.016403526758253024, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.016403526758253024, "tolerance_f1": 0.23804674197160172, + "union_f1": 0.4214127938301568, + "difference": 0.40500926707190377, + "predicted_positive_rate": 0.004714554269318294, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4214127938301568 + "path": "", + "source_priority": 0 }, { - "difference": 0.3810054677452299, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0026456400742115027, - "scope": "train_fire_top20pct", + "label": "Pangu-Weather", "seed": 57, - "source_priority": 0, - "strict_f1": 0.01471129091577786, - "threshold": 0.3, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.01471129091577786, "tolerance_f1": 0.2249788440916092, + "union_f1": 0.3957167586610078, + "difference": 0.3810054677452299, + "predicted_positive_rate": 0.0026456400742115027, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3957167586610078 + "path": "", + "source_priority": 0 }, { - "difference": 0.21893843540252053, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0012771799628942485, - "scope": "full_domain", + "label": "Pangu-Weather", "seed": 7, - "source_priority": 0, - "strict_f1": 0.0037437942540896885, - "threshold": 0.3, + "scope": "full_domain", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.0037437942540896885, "tolerance_f1": 0.12218430629860097, + "union_f1": 0.22268222965661022, + "difference": 0.21893843540252053, + "predicted_positive_rate": 0.0012771799628942485, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22268222965661022 + "path": "", + "source_priority": 0 }, { - "difference": 0.21672193002919604, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.004447412684278223, - "scope": "train_fire_top05pct", + "label": "Pangu-Weather", "seed": 7, - "source_priority": 0, - "strict_f1": 0.010515247108307044, - "threshold": 0.3, + "scope": "train_fire_top05pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.010515247108307044, "tolerance_f1": 0.12664654352782623, + "union_f1": 0.22723717713750308, + "difference": 0.21672193002919604, + "predicted_positive_rate": 0.004447412684278223, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22723717713750308 + "path": "", + "source_priority": 0 }, { - "difference": 0.18892752891362113, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0033887404768972, - "scope": "train_fire_top10pct", + "label": "Pangu-Weather", "seed": 7, - "source_priority": 0, - "strict_f1": 0.00788436268068331, - "threshold": 0.3, + "scope": "train_fire_top10pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.00788436268068331, "tolerance_f1": 0.1079162350962175, + "union_f1": 0.19681189159430446, + "difference": 0.18892752891362113, + "predicted_positive_rate": 0.0033887404768972, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.19681189159430446 + "path": "", + "source_priority": 0 }, { - "difference": 0.17487748330642924, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.001932591218305504, - "scope": "train_fire_top20pct", - "seed": 7, - "source_priority": 0, - "strict_f1": 0.0070011668611435224, - "threshold": 0.3, + "label": "Pangu-Weather", + "seed": 7, + "scope": "train_fire_top20pct", "threshold_key": "0.3000", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.3, + "strict_f1": 0.0070011668611435224, "tolerance_f1": 0.09937369519832985, + "union_f1": 0.18187865016757276, + "difference": 0.17487748330642924, + "predicted_positive_rate": 0.001932591218305504, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.18187865016757276 + "path": "", + "source_priority": 0 }, { - "difference": 0.4653493031095141, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.029129251700680273, - "scope": "full_domain", + "label": "Pangu-Weather", "seed": 97, - "source_priority": 0, - "strict_f1": 0.0026024230327071516, - "threshold": 0.15, + "scope": "full_domain", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.0026024230327071516, "tolerance_f1": 0.2264875300093167, + "union_f1": 0.46795172614222125, + "difference": 0.4653493031095141, + "predicted_positive_rate": 0.029129251700680273, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.46795172614222125 + "path": "", + "source_priority": 0 }, { - "difference": 0.5794332814358789, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.056067576926882357, - "scope": "train_fire_top05pct", + "label": "Pangu-Weather", "seed": 97, - "source_priority": 0, - "strict_f1": 0.015850265576258328, - "threshold": 0.15, + "scope": "train_fire_top05pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.015850265576258328, "tolerance_f1": 0.3098064830268362, + "union_f1": 0.5952835470121373, + "difference": 0.5794332814358789, + "predicted_positive_rate": 0.056067576926882357, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5952835470121373 + "path": "", + "source_priority": 0 }, { - "difference": 0.5050483949122122, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.03677772830711388, - "scope": "train_fire_top10pct", + "label": "Pangu-Weather", "seed": 97, - "source_priority": 0, - "strict_f1": 0.012206999545484059, - "threshold": 0.15, + "scope": "train_fire_top10pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.012206999545484059, "tolerance_f1": 0.25501453744070673, + "union_f1": 0.5172553944576963, + "difference": 0.5050483949122122, + "predicted_positive_rate": 0.03677772830711388, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5172553944576963 + "path": "", + "source_priority": 0 }, { - "difference": 0.44522188799641144, - "label": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.026268398268398267, - "scope": "train_fire_top20pct", + "label": "Pangu-Weather", "seed": 97, - "source_priority": 0, - "strict_f1": 0.008937574479787332, - "threshold": 0.15, + "scope": "train_fire_top20pct", "threshold_key": "0.1500", "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", + "threshold": 0.15, + "strict_f1": 0.008937574479787332, "tolerance_f1": 0.226729727407505, + "union_f1": 0.45415946247619876, + "difference": 0.44522188799641144, + "predicted_positive_rate": 0.026268398268398267, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.45415946247619876 + "path": "", + "source_priority": 0 }, { - "difference": 0.5480197595988799, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.016084972170686457, - "scope": "full_domain", + "label": "Reference", "seed": 1, - "source_priority": 1, - "strict_f1": 0.004136112962850736, - "threshold": 0.95, + "scope": "full_domain", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.004136112962850736, "tolerance_f1": 0.2985122125101588, + "union_f1": 0.5521558725617307, + "difference": 0.5480197595988799, + "predicted_positive_rate": 0.016084972170686457, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5521558725617307 + "path": "", + "source_priority": 1 }, { - "difference": 0.6696707231715443, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.023750865736618186, - "scope": "train_fire_top05pct", + "label": "Reference", "seed": 1, - "source_priority": 1, - "strict_f1": 0.03321760345312939, - "threshold": 0.95, + "scope": "train_fire_top05pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.03321760345312939, "tolerance_f1": 0.39555976109165375, + "union_f1": 0.7028883266246737, + "difference": 0.6696707231715443, + "predicted_positive_rate": 0.023750865736618186, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7028883266246737 + "path": "", + "source_priority": 1 }, { - "difference": 0.6666121634162013, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.011875432868309093, - "scope": "train_fire_top10pct", + "label": "Reference", "seed": 1, - "source_priority": 1, - "strict_f1": 0.033186462923033656, - "threshold": 0.95, + "scope": "train_fire_top10pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.033186462923033656, "tolerance_f1": 0.3945515652074689, + "union_f1": 0.699798626339235, + "difference": 0.6666121634162013, + "predicted_positive_rate": 0.011875432868309093, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.699798626339235 + "path": "", + "source_priority": 1 }, { - "difference": 0.6388534040739275, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0059381570810142234, - "scope": "train_fire_top20pct", + "label": "Reference", "seed": 1, - "source_priority": 1, - "strict_f1": 0.032893514216688347, - "threshold": 0.95, + "scope": "train_fire_top20pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.032893514216688347, "tolerance_f1": 0.3852239693382854, + "union_f1": 0.6717469182906158, + "difference": 0.6388534040739275, + "predicted_positive_rate": 0.0059381570810142234, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6717469182906158 + "path": "", + "source_priority": 1 }, { - "difference": 0.5862595758183573, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.021380705009276437, - "scope": "full_domain", + "label": "Reference", "seed": 123, - "source_priority": 1, - "strict_f1": 0.0026426577586601383, - "threshold": 0.95, + "scope": "full_domain", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.0026426577586601383, "tolerance_f1": 0.2756667293290642, + "union_f1": 0.5889022335770174, + "difference": 0.5862595758183573, + "predicted_positive_rate": 0.021380705009276437, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5889022335770174 + "path": "", + "source_priority": 1 }, { - "difference": 0.6960690170722506, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04860987434451371, - "scope": "train_fire_top05pct", + "label": "Reference", "seed": 123, - "source_priority": 1, - "strict_f1": 0.02462935239291061, - "threshold": 0.9, + "scope": "train_fire_top05pct", "threshold_key": "0.9", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.9, + "strict_f1": 0.02462935239291061, "tolerance_f1": 0.3702273155255325, + "union_f1": 0.7206983694651612, + "difference": 0.6960690170722506, + "predicted_positive_rate": 0.04860987434451371, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7206983694651612 + "path": "", + "source_priority": 1 }, { - "difference": 0.6932811848453125, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.024304937172256853, - "scope": "train_fire_top10pct", + "label": "Reference", "seed": 123, - "source_priority": 1, - "strict_f1": 0.02461746391852102, - "threshold": 0.9, + "scope": "train_fire_top10pct", "threshold_key": "0.9", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.9, + "strict_f1": 0.02461746391852102, "tolerance_f1": 0.369479820868587, + "union_f1": 0.7178986487638336, + "difference": 0.6932811848453125, + "predicted_positive_rate": 0.024304937172256853, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7178986487638336 + "path": "", + "source_priority": 1 }, { - "difference": 0.6678424934752141, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.012153370439084725, - "scope": "train_fire_top20pct", + "label": "Reference", "seed": 123, - "source_priority": 1, - "strict_f1": 0.024505093215452623, - "threshold": 0.9, + "scope": "train_fire_top20pct", "threshold_key": "0.9", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.9, + "strict_f1": 0.024505093215452623, "tolerance_f1": 0.36252633147981417, + "union_f1": 0.6923475866906666, + "difference": 0.6678424934752141, + "predicted_positive_rate": 0.012153370439084725, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6923475866906666 + "path": "", + "source_priority": 1 }, { - "difference": 0.6194077454910121, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.019636239950525665, - "scope": "full_domain", + "label": "Reference", "seed": 42, - "source_priority": 1, - "strict_f1": 0.0045296167247386755, - "threshold": 0.95, + "scope": "full_domain", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.0045296167247386755, "tolerance_f1": 0.3056545098251043, + "union_f1": 0.6239373622157508, + "difference": 0.6194077454910121, + "predicted_positive_rate": 0.019636239950525665, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6239373622157508 + "path": "", + "source_priority": 1 }, { - "difference": 0.7095179230119556, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.033442168793905214, - "scope": "train_fire_top05pct", + "label": "Reference", "seed": 42, - "source_priority": 1, - "strict_f1": 0.038147512864494, - "threshold": 0.95, + "scope": "train_fire_top05pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.038147512864494, "tolerance_f1": 0.4063154181985604, + "union_f1": 0.7476654358764496, + "difference": 0.7095179230119556, + "predicted_positive_rate": 0.033442168793905214, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7476654358764496 + "path": "", + "source_priority": 1 }, { - "difference": 0.7063341380484894, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.016721084396952607, - "scope": "train_fire_top10pct", + "label": "Reference", "seed": 42, - "source_priority": 1, - "strict_f1": 0.0381213575591361, - "threshold": 0.95, + "scope": "train_fire_top10pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.0381213575591361, "tolerance_f1": 0.4052951105632299, + "union_f1": 0.7444554956076256, + "difference": 0.7063341380484894, + "predicted_positive_rate": 0.016721084396952607, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7444554956076256 + "path": "", + "source_priority": 1 }, { - "difference": 0.677407206441351, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.008361162646876932, - "scope": "train_fire_top20pct", + "label": "Reference", "seed": 42, - "source_priority": 1, - "strict_f1": 0.03787465940054495, - "threshold": 0.95, + "scope": "train_fire_top20pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.03787465940054495, "tolerance_f1": 0.39585180444145995, + "union_f1": 0.7152818658418959, + "difference": 0.677407206441351, + "predicted_positive_rate": 0.008361162646876932, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7152818658418959 + "path": "", + "source_priority": 1 }, { - "difference": 0.601348973225457, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.014596289424860853, - "scope": "full_domain", + "label": "Reference", "seed": 7, - "source_priority": 1, - "strict_f1": 0.006568199246490847, - "threshold": 0.95, + "scope": "full_domain", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.006568199246490847, "tolerance_f1": 0.29937890424270136, + "union_f1": 0.6079171724719479, + "difference": 0.601348973225457, + "predicted_positive_rate": 0.014596289424860853, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6079171724719479 + "path": "", + "source_priority": 1 }, { - "difference": 0.7136230759527924, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02449292569506283, - "scope": "train_fire_top05pct", + "label": "Reference", "seed": 7, - "source_priority": 1, - "strict_f1": 0.04873596787441818, - "threshold": 0.95, + "scope": "train_fire_top05pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.04873596787441818, "tolerance_f1": 0.40145293653054454, + "union_f1": 0.7623590438272105, + "difference": 0.7136230759527924, + "predicted_positive_rate": 0.02449292569506283, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7623590438272105 + "path": "", + "source_priority": 1 }, { - "difference": 0.7104053704635315, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.012246462847531414, - "scope": "train_fire_top10pct", + "label": "Reference", "seed": 7, - "source_priority": 1, - "strict_f1": 0.0486915291328531, - "threshold": 0.95, + "scope": "train_fire_top10pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.0486915291328531, "tolerance_f1": 0.4004617740449798, + "union_f1": 0.7590968995963846, + "difference": 0.7104053704635315, + "predicted_positive_rate": 0.012246462847531414, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7590968995963846 + "path": "", + "source_priority": 1 }, { - "difference": 0.6811711568125168, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.006123685837971553, - "scope": "train_fire_top20pct", + "label": "Reference", "seed": 7, - "source_priority": 1, - "strict_f1": 0.04827336828783222, - "threshold": 0.95, + "scope": "train_fire_top20pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.04827336828783222, "tolerance_f1": 0.39128423640663973, + "union_f1": 0.7294445251003491, + "difference": 0.6811711568125168, + "predicted_positive_rate": 0.006123685837971553, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7294445251003491 + "path": "", + "source_priority": 1 }, { - "difference": 0.5755105867935725, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.012341125541125542, - "scope": "full_domain", + "label": "Reference", "seed": 99, - "source_priority": 1, - "strict_f1": 0.004855561780634761, - "threshold": 0.95, + "scope": "full_domain", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.004855561780634761, "tolerance_f1": 0.30820726968158313, + "union_f1": 0.5803661485742072, + "difference": 0.5755105867935725, + "predicted_positive_rate": 0.012341125541125542, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5803661485742072 + "path": "", + "source_priority": 1 }, { - "difference": 0.6744974001959292, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.022801028989809045, - "scope": "train_fire_top05pct", + "label": "Reference", "seed": 99, - "source_priority": 1, - "strict_f1": 0.0332911515623479, - "threshold": 0.95, + "scope": "train_fire_top05pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.0332911515623479, "tolerance_f1": 0.3895272012200133, + "union_f1": 0.707788551758277, + "difference": 0.6744974001959292, + "predicted_positive_rate": 0.022801028989809045, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.707788551758277 + "path": "", + "source_priority": 1 }, { - "difference": 0.6715102385713686, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.011400514494904522, - "scope": "train_fire_top10pct", + "label": "Reference", "seed": 99, - "source_priority": 1, - "strict_f1": 0.033258776621608484, - "threshold": 0.95, + "scope": "train_fire_top10pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.033258776621608484, "tolerance_f1": 0.38853542457185924, + "union_f1": 0.7047690151929771, + "difference": 0.6715102385713686, + "predicted_positive_rate": 0.011400514494904522, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7047690151929771 + "path": "", + "source_priority": 1 }, { - "difference": 0.6443640032843302, - "label": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.005700680272108844, - "scope": "train_fire_top20pct", + "label": "Reference", "seed": 99, - "source_priority": 1, - "strict_f1": 0.03295432645981885, - "threshold": 0.95, + "scope": "train_fire_top20pct", "threshold_key": "0.95", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.95, + "strict_f1": 0.03295432645981885, "tolerance_f1": 0.37935948303093225, + "union_f1": 0.6773183297441491, + "difference": 0.6443640032843302, + "predicted_positive_rate": 0.005700680272108844, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6773183297441491 + "path": "", + "source_priority": 1 }, { - "difference": 0.14280219707769823, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.21716512059369203, - "scope": "full_domain", + "label": "Aurora", "seed": 1, - "source_priority": 2, - "strict_f1": 0.0004938156417811271, - "threshold": 0.0002848515287041664, + "scope": "full_domain", "threshold_key": "0.00028485153", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002848515287041664, + "strict_f1": 0.0004938156417811271, "tolerance_f1": 0.05024333401294363, + "union_f1": 0.14329601271947937, + "difference": 0.14280219707769823, + "predicted_positive_rate": 0.21716512059369203, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.14329601271947937 + "path": "", + "source_priority": 2 }, { - "difference": 0.4699697200479976, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.08275452656574651, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 1, - "source_priority": 2, - "strict_f1": 0.026484309350641822, - "threshold": 0.0002847809228114784, + "scope": "train_fire_top05pct", "threshold_key": "0.00028478092", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002847809228114784, + "strict_f1": 0.026484309350641822, "tolerance_f1": 0.24351172066402033, + "union_f1": 0.4964540293986394, + "difference": 0.4699697200479976, + "predicted_positive_rate": 0.08275452656574651, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4964540293986394 + "path": "", + "source_priority": 2 }, { - "difference": 0.46744943745251477, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04137726328287326, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 1, - "source_priority": 2, - "strict_f1": 0.026476637409113293, - "threshold": 0.0002847809228114784, + "scope": "train_fire_top10pct", "threshold_key": "0.00028478092", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002847809228114784, + "strict_f1": 0.026476637409113293, "tolerance_f1": 0.24290193233931348, + "union_f1": 0.49392607486162804, + "difference": 0.46744943745251477, + "predicted_positive_rate": 0.04137726328287326, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.49392607486162804 + "path": "", + "source_priority": 2 }, { - "difference": 0.24417949078081141, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7199758812615955, - "scope": "full_domain", + "label": "Aurora", "seed": 123, - "source_priority": 2, - "strict_f1": 0.0006488102455003172, - "threshold": 0.00028207036666572094, + "scope": "full_domain", "threshold_key": "0.00028207037", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028207036666572094, + "strict_f1": 0.0006488102455003172, "tolerance_f1": 0.09040502340162183, + "union_f1": 0.24482830102631173, + "difference": 0.24417949078081141, + "predicted_positive_rate": 0.7199758812615955, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.24482830102631173 + "path": "", + "source_priority": 2 }, { - "difference": 0.3890802257636093, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.921086870485802, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 123, - "source_priority": 2, - "strict_f1": 0.005618158058227469, - "threshold": 0.0002819437358994037, + "scope": "train_fire_top05pct", "threshold_key": "0.00028194374", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002819437358994037, + "strict_f1": 0.005618158058227469, "tolerance_f1": 0.16005632828519903, + "union_f1": 0.39469838382183675, + "difference": 0.3890802257636093, + "predicted_positive_rate": 0.921086870485802, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39469838382183675 + "path": "", + "source_priority": 2 }, { - "difference": 0.2284351905540453, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9565152864351439, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 123, - "source_priority": 2, - "strict_f1": 0.00275028568108307, - "threshold": 0.00028190662851557136, + "scope": "train_fire_top10pct", "threshold_key": "0.00028190663", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028190662851557136, + "strict_f1": 0.00275028568108307, "tolerance_f1": 0.08722255267001126, + "union_f1": 0.23118547623512836, + "difference": 0.2284351905540453, + "predicted_positive_rate": 0.9565152864351439, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23118547623512836 + "path": "", + "source_priority": 2 }, { - "difference": 0.2520341870433926, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.5915943104514533, - "scope": "full_domain", + "label": "Aurora", "seed": 42, - "source_priority": 2, - "strict_f1": 0.0007076278562951189, - "threshold": 0.00028167077107355, + "scope": "full_domain", "threshold_key": "0.00028167077", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028167077107355, + "strict_f1": 0.0007076278562951189, "tolerance_f1": 0.09343380358557288, + "union_f1": 0.2527418148996877, + "difference": 0.2520341870433926, + "predicted_positive_rate": 0.5915943104514533, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2527418148996877 + "path": "", + "source_priority": 2 }, { - "difference": 0.25643683344302254, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.48403581676066093, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 42, - "source_priority": 2, - "strict_f1": 0.00618071474679909, - "threshold": 0.00028234132332727313, + "scope": "train_fire_top05pct", "threshold_key": "0.00028234132", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028234132332727313, + "strict_f1": 0.00618071474679909, "tolerance_f1": 0.10264712599745741, + "union_f1": 0.26261754818982164, + "difference": 0.25643683344302254, + "predicted_positive_rate": 0.48403581676066093, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.26261754818982164 + "path": "", + "source_priority": 2 }, { - "difference": 0.28330440732569195, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.735743791431681, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 42, - "source_priority": 2, - "strict_f1": 0.0035136704264394467, - "threshold": 0.00028129236306995153, + "scope": "train_fire_top10pct", "threshold_key": "0.00028129236", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028129236306995153, + "strict_f1": 0.0035136704264394467, "tolerance_f1": 0.11052705132032803, + "union_f1": 0.2868180777521314, + "difference": 0.28330440732569195, + "predicted_positive_rate": 0.735743791431681, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2868180777521314 + "path": "", + "source_priority": 2 }, { - "difference": 0.2611249045411472, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.6671256648113791, - "scope": "full_domain", + "label": "Aurora", "seed": 7, - "source_priority": 2, - "strict_f1": 0.0007124228455695601, - "threshold": 0.0002701166733984173, + "scope": "full_domain", "threshold_key": "0.00027011667", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002701166733984173, + "strict_f1": 0.0007124228455695601, "tolerance_f1": 0.09742777619989017, + "union_f1": 0.2618373273867167, + "difference": 0.2611249045411472, + "predicted_positive_rate": 0.6671256648113791, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2618373273867167 + "path": "", + "source_priority": 2 }, { - "difference": 0.38950157948139946, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9150341347580885, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 7, - "source_priority": 2, - "strict_f1": 0.00565521517042468, - "threshold": 0.00026978866662830114, + "scope": "train_fire_top05pct", "threshold_key": "0.00026978867", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00026978866662830114, + "strict_f1": 0.00565521517042468, "tolerance_f1": 0.16052688500868942, + "union_f1": 0.39515679465182413, + "difference": 0.38950157948139946, + "predicted_positive_rate": 0.9150341347580885, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39515679465182413 + "path": "", + "source_priority": 2 }, { - "difference": 0.2284351905540453, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9565152864351439, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 7, - "source_priority": 2, - "strict_f1": 0.00275028568108307, - "threshold": 0.00026914896443486214, + "scope": "train_fire_top10pct", "threshold_key": "0.00026914896", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00026914896443486214, + "strict_f1": 0.00275028568108307, "tolerance_f1": 0.08722255267001126, + "union_f1": 0.23118547623512836, + "difference": 0.2284351905540453, + "predicted_positive_rate": 0.9565152864351439, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23118547623512836 + "path": "", + "source_priority": 2 }, { - "difference": 0.25176783806212244, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.5833835497835498, - "scope": "full_domain", + "label": "Aurora", "seed": 99, - "source_priority": 2, - "strict_f1": 0.0007150400212604667, - "threshold": 0.0002832056488841772, + "scope": "full_domain", "threshold_key": "0.00028320565", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002832056488841772, + "strict_f1": 0.0007150400212604667, "tolerance_f1": 0.09353511619384626, + "union_f1": 0.25248287808338293, + "difference": 0.25176783806212244, + "predicted_positive_rate": 0.5833835497835498, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.25248287808338293 + "path": "", + "source_priority": 2 }, { - "difference": 0.21988543507354893, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.4582789156030474, - "scope": "train_fire_top05pct", + "label": "Aurora", "seed": 99, - "source_priority": 2, - "strict_f1": 0.0053561461508737284, - "threshold": 0.00028413085965439677, + "scope": "train_fire_top05pct", "threshold_key": "0.00028413086", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028413085965439677, + "strict_f1": 0.0053561461508737284, "tolerance_f1": 0.08994268846280176, + "union_f1": 0.22524158122442267, + "difference": 0.21988543507354893, + "predicted_positive_rate": 0.4582789156030474, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22524158122442267 + "path": "", + "source_priority": 2 }, { - "difference": 0.2799506337602843, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.749557237558128, - "scope": "train_fire_top10pct", + "label": "Aurora", "seed": 99, - "source_priority": 2, - "strict_f1": 0.0034589138681033788, - "threshold": 0.000282647815765813, + "scope": "train_fire_top10pct", "threshold_key": "0.00028264782", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000282647815765813, + "strict_f1": 0.0034589138681033788, "tolerance_f1": 0.10903157063807417, + "union_f1": 0.28340954762838766, + "difference": 0.2799506337602843, + "predicted_positive_rate": 0.749557237558128, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.28340954762838766 + "path": "", + "source_priority": 2 }, { - "difference": 0.6362769893231447, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.041071861471861475, - "scope": "full_domain", + "label": "ClimaX", "seed": 1, - "source_priority": 2, - "strict_f1": 0.003394935139973715, - "threshold": 0.001442534703828549, + "scope": "full_domain", "threshold_key": "0.0014425347", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.001442534703828549, + "strict_f1": 0.003394935139973715, "tolerance_f1": 0.3112148878371163, + "union_f1": 0.6396719244631184, + "difference": 0.6362769893231447, + "predicted_positive_rate": 0.041071861471861475, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6396719244631184 + "path": "", + "source_priority": 2 }, { - "difference": 0.6341272298959009, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.13341248639556744, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 1, - "source_priority": 2, - "strict_f1": 0.011492789729228419, - "threshold": 0.0014882655838293205, + "scope": "train_fire_top05pct", "threshold_key": "0.0014882656", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0014882655838293205, + "strict_f1": 0.011492789729228419, "tolerance_f1": 0.3186326966868806, + "union_f1": 0.6456200196251293, + "difference": 0.6341272298959009, + "predicted_positive_rate": 0.13341248639556744, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6456200196251293 + "path": "", + "source_priority": 2 }, { - "difference": 0.6312315933404747, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.06670624319778372, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 1, - "source_priority": 2, - "strict_f1": 0.011490700169087834, - "threshold": 0.0014882655838293205, + "scope": "train_fire_top10pct", "threshold_key": "0.0014882656", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0014882655838293205, + "strict_f1": 0.011490700169087834, "tolerance_f1": 0.3179193199949169, + "union_f1": 0.6427222935095626, + "difference": 0.6312315933404747, + "predicted_positive_rate": 0.06670624319778372, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6427222935095626 + "path": "", + "source_priority": 2 }, { - "difference": 0.604731231099908, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.12421632653061225, - "scope": "full_domain", + "label": "ClimaX", "seed": 123, - "source_priority": 2, - "strict_f1": 0.002897888198757764, - "threshold": 0.0005393968409013734, + "scope": "full_domain", "threshold_key": "0.00053939684", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0005393968409013734, + "strict_f1": 0.002897888198757764, "tolerance_f1": 0.2855276491036767, + "union_f1": 0.6076291192986658, + "difference": 0.604731231099908, + "predicted_positive_rate": 0.12421632653061225, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6076291192986658 + "path": "", + "source_priority": 2 }, { - "difference": 0.7052352926158846, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.3074774908479272, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 123, - "source_priority": 2, - "strict_f1": 0.012252516711603196, - "threshold": 0.0005436268660297483, + "scope": "train_fire_top05pct", "threshold_key": "0.00054362687", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0005436268660297483, + "strict_f1": 0.012252516711603196, "tolerance_f1": 0.3566174282966785, + "union_f1": 0.7174878093274878, + "difference": 0.7052352926158846, + "predicted_positive_rate": 0.3074774908479272, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7174878093274878 + "path": "", + "source_priority": 2 }, { - "difference": 0.6886984637018222, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.14598421885821708, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 123, - "source_priority": 2, - "strict_f1": 0.010293697838827224, - "threshold": 0.0006776005131691771, + "scope": "train_fire_top10pct", "threshold_key": "0.00067760051", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0006776005131691771, + "strict_f1": 0.010293697838827224, "tolerance_f1": 0.34872837593596695, + "union_f1": 0.6989921615406495, + "difference": 0.6886984637018222, + "predicted_positive_rate": 0.14598421885821708, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6989921615406495 + "path": "", + "source_priority": 2 }, { - "difference": 0.615867982755411, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.037147804576376, - "scope": "full_domain", + "label": "ClimaX", "seed": 42, - "source_priority": 2, - "strict_f1": 0.004637761701086003, - "threshold": 0.0014632086704014654, + "scope": "full_domain", "threshold_key": "0.0014632087", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0014632086704014654, + "strict_f1": 0.004637761701086003, "tolerance_f1": 0.31289270592627744, + "union_f1": 0.620505744456497, + "difference": 0.615867982755411, + "predicted_positive_rate": 0.037147804576376, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.620505744456497 + "path": "", + "source_priority": 2 }, { - "difference": 0.6028322641867834, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.14358118135945386, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 42, - "source_priority": 2, - "strict_f1": 0.014246556800108287, - "threshold": 0.0014632082671483923, + "scope": "train_fire_top05pct", "threshold_key": "0.0014632083", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0014632082671483923, + "strict_f1": 0.014246556800108287, "tolerance_f1": 0.32161191611096424, + "union_f1": 0.6170788209868917, + "difference": 0.6028322641867834, + "predicted_positive_rate": 0.14358118135945386, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6170788209868917 + "path": "", + "source_priority": 2 }, { - "difference": 0.5999371340636104, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.07179059067972693, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 42, - "source_priority": 2, - "strict_f1": 0.014244146704560834, - "threshold": 0.0014632082671483923, + "scope": "train_fire_top10pct", "threshold_key": "0.0014632083", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0014632082671483923, + "strict_f1": 0.014244146704560834, "tolerance_f1": 0.3208182317552862, + "union_f1": 0.6141812807681712, + "difference": 0.5999371340636104, + "predicted_positive_rate": 0.07179059067972693, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6141812807681712 + "path": "", + "source_priority": 2 }, { - "difference": 0.46922312530335947, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.025392455163883734, - "scope": "full_domain", + "label": "ClimaX", "seed": 7, - "source_priority": 2, - "strict_f1": 0.0027501821392557145, - "threshold": 0.0013572092986540852, + "scope": "full_domain", "threshold_key": "0.0013572093", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0013572092986540852, + "strict_f1": 0.0027501821392557145, "tolerance_f1": 0.24162386260287075, + "union_f1": 0.4719733074426152, + "difference": 0.46922312530335947, + "predicted_positive_rate": 0.025392455163883734, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4719733074426152 + "path": "", + "source_priority": 2 }, { - "difference": 0.7267635176891648, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.3068887899475611, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 7, - "source_priority": 2, - "strict_f1": 0.013106997858124742, - "threshold": 0.0007465386633598005, + "scope": "train_fire_top05pct", "threshold_key": "0.00074653866", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0007465386633598005, + "strict_f1": 0.013106997858124742, "tolerance_f1": 0.3655334236435691, + "union_f1": 0.7398705155472896, + "difference": 0.7267635176891648, + "predicted_positive_rate": 0.3068887899475611, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7398705155472896 + "path": "", + "source_priority": 2 }, { - "difference": 0.7230353684802844, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.15368556446027506, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 7, - "source_priority": 2, - "strict_f1": 0.01308555880921415, - "threshold": 0.0007519771209341887, + "scope": "train_fire_top10pct", "threshold_key": "0.00075197712", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0007519771209341887, + "strict_f1": 0.01308555880921415, "tolerance_f1": 0.36399388919405107, + "union_f1": 0.7361209272894985, + "difference": 0.7230353684802844, + "predicted_positive_rate": 0.15368556446027506, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7361209272894985 + "path": "", + "source_priority": 2 }, { - "difference": 0.664031416861268, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.055986765615337046, - "scope": "full_domain", + "label": "ClimaX", "seed": 99, - "source_priority": 2, - "strict_f1": 0.003717439410136951, - "threshold": 0.001021229882830197, + "scope": "full_domain", "threshold_key": "0.0010212299", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.001021229882830197, + "strict_f1": 0.003717439410136951, "tolerance_f1": 0.33641526956922785, + "union_f1": 0.6677488562714049, + "difference": 0.664031416861268, + "predicted_positive_rate": 0.055986765615337046, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6677488562714049 + "path": "", + "source_priority": 2 }, { - "difference": 0.7272887839717904, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.3090951815573365, - "scope": "train_fire_top05pct", + "label": "ClimaX", "seed": 99, - "source_priority": 2, - "strict_f1": 0.013585576434738445, - "threshold": 0.0006166165592580956, + "scope": "train_fire_top05pct", "threshold_key": "0.00061661656", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0006166165592580956, + "strict_f1": 0.013585576434738445, "tolerance_f1": 0.3665570668664193, + "union_f1": 0.7408743604065289, + "difference": 0.7272887839717904, + "predicted_positive_rate": 0.3090951815573365, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7408743604065289 + "path": "", + "source_priority": 2 }, { - "difference": 0.7230514919507499, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.15408504007123774, - "scope": "train_fire_top10pct", + "label": "ClimaX", "seed": 99, - "source_priority": 2, - "strict_f1": 0.013497596536465794, - "threshold": 0.0006294738670532621, + "scope": "train_fire_top10pct", "threshold_key": "0.00062947387", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0006294738670532621, + "strict_f1": 0.013497596536465794, "tolerance_f1": 0.36524448333846904, + "union_f1": 0.7365490884872157, + "difference": 0.7230514919507499, + "predicted_positive_rate": 0.15408504007123774, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.7365490884872157 + "path": "", + "source_priority": 2 }, { - "difference": 0.1624675064035813, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.21222844774273344, - "scope": "full_domain", + "model_tag": "pangu24", + "label": "Pangu-Weather", "seed": 1, - "source_priority": 2, - "strict_f1": 0.0005518596739603732, - "threshold": 0.00027441306156106293, + "scope": "full_domain", "threshold_key": "0.00027441306", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027441306156106293, + "strict_f1": 0.0005518596739603732, "tolerance_f1": 0.06046181002685303, + "union_f1": 0.16301936607754167, + "difference": 0.1624675064035813, + "predicted_positive_rate": 0.21222844774273344, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16301936607754167 + "path": "", + "source_priority": 2 }, { - "difference": 0.4598793069893796, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.08085485307212822, - "scope": "train_fire_top05pct", "seed": 1, - "source_priority": 2, - "strict_f1": 0.026612927125626056, - "threshold": 0.00027438640245236456, + "scope": "train_fire_top05pct", "threshold_key": "0.0002743864", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027438640245236456, + "strict_f1": 0.026612927125626056, "tolerance_f1": 0.24179585238367046, + "union_f1": 0.4864922341150057, + "difference": 0.4598793069893796, + "predicted_positive_rate": 0.08085485307212822, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4864922341150057 + "path": "", + "source_priority": 2 }, { - "difference": 0.4574594224967701, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04042742653606411, - "scope": "train_fire_top10pct", "seed": 1, - "source_priority": 2, - "strict_f1": 0.02660504251473943, - "threshold": 0.00027438640245236456, + "scope": "train_fire_top10pct", "threshold_key": "0.0002743864", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027438640245236456, + "strict_f1": 0.02660504251473943, "tolerance_f1": 0.2411933776075557, + "union_f1": 0.4840644650115095, + "difference": 0.4574594224967701, + "predicted_positive_rate": 0.04042742653606411, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4840644650115095 + "path": "", + "source_priority": 2 }, { - "difference": 0.24509851810436645, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7109064935064935, - "scope": "full_domain", "seed": 123, - "source_priority": 2, - "strict_f1": 0.0006508233558970649, - "threshold": 0.0002703598001971841, + "scope": "full_domain", "threshold_key": "0.0002703598", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002703598001971841, + "strict_f1": 0.0006508233558970649, "tolerance_f1": 0.09112517773927671, + "union_f1": 0.2457493414602635, + "difference": 0.24509851810436645, + "predicted_positive_rate": 0.7109064935064935, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2457493414602635 + "path": "", + "source_priority": 2 }, { - "difference": 0.38912034744679525, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9211709706144257, - "scope": "train_fire_top05pct", "seed": 123, - "source_priority": 2, - "strict_f1": 0.005639067545640032, - "threshold": 0.0002697413437999785, + "scope": "train_fire_top05pct", "threshold_key": "0.00026974134", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002697413437999785, + "strict_f1": 0.005639067545640032, "tolerance_f1": 0.16012924166858108, + "union_f1": 0.3947594149924353, + "difference": 0.38912034744679525, + "predicted_positive_rate": 0.9211709706144257, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3947594149924353 + "path": "", + "source_priority": 2 }, { - "difference": 0.2284057583930383, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9564299495399228, - "scope": "train_fire_top10pct", "seed": 123, - "source_priority": 2, - "strict_f1": 0.0027453654202759313, - "threshold": 0.0002697413437999785, + "scope": "train_fire_top10pct", "threshold_key": "0.00026974134", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002697413437999785, + "strict_f1": 0.0027453654202759313, "tolerance_f1": 0.08722526453259725, + "union_f1": 0.23115112381331424, + "difference": 0.2284057583930383, + "predicted_positive_rate": 0.9564299495399228, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23115112381331424 + "path": "", + "source_priority": 2 }, { - "difference": 0.2016630331978699, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.8827594310451453, - "scope": "full_domain", "seed": 42, - "source_priority": 2, - "strict_f1": 0.0005395660023552084, - "threshold": 0.00024381681578233838, + "scope": "full_domain", "threshold_key": "0.00024381682", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00024381681578233838, + "strict_f1": 0.0005395660023552084, "tolerance_f1": 0.07610963330033767, + "union_f1": 0.20220259920022512, + "difference": 0.2016630331978699, + "predicted_positive_rate": 0.8827594310451453, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.20220259920022512 + "path": "", + "source_priority": 2 }, { - "difference": 0.3868339716949791, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9162387454239636, - "scope": "train_fire_top05pct", "seed": 42, - "source_priority": 2, - "strict_f1": 0.00568010509540423, - "threshold": 0.0002431461471132934, + "scope": "train_fire_top05pct", "threshold_key": "0.00024314615", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002431461471132934, + "strict_f1": 0.00568010509540423, "tolerance_f1": 0.16063421440737302, + "union_f1": 0.3925140767903833, + "difference": 0.3868339716949791, + "predicted_positive_rate": 0.9162387454239636, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.3925140767903833 + "path": "", + "source_priority": 2 }, { - "difference": 0.2276584353760274, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9562827743148313, - "scope": "train_fire_top10pct", "seed": 42, - "source_priority": 2, - "strict_f1": 0.002750953470140531, - "threshold": 0.00024276715703308582, + "scope": "train_fire_top10pct", "threshold_key": "0.00024276716", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00024276715703308582, + "strict_f1": 0.002750953470140531, "tolerance_f1": 0.08724283500251707, + "union_f1": 0.23040938884616793, + "difference": 0.2276584353760274, + "predicted_positive_rate": 0.9562827743148313, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23040938884616793 + "path": "", + "source_priority": 2 }, { - "difference": 0.24742507585853188, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7071789734075449, - "scope": "full_domain", "seed": 7, - "source_priority": 2, - "strict_f1": 0.0006577494583008349, - "threshold": 0.0002784867538139224, + "scope": "full_domain", "threshold_key": "0.00027848675", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002784867538139224, + "strict_f1": 0.0006577494583008349, "tolerance_f1": 0.09170559714407109, + "union_f1": 0.2480828253168327, + "difference": 0.24742507585853188, + "predicted_positive_rate": 0.7071789734075449, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2480828253168327 + "path": "", + "source_priority": 2 }, { - "difference": 0.38914191998978065, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9213416444048679, - "scope": "train_fire_top05pct", "seed": 7, - "source_priority": 2, - "strict_f1": 0.005648734389013078, - "threshold": 0.0002780304348561913, + "scope": "train_fire_top05pct", "threshold_key": "0.00027803043", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002780304348561913, + "strict_f1": 0.005648734389013078, "tolerance_f1": 0.1601110414083686, + "union_f1": 0.39479065437879374, + "difference": 0.38914191998978065, + "predicted_positive_rate": 0.9213416444048679, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39479065437879374 + "path": "", + "source_priority": 2 }, { - "difference": 0.2284351905540453, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9565152864351439, - "scope": "train_fire_top10pct", "seed": 7, - "source_priority": 2, - "strict_f1": 0.00275028568108307, - "threshold": 0.0002780304348561913, + "scope": "train_fire_top10pct", "threshold_key": "0.00027803043", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002780304348561913, + "strict_f1": 0.00275028568108307, "tolerance_f1": 0.08722255267001126, + "union_f1": 0.23118547623512836, + "difference": 0.2284351905540453, + "predicted_positive_rate": 0.9565152864351439, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23118547623512836 + "path": "", + "source_priority": 2 }, { - "difference": 0.24307011894354136, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.655947804576376, - "scope": "full_domain", "seed": 99, - "source_priority": 2, - "strict_f1": 0.000687991383709005, - "threshold": 0.00027882601716555655, + "scope": "full_domain", "threshold_key": "0.00027882602", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027882601716555655, + "strict_f1": 0.000687991383709005, "tolerance_f1": 0.08961425169301492, + "union_f1": 0.24375811032725037, + "difference": 0.24307011894354136, + "predicted_positive_rate": 0.655947804576376, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.24375811032725037 + "path": "", + "source_priority": 2 }, { - "difference": 0.27868986084470576, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.28134213911150685, - "scope": "train_fire_top05pct", "seed": 99, - "source_priority": 2, - "strict_f1": 0.009651904247534756, - "threshold": 0.00028155450127087533, + "scope": "train_fire_top05pct", "threshold_key": "0.0002815545", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028155450127087533, + "strict_f1": 0.009651904247534756, "tolerance_f1": 0.1110438397786047, + "union_f1": 0.2883417650922405, + "difference": 0.27868986084470576, + "predicted_positive_rate": 0.28134213911150685, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2883417650922405 + "path": "", + "source_priority": 2 }, { - "difference": 0.02818447450809833, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0007742158899772435, - "scope": "train_fire_top10pct", "seed": 99, - "source_priority": 2, - "strict_f1": 0.0011827321111768183, - "threshold": 0.000285720539977774, + "scope": "train_fire_top10pct", "threshold_key": "0.00028572054", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000285720539977774, + "strict_f1": 0.0011827321111768183, "tolerance_f1": 0.019439944457301548, + "union_f1": 0.029367206619275145, + "difference": 0.02818447450809833, + "predicted_positive_rate": 0.0007742158899772435, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.029367206619275145 + "path": "", + "source_priority": 2 }, { - "difference": 0.2215775413347868, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.5006159554730983, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 2, - "strict_f1": 0.0006040339888888368, - "threshold": 0.00028278783429414034, + "scope": "full_domain", "threshold_key": "0.00028278783", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028278783429414034, + "strict_f1": 0.0006040339888888368, "tolerance_f1": 0.07892160427759505, + "union_f1": 0.22218157532367563, + "difference": 0.2215775413347868, + "predicted_positive_rate": 0.5006159554730983, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22218157532367563 + "path": "", + "source_priority": 2 }, { - "difference": 0.38844131515497954, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 2, - "strict_f1": 0.005636060100166944, - "threshold": 0.00018521470337873325, + "scope": "train_fire_top05pct", "threshold_key": "0.0001852147", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00018521470337873325, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 2 }, { - "difference": 0.2282163144290469, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 2, - "strict_f1": 0.002747305898968793, - "threshold": 0.00018521470337873325, + "scope": "train_fire_top10pct", "threshold_key": "0.0001852147", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00018521470337873325, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 2 }, { - "difference": 0.20020081119628272, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9588571428571429, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 2, - "strict_f1": 0.0005057825179019543, - "threshold": 0.00028123805532231927, + "scope": "full_domain", "threshold_key": "0.00028123806", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028123805532231927, + "strict_f1": 0.0005057825179019543, "tolerance_f1": 0.07212988048730175, + "union_f1": 0.20070659371418467, + "difference": 0.20020081119628272, + "predicted_positive_rate": 0.9588571428571429, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.20070659371418467 + "path": "", + "source_priority": 2 }, { - "difference": 0.38844131515497954, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 2, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002438416049699299, + "scope": "train_fire_top05pct", "threshold_key": "0.0002438416", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002438416049699299, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 2 }, { - "difference": 0.2282163144290469, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", - "seed": 123, - "source_priority": 2, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002438416049699299, + "label": "Prithvi-WxC", + "seed": 123, + "scope": "train_fire_top10pct", "threshold_key": "0.0002438416", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002438416049699299, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 2 }, { - "difference": 0.17211673370575886, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.22581076066790354, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 2, - "strict_f1": 0.0005777939978365554, - "threshold": 0.00028831177041865885, + "scope": "full_domain", "threshold_key": "0.00028831177", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028831177041865885, + "strict_f1": 0.0005777939978365554, "tolerance_f1": 0.06182533987250242, + "union_f1": 0.17269452770359542, + "difference": 0.17211673370575886, + "predicted_positive_rate": 0.22581076066790354, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17269452770359542 + "path": "", + "source_priority": 2 }, { - "difference": 0.46663155596188977, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.08637577916295637, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 2, - "strict_f1": 0.02590687977762335, - "threshold": 0.0002883114793803543, + "scope": "train_fire_top05pct", "threshold_key": "0.00028831148", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002883114793803543, + "strict_f1": 0.02590687977762335, "tolerance_f1": 0.23967146506700618, + "union_f1": 0.4925384357395131, + "difference": 0.46663155596188977, + "predicted_positive_rate": 0.08637577916295637, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4925384357395131 + "path": "", + "source_priority": 2 }, { - "difference": 0.46415551438013897, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.043187889581478185, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 2, - "strict_f1": 0.02589968042239822, - "threshold": 0.0002883114793803543, + "scope": "train_fire_top10pct", "threshold_key": "0.00028831148", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002883114793803543, + "strict_f1": 0.02589968042239822, "tolerance_f1": 0.23908194736539978, + "union_f1": 0.4900551948025372, + "difference": 0.46415551438013897, + "predicted_positive_rate": 0.043187889581478185, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4900551948025372 + "path": "", + "source_priority": 2 }, { - "difference": 0.20983886056574044, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9099814471243043, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 2, - "strict_f1": 0.000532941187181528, - "threshold": 0.00028816089616157115, + "scope": "full_domain", "threshold_key": "0.0002881609", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028816089616157115, + "strict_f1": 0.000532941187181528, "tolerance_f1": 0.07586161173658511, + "union_f1": 0.21037180175292197, + "difference": 0.20983886056574044, + "predicted_positive_rate": 0.9099814471243043, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21037180175292197 + "path": "", + "source_priority": 2 }, { - "difference": 0.38844131515497954, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 2, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002606456328066997, + "scope": "train_fire_top05pct", "threshold_key": "0.00026064563", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002606456328066997, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 2 }, { - "difference": 0.2282163144290469, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 2, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002606456328066997, + "scope": "train_fire_top10pct", "threshold_key": "0.00026064563", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002606456328066997, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 2 }, { - "difference": 0.20276874034790468, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.5032875695732839, - "scope": "full_domain", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 2, - "strict_f1": 0.0005418761442097979, - "threshold": 0.0002606518683023751, + "scope": "full_domain", "threshold_key": "0.00026065187", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002606518683023751, + "strict_f1": 0.0005418761442097979, "tolerance_f1": 0.06950848282026648, + "union_f1": 0.2033106164921145, + "difference": 0.20276874034790468, + "predicted_positive_rate": 0.5032875695732839, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2033106164921145 + "path": "", + "source_priority": 2 }, { - "difference": 0.4264130583920869, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.013950727218759276, - "scope": "train_fire_top05pct", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 2, - "strict_f1": 0.027781926811053025, - "threshold": 0.0002606526104500517, + "scope": "train_fire_top05pct", "threshold_key": "0.00026065261", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002606526104500517, + "strict_f1": 0.027781926811053025, "tolerance_f1": 0.24416957191197788, + "union_f1": 0.4541949852031399, + "difference": 0.4264130583920869, + "predicted_positive_rate": 0.013950727218759276, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4541949852031399 + "path": "", + "source_priority": 2 }, { - "difference": 0.4238775751761356, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.006975363609379638, - "scope": "train_fire_top10pct", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 2, - "strict_f1": 0.02774049217002237, - "threshold": 0.0002606526104500517, + "scope": "train_fire_top10pct", "threshold_key": "0.00026065261", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002606526104500517, + "strict_f1": 0.02774049217002237, "tolerance_f1": 0.24342288270264262, + "union_f1": 0.451618067346158, + "difference": 0.4238775751761356, + "predicted_positive_rate": 0.006975363609379638, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.451618067346158 + "path": "", + "source_priority": 2 }, { - "difference": 0.12941486750349504, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.3248534322820037, - "scope": "full_domain", + "label": "StormCast", "seed": 1, - "source_priority": 2, - "strict_f1": 0.00043296285460247506, - "threshold": 0.000301085616229102, + "scope": "full_domain", "threshold_key": "0.00030108562", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000301085616229102, + "strict_f1": 0.00043296285460247506, "tolerance_f1": 0.04396525865017935, + "union_f1": 0.12984783035809752, + "difference": 0.12941486750349504, + "predicted_positive_rate": 0.3248534322820037, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.12984783035809752 + "path": "", + "source_priority": 2 }, { - "difference": 0.46737645889869145, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.09290590679726922, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 1, - "source_priority": 2, - "strict_f1": 0.023876731839958566, - "threshold": 0.000301085616229102, + "scope": "train_fire_top05pct", "threshold_key": "0.00030108562", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000301085616229102, + "strict_f1": 0.023876731839958566, "tolerance_f1": 0.23852766585861737, + "union_f1": 0.49125319073865004, + "difference": 0.46737645889869145, + "predicted_positive_rate": 0.09290590679726922, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.49125319073865004 + "path": "", + "source_priority": 2 }, { - "difference": 0.4648919131947094, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.04645295339863461, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 1, - "source_priority": 2, - "strict_f1": 0.023870550161812297, - "threshold": 0.000301085616229102, + "scope": "train_fire_top10pct", "threshold_key": "0.00030108562", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000301085616229102, + "strict_f1": 0.023870550161812297, "tolerance_f1": 0.23793892088542526, + "union_f1": 0.4887624633565217, + "difference": 0.4648919131947094, + "predicted_positive_rate": 0.04645295339863461, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4887624633565217 + "path": "", + "source_priority": 2 }, { - "difference": 0.22393621200451744, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7873692022263451, - "scope": "full_domain", + "label": "StormCast", "seed": 123, - "source_priority": 2, - "strict_f1": 0.0005917233172122231, - "threshold": 0.0002731947461143136, + "scope": "full_domain", "threshold_key": "0.00027319475", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002731947461143136, + "strict_f1": 0.0005917233172122231, "tolerance_f1": 0.08255863031622387, + "union_f1": 0.22452793532172965, + "difference": 0.22393621200451744, + "predicted_positive_rate": 0.7873692022263451, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.22452793532172965 + "path": "", + "source_priority": 2 }, { - "difference": 0.38844131515497954, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9234194122885129, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 123, - "source_priority": 2, - "strict_f1": 0.005636060100166944, - "threshold": 0.0002716928720474243, + "scope": "train_fire_top05pct", "threshold_key": "0.00027169287", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002716928720474243, + "strict_f1": 0.005636060100166944, "tolerance_f1": 0.15977956000729543, + "union_f1": 0.39407737525514647, + "difference": 0.38844131515497954, + "predicted_positive_rate": 0.9234194122885129, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39407737525514647 + "path": "", + "source_priority": 2 }, { - "difference": 0.2282163144290469, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9575541703769664, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 123, - "source_priority": 2, - "strict_f1": 0.002747305898968793, - "threshold": 0.0002716928720474243, + "scope": "train_fire_top10pct", "threshold_key": "0.00027169287", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002716928720474243, + "strict_f1": 0.002747305898968793, "tolerance_f1": 0.08713204456539597, + "union_f1": 0.2309636203280157, + "difference": 0.2282163144290469, + "predicted_positive_rate": 0.9575541703769664, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2309636203280157 + "path": "", + "source_priority": 2 }, { - "difference": 0.26027328981051695, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.62891280148423, - "scope": "full_domain", + "label": "StormCast", "seed": 42, - "source_priority": 2, - "strict_f1": 0.0007179477702826634, - "threshold": 0.0002782046503853053, + "scope": "full_domain", "threshold_key": "0.00027820465", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002782046503853053, + "strict_f1": 0.0007179477702826634, "tolerance_f1": 0.09668169596405224, + "union_f1": 0.2609912375807996, + "difference": 0.26027328981051695, + "predicted_positive_rate": 0.62891280148423, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2609912375807996 + "path": "", + "source_priority": 2 }, { - "difference": 0.2505963038813504, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.43306619174829325, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 42, - "source_priority": 2, - "strict_f1": 0.006835665824509611, - "threshold": 0.00027877578395418823, + "scope": "train_fire_top05pct", "threshold_key": "0.00027877578", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027877578395418823, + "strict_f1": 0.006835665824509611, "tolerance_f1": 0.10250537509480662, + "union_f1": 0.25743196970586, + "difference": 0.2505963038813504, + "predicted_positive_rate": 0.43306619174829325, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.25743196970586 + "path": "", + "source_priority": 2 }, { - "difference": 0.2840988865146931, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7009498367468091, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 42, - "source_priority": 2, - "strict_f1": 0.00356800070444239, - "threshold": 0.00027818421949632466, + "scope": "train_fire_top10pct", "threshold_key": "0.00027818422", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027818421949632466, + "strict_f1": 0.00356800070444239, "tolerance_f1": 0.1112312688602769, + "union_f1": 0.2876668872191355, + "difference": 0.2840988865146931, + "predicted_positive_rate": 0.7009498367468091, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2876668872191355 + "path": "", + "source_priority": 2 }, { - "difference": 0.24941796232451194, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7037476808905381, - "scope": "full_domain", + "label": "StormCast", "seed": 7, - "source_priority": 2, - "strict_f1": 0.0006714969233599232, - "threshold": 0.0002892915508709848, + "scope": "full_domain", "threshold_key": "0.00028929155", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002892915508709848, + "strict_f1": 0.0006714969233599232, "tolerance_f1": 0.09264506128556889, + "union_f1": 0.25008945924787185, + "difference": 0.24941796232451194, + "predicted_positive_rate": 0.7037476808905381, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.25008945924787185 + "path": "", + "source_priority": 2 }, { - "difference": 0.38914191998978065, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9213416444048679, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 7, - "source_priority": 2, - "strict_f1": 0.005648734389013078, - "threshold": 0.0002883674460463226, + "scope": "train_fire_top05pct", "threshold_key": "0.00028836745", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002883674460463226, + "strict_f1": 0.005648734389013078, "tolerance_f1": 0.1601110414083686, + "union_f1": 0.39479065437879374, + "difference": 0.38914191998978065, + "predicted_positive_rate": 0.9213416444048679, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.39479065437879374 + "path": "", + "source_priority": 2 }, { - "difference": 0.2284351905540453, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9565152864351439, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 7, - "source_priority": 2, - "strict_f1": 0.00275028568108307, - "threshold": 0.0002883674460463226, + "scope": "train_fire_top10pct", "threshold_key": "0.00028836745", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002883674460463226, + "strict_f1": 0.00275028568108307, "tolerance_f1": 0.08722255267001126, + "union_f1": 0.23118547623512836, + "difference": 0.2284351905540453, + "predicted_positive_rate": 0.9565152864351439, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.23118547623512836 + "path": "", + "source_priority": 2 }, { - "difference": 0.2529130183198567, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.5876512059369202, - "scope": "full_domain", + "label": "StormCast", "seed": 99, - "source_priority": 2, - "strict_f1": 0.0007136363665052921, - "threshold": 0.0002819171058945358, + "scope": "full_domain", "threshold_key": "0.00028191711", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002819171058945358, + "strict_f1": 0.0007136363665052921, "tolerance_f1": 0.09390595738710937, + "union_f1": 0.253626654686362, + "difference": 0.2529130183198567, + "predicted_positive_rate": 0.5876512059369202, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.253626654686362 + "path": "", + "source_priority": 2 }, { - "difference": 0.2658654982272957, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.5093499554764025, - "scope": "train_fire_top05pct", + "label": "StormCast", "seed": 99, - "source_priority": 2, - "strict_f1": 0.005865442686314773, - "threshold": 0.000282936030998826, + "scope": "train_fire_top05pct", "threshold_key": "0.00028293603", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000282936030998826, + "strict_f1": 0.005865442686314773, "tolerance_f1": 0.10517133941552075, + "union_f1": 0.2717309409136105, + "difference": 0.2658654982272957, + "predicted_positive_rate": 0.5093499554764025, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.2717309409136105 + "path": "", + "source_priority": 2 }, { - "difference": 0.2816775567740636, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7423567824280202, - "scope": "train_fire_top10pct", + "label": "StormCast", "seed": 99, - "source_priority": 2, - "strict_f1": 0.003485751823118051, - "threshold": 0.00028128057601861656, + "scope": "train_fire_top10pct", "threshold_key": "0.00028128058", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028128057601861656, + "strict_f1": 0.003485751823118051, "tolerance_f1": 0.10982263054722373, + "union_f1": 0.28516330859718164, + "difference": 0.2816775567740636, + "predicted_positive_rate": 0.7423567824280202, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.28516330859718164 + "path": "", + "source_priority": 2 }, { - "difference": 0.44473130599773897, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.02069016697588126, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 1, - "source_priority": 3, - "strict_f1": 0.026403975040443728, - "threshold": 0.0002847809228114784, + "scope": "train_fire_top20pct", "threshold_key": "0.00028478092", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002847809228114784, + "strict_f1": 0.026403975040443728, "tolerance_f1": 0.23725772191057362, + "union_f1": 0.4711352810381827, + "difference": 0.44473130599773897, + "predicted_positive_rate": 0.02069016697588126, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4711352810381827 + "path": "", + "source_priority": 3 }, { - "difference": 0.16866850620027352, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 123, - "source_priority": 3, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 3 }, { - "difference": 0.21405981286832887, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7109764996907854, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 42, - "source_priority": 3, - "strict_f1": 0.00196209796760366, - "threshold": 0.00028130068676546216, + "scope": "train_fire_top20pct", "threshold_key": "0.00028130069", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028130068676546216, + "strict_f1": 0.00196209796760366, "tolerance_f1": 0.08141778117479781, + "union_f1": 0.21602191083593253, + "difference": 0.21405981286832887, + "predicted_positive_rate": 0.7109764996907854, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21602191083593253 + "path": "", + "source_priority": 3 }, { - "difference": 0.17276769792528876, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9366672850958565, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 7, - "source_priority": 3, - "strict_f1": 0.0014870516331508276, - "threshold": 0.0002701335761230439, + "scope": "train_fire_top20pct", "threshold_key": "0.00027013358", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002701335761230439, + "strict_f1": 0.0014870516331508276, "tolerance_f1": 0.06425881062262738, + "union_f1": 0.1742547495584396, + "difference": 0.17276769792528876, + "predicted_positive_rate": 0.9366672850958565, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1742547495584396 + "path": "", + "source_priority": 3 }, { - "difference": 0.2137212534415811, - "label": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7215015460729747, - "scope": "train_fire_top20pct", + "label": "Aurora", "seed": 99, - "source_priority": 3, - "strict_f1": 0.0019489171350575599, - "threshold": 0.0002826936251949519, + "scope": "train_fire_top20pct", "threshold_key": "0.00028269363", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002826936251949519, + "strict_f1": 0.0019489171350575599, "tolerance_f1": 0.08112237765113199, + "union_f1": 0.21567017057663868, + "difference": 0.2137212534415811, + "predicted_positive_rate": 0.7215015460729747, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21567017057663868 + "path": "", + "source_priority": 3 }, { - "difference": 0.5984324485861181, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.030718614718614718, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 1, - "source_priority": 3, - "strict_f1": 0.011488825936418005, - "threshold": 0.0015083511573617753, + "scope": "train_fire_top20pct", "threshold_key": "0.0015083512", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0015083511573617753, + "strict_f1": 0.011488825936418005, "tolerance_f1": 0.30575189275627285, + "union_f1": 0.6099212745225361, + "difference": 0.5984324485861181, + "predicted_positive_rate": 0.030718614718614718, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6099212745225361 + "path": "", + "source_priority": 3 }, { - "difference": 0.5828418325957657, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.12288311688311689, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 123, - "source_priority": 3, - "strict_f1": 0.006734646906365392, - "threshold": 0.0006820998141378855, + "scope": "train_fire_top20pct", "threshold_key": "0.00068209981", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0006820998141378855, + "strict_f1": 0.006734646906365392, "tolerance_f1": 0.2833418180729231, + "union_f1": 0.5895764795021311, + "difference": 0.5828418325957657, + "predicted_positive_rate": 0.12288311688311689, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.5895764795021311 + "path": "", + "source_priority": 3 }, { - "difference": 0.6150773048223223, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.041087198515769946, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 42, - "source_priority": 3, - "strict_f1": 0.013757803485310216, - "threshold": 0.0013755693906898965, + "scope": "train_fire_top20pct", "threshold_key": "0.0013755694", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0013755693906898965, + "strict_f1": 0.013757803485310216, "tolerance_f1": 0.32901058451573606, + "union_f1": 0.6288351083076325, + "difference": 0.6150773048223223, + "predicted_positive_rate": 0.041087198515769946, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.6288351083076325 + "path": "", + "source_priority": 3 }, { - "difference": 0.47288261881572224, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.020547928262213976, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 7, - "source_priority": 3, - "strict_f1": 0.010760193101843772, - "threshold": 0.0013572138352511622, + "scope": "train_fire_top20pct", "threshold_key": "0.0013572138", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0013572138352511622, + "strict_f1": 0.010760193101843772, "tolerance_f1": 0.24066482674710968, + "union_f1": 0.48364281191756603, + "difference": 0.47288261881572224, + "predicted_positive_rate": 0.020547928262213976, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.48364281191756603 + "path": "", + "source_priority": 3 }, { - "difference": 0.6825790409197952, - "label": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.07197278911564625, - "scope": "train_fire_top20pct", + "label": "ClimaX", "seed": 99, - "source_priority": 3, - "strict_f1": 0.008694912370256934, - "threshold": 0.0009633156281329466, + "scope": "train_fire_top20pct", "threshold_key": "0.00096331563", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0009633156281329466, + "strict_f1": 0.008694912370256934, "tolerance_f1": 0.35193063179186285, + "union_f1": 0.691273953290052, + "difference": 0.6825790409197952, + "predicted_positive_rate": 0.07197278911564625, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.691273953290052 + "path": "", + "source_priority": 3 }, { - "difference": 0.43562410120810985, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0202152133580705, - "scope": "train_fire_top20pct", "seed": 1, - "source_priority": 3, - "strict_f1": 0.026530371070668874, - "threshold": 0.00027438640245236456, + "scope": "train_fire_top20pct", "threshold_key": "0.0002743864", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00027438640245236456, + "strict_f1": 0.026530371070668874, "tolerance_f1": 0.23561614605975847, + "union_f1": 0.4621544722787787, + "difference": 0.43562410120810985, + "predicted_positive_rate": 0.0202152133580705, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4621544722787787 + "path": "", + "source_priority": 3 }, { - "difference": 0.16803346217809073, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9786487322201608, - "scope": "train_fire_top20pct", "seed": 123, - "source_priority": 3, - "strict_f1": 0.0014649840398829276, - "threshold": 0.0002300136402482167, + "scope": "train_fire_top20pct", "threshold_key": "0.00023001364", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002300136402482167, + "strict_f1": 0.0014649840398829276, "tolerance_f1": 0.062212235501567396, + "union_f1": 0.16949844621797366, + "difference": 0.16803346217809073, + "predicted_positive_rate": 0.9786487322201608, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16949844621797366 + "path": "", + "source_priority": 3 }, { - "difference": 0.16835043410148648, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9766338899196042, - "scope": "train_fire_top20pct", + "model_tag": "pangu24", + "label": "Pangu-Weather", "seed": 42, - "source_priority": 3, - "strict_f1": 0.0014680041534393378, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.0014680041534393378, "tolerance_f1": 0.062336581960243484, + "union_f1": 0.16981843825492582, + "difference": 0.16835043410148648, + "predicted_positive_rate": 0.9766338899196042, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16981843825492582 + "path": "", + "source_priority": 3 }, { - "difference": 0.16739547174950506, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9827272727272728, - "scope": "train_fire_top20pct", "seed": 7, - "source_priority": 3, - "strict_f1": 0.0014589084597825474, - "threshold": 0.00023887513088993728, + "scope": "train_fire_top20pct", "threshold_key": "0.00023887513", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00023887513088993728, + "strict_f1": 0.0014589084597825474, "tolerance_f1": 0.06196203978961812, + "union_f1": 0.1688543802092876, + "difference": 0.16739547174950506, + "predicted_positive_rate": 0.9827272727272728, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.1688543802092876 + "path": "", + "source_priority": 3 }, { - "difference": 0.08847200309630879, + "model_tag": "pangu24", "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.34681447124304265, - "scope": "train_fire_top20pct", "seed": 99, - "source_priority": 3, - "strict_f1": 0.0021531819937041672, - "threshold": 0.00028135377215221524, + "scope": "train_fire_top20pct", "threshold_key": "0.00028135377", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00028135377215221524, + "strict_f1": 0.0021531819937041672, "tolerance_f1": 0.031681254572138745, + "union_f1": 0.09062518509001295, + "difference": 0.08847200309630879, + "predicted_positive_rate": 0.34681447124304265, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.09062518509001295 + "path": "", + "source_priority": 3 }, { - "difference": 0.16866850620027352, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 1, - "source_priority": 3, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002, + "scope": "train_fire_top20pct", "threshold_key": "0.0002", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 3 }, { - "difference": 0.16866850620027352, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 123, - "source_priority": 3, - "strict_f1": 0.001471035812113219, - "threshold": 0.0002438417504890822, + "scope": "train_fire_top20pct", "threshold_key": "0.00024384175", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002438417504890822, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 3 }, { - "difference": 0.44182468955354215, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.021595547309833023, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 42, - "source_priority": 3, - "strict_f1": 0.025831485587583147, - "threshold": 0.0002883114793803543, + "scope": "train_fire_top20pct", "threshold_key": "0.00028831148", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002883114793803543, + "strict_f1": 0.025831485587583147, "tolerance_f1": 0.23362286716686329, + "union_f1": 0.4676561751411253, + "difference": 0.44182468955354215, + "predicted_positive_rate": 0.021595547309833023, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4676561751411253 + "path": "", + "source_priority": 3 }, { - "difference": 0.16866850620027352, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.9746196660482375, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 7, - "source_priority": 3, - "strict_f1": 0.001471035812113219, - "threshold": 0.000260645778325852, + "scope": "train_fire_top20pct", "threshold_key": "0.00026064578", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.000260645778325852, + "strict_f1": 0.001471035812113219, "tolerance_f1": 0.06246138809231819, + "union_f1": 0.17013954201238674, + "difference": 0.16866850620027352, + "predicted_positive_rate": 0.9746196660482375, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.17013954201238674 + "path": "", + "source_priority": 3 }, { - "difference": 0.4011682044338078, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.0034879406307977735, - "scope": "train_fire_top20pct", + "label": "Prithvi-WxC", "seed": 99, - "source_priority": 3, - "strict_f1": 0.027352941176470587, - "threshold": 0.0002606526104500517, + "scope": "train_fire_top20pct", "threshold_key": "0.00026065261", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002606526104500517, + "strict_f1": 0.027352941176470587, "tolerance_f1": 0.2365506772536198, + "union_f1": 0.4285211456102784, + "difference": 0.4011682044338078, + "predicted_positive_rate": 0.0034879406307977735, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4285211456102784 + "path": "", + "source_priority": 3 }, { - "difference": 0.4425322203556718, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.020630797773654917, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 1, - "source_priority": 3, - "strict_f1": 0.022074159907300116, - "threshold": 0.00030111230444163084, + "scope": "train_fire_top20pct", "threshold_key": "0.0003011123", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00030111230444163084, + "strict_f1": 0.022074159907300116, "tolerance_f1": 0.23443811051591282, + "union_f1": 0.4646063802629719, + "difference": 0.4425322203556718, + "predicted_positive_rate": 0.020630797773654917, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.4646063802629719 + "path": "", + "source_priority": 3 }, { - "difference": 0.16790877204956894, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.979443413729128, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 123, - "source_priority": 3, - "strict_f1": 0.0014637962799384197, - "threshold": 0.0002497032401151955, + "scope": "train_fire_top20pct", "threshold_key": "0.00024970324", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002497032401151955, + "strict_f1": 0.0014637962799384197, "tolerance_f1": 0.062163327857870446, + "union_f1": 0.16937256832950737, + "difference": 0.16790877204956894, + "predicted_positive_rate": 0.979443413729128, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16937256832950737 + "path": "", + "source_priority": 3 }, { - "difference": 0.21164756227103257, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.6907606679035251, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 42, - "source_priority": 3, - "strict_f1": 0.0020248273888312522, - "threshold": 0.0002781348885037005, + "scope": "train_fire_top20pct", "threshold_key": "0.00027813489", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002781348885037005, + "strict_f1": 0.0020248273888312522, "tolerance_f1": 0.08079366172039935, + "union_f1": 0.21367238965986382, + "difference": 0.21164756227103257, + "predicted_positive_rate": 0.6907606679035251, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21367238965986382 + "path": "", + "source_priority": 3 }, { - "difference": 0.16790877204956894, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.979443413729128, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 7, - "source_priority": 3, - "strict_f1": 0.0014637962799384197, - "threshold": 0.00021927931811660528, + "scope": "train_fire_top20pct", "threshold_key": "0.00021927932", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.00021927931811660528, + "strict_f1": 0.0014637962799384197, "tolerance_f1": 0.062163327857870446, + "union_f1": 0.16937256832950737, + "difference": 0.16790877204956894, + "predicted_positive_rate": 0.979443413729128, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.16937256832950737 + "path": "", + "source_priority": 3 }, { - "difference": 0.21401759488544256, - "label": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "predicted_positive_rate": 0.7217068645640075, - "scope": "train_fire_top20pct", + "label": "StormCast", "seed": 99, - "source_priority": 3, - "strict_f1": 0.001948363237912615, - "threshold": 0.0002812395105138421, + "scope": "train_fire_top20pct", "threshold_key": "0.00028123951", "threshold_selector": "strict_f1_nondegenerate", - "tolerance_combo": "t0_s8", + "threshold": 0.0002812395105138421, + "strict_f1": 0.001948363237912615, "tolerance_f1": 0.08122801407099456, + "union_f1": 0.21596595812335517, + "difference": 0.21401759488544256, + "predicted_positive_rate": 0.7217068645640075, + "tolerance_combo": "t0_s8", "union_combo": "t3_s8", - "union_f1": 0.21596595812335517 + "path": "", + "source_priority": 3 } ], - "run_roots": [ - "REDACTED_LOCAL_RESULT_PATH" - ], - "seed_variation_issues": [], "summary": [ { + "model_tag": "alphaearth", + "label": "AlphaEarth", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.02060627238126987, + "std": 0.004404166549372983, + "min": 0.015582982831138383, + "max": 0.02585193889541716 + }, + "tolerance_f1": { + "n": 5, + "mean": 0.2944756804111831, + "std": 0.06006423929515853, + "min": 0.19137568834619276, + "max": 0.3412019506417814 + }, + "union_f1": { + "n": 5, + "mean": 0.3742857130975682, + "std": 0.09945849036660906, + "min": 0.21572734820321318, + "max": 0.4879949976282558 + }, "difference": { - "max": 0.4724120147971174, + "n": 5, "mean": 0.3536794407162983, + "std": 0.10027055938000065, "min": 0.1971458843464264, - "n": 5, - "std": 0.10027055938000065 + "max": 0.4724120147971174 }, - "label": "AlphaEarth", - "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.0015512677798392085, + "n": 5, "mean": 0.000691230674087817, + "std": 0.0004992791583911452, "min": 0.00030327767470624613, - "n": 5, - "std": 0.0004992791583911452 + "max": 0.0015512677798392085 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.38916402243638726 + "value": 0.01841620626151013, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1971458843464264 + "value": 0.018581463856786766, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3681553641584426 + "value": 0.024598770061496925, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.3415199178431181 + "value": 0.02585193889541716, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.4724120147971174 + "value": 0.015582982831138383, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0006305504019789734 + "value": 0.3233671389763082, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00030327767470624613 + "value": 0.19137568834619276, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0005820655534941249 + "value": 0.3224566312114399, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.00038899196042053183 + "value": 0.29397699288019324, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0015512677798392085 + "value": 0.3412019506417814, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.01841620626151013 + "value": 0.4075802286978974, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.018581463856786766 + "value": 0.21572734820321318, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.024598770061496925 + "value": 0.3927541342199395, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.02585193889541716 + "value": 0.36737185673853523, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.015582982831138383 + "value": 0.4879949976282558, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3233671389763082 + "value": 0.38916402243638726, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.19137568834619276 + "value": 0.1971458843464264, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3224566312114399 + "value": 0.3681553641584426, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.29397699288019324 + "value": 0.3415199178431181, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.3412019506417814 + "value": 0.4724120147971174, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4075802286978974 + "value": 0.0006305504019789734, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.21572734820321318 + "value": 0.00030327767470624613, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3927541342199395 + "value": 0.0005820655534941249, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.36737185673853523 + "value": 0.00038899196042053183, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.4879949976282558 + "value": 0.0015512677798392085, + "path": "" } ] }, - "strict_f1": { - "max": 0.02585193889541716, - "mean": 0.02060627238126987, - "min": 0.015582982831138383, - "n": 5, - "std": 0.004404166549372983 - }, "thresholds": [ 0.005, 0.01, 0.02 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "alphaearth", + "label": "AlphaEarth", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.06913347589828964, + "std": 0.008450070468719153, + "min": 0.05801376597836775, + "max": 0.07774936061381074 + }, "tolerance_f1": { - "max": 0.3412019506417814, - "mean": 0.2944756804111831, - "min": 0.19137568834619276, "n": 5, - "std": 0.06006423929515853 + "mean": 0.42878982650540687, + "std": 0.04608715441056064, + "min": 0.348489928787613, + "max": 0.46103410735073025 }, "union_f1": { - "max": 0.4879949976282558, - "mean": 0.3742857130975682, - "min": 0.21572734820321318, - "n": 5, - "std": 0.09945849036660906 - } - }, - { - "difference": { - "max": 0.572037609102215, - "mean": 0.44831513328161643, - "min": 0.32645966955454164, "n": 5, - "std": 0.09076276520267298 + "mean": 0.5174486091799061, + "std": 0.08732060818913379, + "min": 0.39499846060642174, + "max": 0.6362946372146648 + }, + "difference": { + "n": 5, + "mean": 0.44831513328161643, + "std": 0.09076276520267298, + "min": 0.32645966955454164, + "max": 0.572037609102215 }, - "label": "AlphaEarth", - "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.005397249431087365, + "n": 5, "mean": 0.002825764321757198, + "std": 0.0014965985811042732, "min": 0.001496487582863362, - "n": 5, - "std": 0.0014965985811042732 + "max": 0.005397249431087365 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4907581884518499 + "value": 0.05801376597836775, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.32645966955454164 + "value": 0.06853879105188007, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.42865099938480467 + "value": 0.07774936061381074, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.42366919991467084 + "value": 0.07710843373493977, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.572037609102215 + "value": 0.06425702811244981, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0024215889977243494 + "value": 0.45453465315678554, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0025873157217769865 + "value": 0.348489928787613, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.002226179875333927 + "value": 0.46103410735073025, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.001496487582863362 + "value": 0.4329934518241347, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.005397249431087365 + "value": 0.44689699140777084, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.05801376597836775 + "value": 0.5487719544302176, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.06853879105188007 + "value": 0.39499846060642174, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.07774936061381074 + "value": 0.5064003599986154, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.07710843373493977 + "value": 0.5007776336496106, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.06425702811244981 + "value": 0.6362946372146648, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.45453465315678554 + "value": 0.4907581884518499, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.348489928787613 + "value": 0.32645966955454164, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.46103410735073025 + "value": 0.42865099938480467, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4329934518241347 + "value": 0.42366919991467084, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.44689699140777084 + "value": 0.572037609102215, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5487719544302176 + "value": 0.0024215889977243494, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.39499846060642174 + "value": 0.0025873157217769865, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.5064003599986154 + "value": 0.002226179875333927, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5007776336496106 + "value": 0.001496487582863362, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6362946372146648 + "value": 0.005397249431087365, + "path": "" } ] }, - "strict_f1": { - "max": 0.07774936061381074, - "mean": 0.06913347589828964, - "min": 0.05801376597836775, - "n": 5, - "std": 0.008450070468719153 - }, "thresholds": [ 0.005, 0.01 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "alphaearth", + "label": "AlphaEarth", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0663656378441958, + "std": 0.009901307723431575, + "min": 0.05725646123260437, + "max": 0.07711821410451548 + }, "tolerance_f1": { - "max": 0.46103410735073025, - "mean": 0.42878982650540687, - "min": 0.348489928787613, "n": 5, - "std": 0.04608715441056064 + "mean": 0.41898143755043915, + "std": 0.05945364415661427, + "min": 0.31417824074074074, + "max": 0.4570756272994282 }, "union_f1": { - "max": 0.6362946372146648, - "mean": 0.5174486091799061, - "min": 0.39499846060642174, "n": 5, - "std": 0.08732060818913379 - } - }, - { + "mean": 0.5057119561083213, + "std": 0.10005714147402384, + "min": 0.3547275267815556, + "max": 0.6310773342718137 + }, "difference": { - "max": 0.5674882177936724, + "n": 5, "mean": 0.43934631826412546, + "std": 0.0991562015356911, "min": 0.29747106554895125, - "n": 5, - "std": 0.0991562015356911 + "max": 0.5674882177936724 }, - "label": "AlphaEarth", - "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.002728307113881468, + "n": 5, "mean": 0.001524438507964777, + "std": 0.0007698172196033509, "min": 0.0007507173246264965, - "n": 5, - "std": 0.0007698172196033509 + "max": 0.002728307113881468 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.48639654658611775 + "value": 0.05730937348227295, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.29747106554895125 + "value": 0.05725646123260437, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.42494003459095464 + "value": 0.07711821410451548, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.42043572680093133 + "value": 0.07655502392344497, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5674882177936724 + "value": 0.06358911647814125, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0012293459978232908 + "value": 0.4503813713224908, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.001793311566241219 + "value": 0.31417824074074074, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.00112051053725141 + "value": 0.4570756272994282, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0007507173246264965 + "value": 0.4297532170909699, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.002728307113881468 + "value": 0.4435187312985661, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.05730937348227295 + "value": 0.5437059200683907, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.05725646123260437 + "value": 0.3547275267815556, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.07711821410451548 + "value": 0.5020582486954701, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.07655502392344497 + "value": 0.4969907507243763, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.06358911647814125 + "value": 0.6310773342718137, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4503813713224908 + "value": 0.48639654658611775, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.31417824074074074 + "value": 0.29747106554895125, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4570756272994282 + "value": 0.42494003459095464, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4297532170909699 + "value": 0.42043572680093133, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.4435187312985661 + "value": 0.5674882177936724, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5437059200683907 + "value": 0.0012293459978232908, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3547275267815556 + "value": 0.001793311566241219, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.5020582486954701 + "value": 0.00112051053725141, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4969907507243763 + "value": 0.0007507173246264965, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6310773342718137 + "value": 0.002728307113881468, + "path": "" } ] }, - "strict_f1": { - "max": 0.07711821410451548, - "mean": 0.0663656378441958, - "min": 0.05725646123260437, - "n": 5, - "std": 0.009901307723431575 - }, "thresholds": [ 0.005, 0.01 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "alphaearth", + "label": "AlphaEarth", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.06190818589945194, + "std": 0.011329878082076436, + "min": 0.047398843930635835, + "max": 0.07357212003872217 + }, "tolerance_f1": { - "max": 0.4570756272994282, - "mean": 0.41898143755043915, - "min": 0.31417824074074074, "n": 5, - "std": 0.05945364415661427 + "mean": 0.38832498285553635, + "std": 0.07496564230623358, + "min": 0.25536195152859176, + "max": 0.42986611450839224 }, "union_f1": { - "max": 0.6310773342718137, - "mean": 0.5057119561083213, - "min": 0.3547275267815556, "n": 5, - "std": 0.10005714147402384 - } - }, - { + "mean": 0.4638327916863426, + "std": 0.12169697847221474, + "min": 0.2676781973617166, + "max": 0.5992872540844005 + }, "difference": { - "max": 0.5375844952591261, + "n": 5, "mean": 0.4019246057868907, + "std": 0.11678824405387903, "min": 0.22027935343108077, - "n": 5, - "std": 0.11678824405387903 + "max": 0.5375844952591261 }, - "label": "AlphaEarth", - "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.0013673469387755102, + "n": 5, "mean": 0.0006557823129251701, + "std": 0.00041445475311508546, "min": 0.00035250463821892396, - "n": 5, - "std": 0.00041445475311508546 + "max": 0.0013673469387755102 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4576561443425643 + "value": 0.0544280442804428, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.22027935343108077 + "value": 0.047398843930635835, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.39876530350195205 + "value": 0.07357212003872217, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.3953377323997304 + "value": 0.0724391624221845, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5375844952591261 + "value": 0.06170275882527441, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0006233766233766234 + "value": 0.42554776574916503, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00035250463821892396 + "value": 0.25536195152859176, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0005602968460111318 + "value": 0.42986611450839224, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0003753865182436611 + "value": 0.40493051697817745, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0013673469387755102 + "value": 0.4259185655133553, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0544280442804428 + "value": 0.5120841886230071, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.047398843930635835 + "value": 0.2676781973617166, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.07357212003872217 + "value": 0.4723374235406742, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0724391624221845 + "value": 0.4677768948219149, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.06170275882527441 + "value": 0.5992872540844005, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.42554776574916503 + "value": 0.4576561443425643, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.25536195152859176 + "value": 0.22027935343108077, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.42986611450839224 + "value": 0.39876530350195205, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.40493051697817745 + "value": 0.3953377323997304, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.4259185655133553 + "value": 0.5375844952591261, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5120841886230071 + "value": 0.0006233766233766234, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2676781973617166 + "value": 0.00035250463821892396, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4723374235406742 + "value": 0.0005602968460111318, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4677768948219149 + "value": 0.0003753865182436611, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5992872540844005 + "value": 0.0013673469387755102, + "path": "" } ] }, - "strict_f1": { - "max": 0.07357212003872217, - "mean": 0.06190818589945194, - "min": 0.047398843930635835, - "n": 5, - "std": 0.011329878082076436 - }, "thresholds": [ 0.005, 0.01, 0.02 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "aurora", + "label": "Aurora", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.000655543322081318, + "std": 9.445781201845953e-05, + "min": 0.0004938156417811271, + "max": 0.0007150400212604667 + }, "tolerance_f1": { - "max": 0.42986611450839224, - "mean": 0.38832498285553635, - "min": 0.25536195152859176, "n": 5, - "std": 0.07496564230623358 + "mean": 0.08500901067877495, + "std": 0.01959379095008962, + "min": 0.05024333401294363, + "max": 0.09742777619989017 }, "union_f1": { - "max": 0.5992872540844005, - "mean": 0.4638327916863426, - "min": 0.2676781973617166, "n": 5, - "std": 0.12169697847221474 - } - }, - { + "mean": 0.23103726682311568, + "std": 0.04941752137736377, + "min": 0.14329601271947937, + "max": 0.2618373273867167 + }, "difference": { - "max": 0.2611249045411472, + "n": 5, "mean": 0.23038172350103436, + "std": 0.04932517127284255, "min": 0.14280219707769823, - "n": 5, - "std": 0.04932517127284255 + "max": 0.2611249045411472 }, - "label": "Aurora", - "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.7199758812615955, + "n": 5, "mean": 0.5558489053803339, + "std": 0.1975239087458258, "min": 0.21716512059369203, - "n": 5, - "std": 0.1975239087458258 + "max": 0.7199758812615955 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.14280219707769823 + "value": 0.0004938156417811271, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2611249045411472 + "value": 0.0007124228455695601, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2520341870433926 + "value": 0.0007076278562951189, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.25176783806212244 + "value": 0.0007150400212604667, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.24417949078081141 + "value": 0.0006488102455003172, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.21716512059369203 + "value": 0.05024333401294363, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.6671256648113791 + "value": 0.09742777619989017, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.5915943104514533 + "value": 0.09343380358557288, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5833835497835498 + "value": 0.09353511619384626, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7199758812615955 + "value": 0.09040502340162183, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0004938156417811271 + "value": 0.14329601271947937, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0007124228455695601 + "value": 0.2618373273867167, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0007076278562951189 + "value": 0.2527418148996877, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0007150400212604667 + "value": 0.25248287808338293, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0006488102455003172 + "value": 0.24482830102631173, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.05024333401294363 + "value": 0.14280219707769823, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.09742777619989017 + "value": 0.2611249045411472, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.09343380358557288 + "value": 0.2520341870433926, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.09353511619384626 + "value": 0.25176783806212244, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.09040502340162183 + "value": 0.24417949078081141, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.14329601271947937 + "value": 0.21716512059369203, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2618373273867167 + "value": 0.6671256648113791, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2527418148996877 + "value": 0.5915943104514533, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.25248287808338293 + "value": 0.5833835497835498, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.24482830102631173 + "value": 0.7199758812615955, + "path": "" } ] }, - "strict_f1": { - "max": 0.0007150400212604667, - "mean": 0.000655543322081318, - "min": 0.0004938156417811271, - "n": 5, - "std": 9.445781201845953e-05 - }, "thresholds": [ 0.0002701166733984173, 0.00028167077107355, @@ -7511,192 +7526,192 @@ 0.0002832056488841772, 0.0002848515287041664 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "aurora", + "label": "Aurora", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.009858908695393357, + "std": 0.009298695250165839, + "min": 0.0053561461508737284, + "max": 0.026484309350641822 + }, "tolerance_f1": { - "max": 0.09742777619989017, - "mean": 0.08500901067877495, - "min": 0.05024333401294363, "n": 5, - "std": 0.01959379095008962 + "mean": 0.1513369496836336, + "std": 0.06082068957436595, + "min": 0.08994268846280176, + "max": 0.24351172066402033 }, "union_f1": { - "max": 0.2618373273867167, - "mean": 0.23103726682311568, - "min": 0.14329601271947937, "n": 5, - "std": 0.04941752137736377 - } - }, - { + "mean": 0.35483366745730893, + "std": 0.11019233024683481, + "min": 0.22524158122442267, + "max": 0.4964540293986394 + }, "difference": { - "max": 0.4699697200479976, + "n": 5, "mean": 0.3449747587619155, + "std": 0.10372797772336852, "min": 0.21988543507354893, - "n": 5, - "std": 0.10372797772336852 + "max": 0.4699697200479976 }, - "label": "Aurora", - "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.921086870485802, + "n": 5, "mean": 0.572238052834669, + "std": 0.35339959438797136, "min": 0.08275452656574651, - "n": 5, - "std": 0.35339959438797136 + "max": 0.921086870485802 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4699697200479976 + "value": 0.026484309350641822, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.38950157948139946 + "value": 0.00565521517042468, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.25643683344302254 + "value": 0.00618071474679909, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.21988543507354893 + "value": 0.0053561461508737284, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.3890802257636093 + "value": 0.005618158058227469, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.08275452656574651 + "value": 0.24351172066402033, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9150341347580885 + "value": 0.16052688500868942, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.48403581676066093 + "value": 0.10264712599745741, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4582789156030474 + "value": 0.08994268846280176, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.921086870485802 + "value": 0.16005632828519903, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.026484309350641822 + "value": 0.4964540293986394, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00565521517042468 + "value": 0.39515679465182413, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.00618071474679909 + "value": 0.26261754818982164, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0053561461508737284 + "value": 0.22524158122442267, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.005618158058227469 + "value": 0.39469838382183675, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.24351172066402033 + "value": 0.4699697200479976, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.16052688500868942 + "value": 0.38950157948139946, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.10264712599745741 + "value": 0.25643683344302254, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.08994268846280176 + "value": 0.21988543507354893, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16005632828519903 + "value": 0.3890802257636093, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4964540293986394 + "value": 0.08275452656574651, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.39515679465182413 + "value": 0.9150341347580885, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.26261754818982164 + "value": 0.48403581676066093, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.22524158122442267 + "value": 0.4582789156030474, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.39469838382183675 + "value": 0.921086870485802, + "path": "" } ] }, - "strict_f1": { - "max": 0.026484309350641822, - "mean": 0.009858908695393357, - "min": 0.0053561461508737284, - "n": 5, - "std": 0.009298695250165839 - }, "thresholds": [ 0.00026978866662830114, 0.0002819437358994037, @@ -7704,192 +7719,192 @@ 0.00028413085965439677, 0.0002847809228114784 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "aurora", + "label": "Aurora", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.007789958613164452, + "std": 0.01045266903938076, + "min": 0.00275028568108307, + "max": 0.026476637409113293 + }, "tolerance_f1": { - "max": 0.24351172066402033, - "mean": 0.1513369496836336, - "min": 0.08994268846280176, "n": 5, - "std": 0.06082068957436595 + "mean": 0.12738113192754766, + "std": 0.06555769355379562, + "min": 0.08722255267001126, + "max": 0.24290193233931348 }, "union_f1": { - "max": 0.4964540293986394, - "mean": 0.35483366745730893, - "min": 0.22524158122442267, - "n": 5, - "std": 0.11019233024683481 - } - }, - { - "difference": { - "max": 0.46744943745251477, - "mean": 0.2975149719293163, - "min": 0.2284351905540453, "n": 5, - "std": 0.09865622266744696 + "mean": 0.30530493054248076, + "std": 0.10884219611087281, + "min": 0.23118547623512836, + "max": 0.49392607486162804 + }, + "difference": { + "n": 5, + "mean": 0.2975149719293163, + "std": 0.09865622266744696, + "min": 0.2284351905540453, + "max": 0.46744943745251477 }, - "label": "Aurora", - "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9565152864351439, + "n": 5, "mean": 0.687941773028594, + "std": 0.37695842871011653, "min": 0.04137726328287326, - "n": 5, - "std": 0.37695842871011653 + "max": 0.9565152864351439 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.46744943745251477 + "value": 0.026476637409113293, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2284351905540453 + "value": 0.00275028568108307, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.28330440732569195 + "value": 0.0035136704264394467, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2799506337602843 + "value": 0.0034589138681033788, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2284351905540453 + "value": 0.00275028568108307, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.04137726328287326 + "value": 0.24290193233931348, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9565152864351439 + "value": 0.08722255267001126, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.735743791431681 + "value": 0.11052705132032803, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.749557237558128 + "value": 0.10903157063807417, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9565152864351439 + "value": 0.08722255267001126, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.026476637409113293 + "value": 0.49392607486162804, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00275028568108307 + "value": 0.23118547623512836, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0035136704264394467 + "value": 0.2868180777521314, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0034589138681033788 + "value": 0.28340954762838766, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.00275028568108307 + "value": 0.23118547623512836, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.24290193233931348 + "value": 0.46744943745251477, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.08722255267001126 + "value": 0.2284351905540453, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.11052705132032803 + "value": 0.28330440732569195, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.10903157063807417 + "value": 0.2799506337602843, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.08722255267001126 + "value": 0.2284351905540453, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.49392607486162804 + "value": 0.04137726328287326, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.23118547623512836 + "value": 0.9565152864351439, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2868180777521314 + "value": 0.735743791431681, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.28340954762838766 + "value": 0.749557237558128, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.23118547623512836 + "value": 0.9565152864351439, + "path": "" } ] }, - "strict_f1": { - "max": 0.026476637409113293, - "mean": 0.007789958613164452, - "min": 0.00275028568108307, - "n": 5, - "std": 0.01045266903938076 - }, "thresholds": [ 0.00026914896443486214, 0.00028129236306995153, @@ -7897,192 +7912,192 @@ 0.000282647815765813, 0.0002847809228114784 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "aurora", + "label": "Aurora", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0066546155176738, + "std": 0.011042800084784003, + "min": 0.001471035812113219, + "max": 0.026403975040443728 + }, "tolerance_f1": { - "max": 0.24290193233931348, - "mean": 0.12738113192754766, - "min": 0.08722255267001126, "n": 5, - "std": 0.06555769355379562 + "mean": 0.1053036158902898, + "std": 0.07430895639255175, + "min": 0.06246138809231819, + "max": 0.23725772191057362 }, "union_f1": { - "max": 0.49392607486162804, - "mean": 0.30530493054248076, - "min": 0.23118547623512836, "n": 5, - "std": 0.10884219611087281 - } - }, - { + "mean": 0.24944433080431602, + "std": 0.12584449940748835, + "min": 0.17013954201238674, + "max": 0.4711352810381827 + }, "difference": { - "max": 0.44473130599773897, + "n": 5, "mean": 0.24278971528664223, + "std": 0.11494327446365726, "min": 0.16866850620027352, - "n": 5, - "std": 0.11494327446365726 + "max": 0.44473130599773897 }, - "label": "Aurora", - "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9746196660482375, + "n": 5, "mean": 0.6728910327767471, + "std": 0.38399138452827886, "min": 0.02069016697588126, - "n": 5, - "std": 0.38399138452827886 + "max": 0.9746196660482375 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.44473130599773897 + "value": 0.026403975040443728, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.17276769792528876 + "value": 0.0014870516331508276, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.21405981286832887 + "value": 0.00196209796760366, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2137212534415811 + "value": 0.0019489171350575599, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16866850620027352 + "value": 0.001471035812113219, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.02069016697588126 + "value": 0.23725772191057362, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9366672850958565 + "value": 0.06425881062262738, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7109764996907854 + "value": 0.08141778117479781, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7215015460729747 + "value": 0.08112237765113199, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9746196660482375 + "value": 0.06246138809231819, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.026403975040443728 + "value": 0.4711352810381827, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0014870516331508276 + "value": 0.1742547495584396, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.00196209796760366 + "value": 0.21602191083593253, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0019489171350575599 + "value": 0.21567017057663868, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.001471035812113219 + "value": 0.17013954201238674, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.23725772191057362 + "value": 0.44473130599773897, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.06425881062262738 + "value": 0.17276769792528876, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.08141778117479781 + "value": 0.21405981286832887, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.08112237765113199 + "value": 0.2137212534415811, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.06246138809231819 + "value": 0.16866850620027352, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4711352810381827 + "value": 0.02069016697588126, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1742547495584396 + "value": 0.9366672850958565, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.21602191083593253 + "value": 0.7109764996907854, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.21567017057663868 + "value": 0.7215015460729747, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.17013954201238674 + "value": 0.9746196660482375, + "path": "" } ] }, - "strict_f1": { - "max": 0.026403975040443728, - "mean": 0.0066546155176738, - "min": 0.001471035812113219, - "n": 5, - "std": 0.011042800084784003 - }, "thresholds": [ 0.0002, 0.0002701335761230439, @@ -8090,192 +8105,192 @@ 0.0002826936251949519, 0.0002847809228114784 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "climax", + "label": "ClimaX", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.00347964131784203, + "std": 0.0007542402263439388, + "min": 0.0027501821392557145, + "max": 0.004637761701086003 + }, "tolerance_f1": { - "max": 0.23725772191057362, - "mean": 0.1053036158902898, - "min": 0.06246138809231819, "n": 5, - "std": 0.07430895639255175 + "mean": 0.2975348750078338, + "std": 0.03607251580095264, + "min": 0.24162386260287075, + "max": 0.33641526956922785 }, "union_f1": { - "max": 0.4711352810381827, - "mean": 0.24944433080431602, - "min": 0.17013954201238674, "n": 5, - "std": 0.12584449940748835 - } - }, - { + "mean": 0.6015057903864602, + "std": 0.07586493774699653, + "min": 0.4719733074426152, + "max": 0.6677488562714049 + }, "difference": { - "max": 0.664031416861268, + "n": 5, "mean": 0.5980261490686182, + "std": 0.07545417523567537, "min": 0.46922312530335947, - "n": 5, - "std": 0.07545417523567537 + "max": 0.664031416861268 }, - "label": "ClimaX", - "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.12421632653061225, + "n": 5, "mean": 0.056763042671614104, + "std": 0.03926077114304812, "min": 0.025392455163883734, - "n": 5, - "std": 0.03926077114304812 + "max": 0.12421632653061225 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6362769893231447 + "value": 0.003394935139973715, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.46922312530335947 + "value": 0.0027501821392557145, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.615867982755411 + "value": 0.004637761701086003, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.664031416861268 + "value": 0.003717439410136951, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.604731231099908 + "value": 0.002897888198757764, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.041071861471861475 + "value": 0.3112148878371163, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.025392455163883734 + "value": 0.24162386260287075, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.037147804576376 + "value": 0.31289270592627744, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.055986765615337046 + "value": 0.33641526956922785, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.12421632653061225 + "value": 0.2855276491036767, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.003394935139973715 + "value": 0.6396719244631184, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0027501821392557145 + "value": 0.4719733074426152, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.004637761701086003 + "value": 0.620505744456497, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.003717439410136951 + "value": 0.6677488562714049, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.002897888198757764 + "value": 0.6076291192986658, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3112148878371163 + "value": 0.6362769893231447, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.24162386260287075 + "value": 0.46922312530335947, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.31289270592627744 + "value": 0.615867982755411, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.33641526956922785 + "value": 0.664031416861268, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2855276491036767 + "value": 0.604731231099908, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6396719244631184 + "value": 0.041071861471861475, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.4719733074426152 + "value": 0.025392455163883734, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.620505744456497 + "value": 0.037147804576376, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.6677488562714049 + "value": 0.055986765615337046, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6076291192986658 + "value": 0.12421632653061225, + "path": "" } ] }, - "strict_f1": { - "max": 0.004637761701086003, - "mean": 0.00347964131784203, - "min": 0.0027501821392557145, - "n": 5, - "std": 0.0007542402263439388 - }, "thresholds": [ 0.0005393968409013734, 0.001021229882830197, @@ -8283,192 +8298,192 @@ 0.001442534703828549, 0.0014632086704014654 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "climax", + "label": "ClimaX", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.012936887506760619, + "std": 0.0010861342333245315, + "min": 0.011492789729228419, + "max": 0.014246556800108287 + }, "tolerance_f1": { - "max": 0.33641526956922785, - "mean": 0.2975348750078338, - "min": 0.24162386260287075, "n": 5, - "std": 0.03607251580095264 + "mean": 0.34579050632090236, + "std": 0.02377186314441771, + "min": 0.3186326966868806, + "max": 0.3665570668664193 }, "union_f1": { - "max": 0.6677488562714049, - "mean": 0.6015057903864602, - "min": 0.4719733074426152, "n": 5, - "std": 0.07586493774699653 - } - }, - { + "mean": 0.6921863051786654, + "std": 0.057214573939719605, + "min": 0.6170788209868917, + "max": 0.7408743604065289 + }, "difference": { - "max": 0.7272887839717904, + "n": 5, "mean": 0.6792494176719048, + "std": 0.05726305372578923, "min": 0.6028322641867834, - "n": 5, - "std": 0.05726305372578923 + "max": 0.7272887839717904 }, - "label": "ClimaX", - "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.3090951815573365, + "n": 5, "mean": 0.24009102602156923, + "std": 0.0928155581273325, "min": 0.13341248639556744, - "n": 5, - "std": 0.0928155581273325 + "max": 0.3090951815573365 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6341272298959009 + "value": 0.011492789729228419, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7267635176891648 + "value": 0.013106997858124742, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6028322641867834 + "value": 0.014246556800108287, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7272887839717904 + "value": 0.013585576434738445, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7052352926158846 + "value": 0.012252516711603196, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.13341248639556744 + "value": 0.3186326966868806, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3068887899475611 + "value": 0.3655334236435691, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.14358118135945386 + "value": 0.32161191611096424, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.3090951815573365 + "value": 0.3665570668664193, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.3074774908479272 + "value": 0.3566174282966785, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.011492789729228419 + "value": 0.6456200196251293, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.013106997858124742 + "value": 0.7398705155472896, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.014246556800108287 + "value": 0.6170788209868917, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.013585576434738445 + "value": 0.7408743604065289, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.012252516711603196 + "value": 0.7174878093274878, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3186326966868806 + "value": 0.6341272298959009, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3655334236435691 + "value": 0.7267635176891648, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.32161191611096424 + "value": 0.6028322641867834, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.3665570668664193 + "value": 0.7272887839717904, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.3566174282966785 + "value": 0.7052352926158846, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6456200196251293 + "value": 0.13341248639556744, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7398705155472896 + "value": 0.3068887899475611, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6170788209868917 + "value": 0.14358118135945386, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7408743604065289 + "value": 0.3090951815573365, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7174878093274878 + "value": 0.3074774908479272, + "path": "" } ] }, - "strict_f1": { - "max": 0.014246556800108287, - "mean": 0.012936887506760619, - "min": 0.011492789729228419, - "n": 5, - "std": 0.0010861342333245315 - }, "thresholds": [ 0.0005436268660297483, 0.0006166165592580956, @@ -8476,192 +8491,192 @@ 0.0014632082671483923, 0.0014882655838293205 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "climax", + "label": "ClimaX", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.012522340011631166, + "std": 0.0016018805893483327, + "min": 0.010293697838827224, + "max": 0.014244146704560834 + }, "tolerance_f1": { - "max": 0.3665570668664193, - "mean": 0.34579050632090236, - "min": 0.3186326966868806, "n": 5, - "std": 0.02377186314441771 + "mean": 0.34334086004373804, + "std": 0.022852052082273543, + "min": 0.3179193199949169, + "max": 0.36524448333846904 }, "union_f1": { - "max": 0.7408743604065289, - "mean": 0.6921863051786654, - "min": 0.6170788209868917, "n": 5, - "std": 0.057214573939719605 - } - }, - { + "mean": 0.6857131503190195, + "std": 0.05537732130173235, + "min": 0.6141812807681712, + "max": 0.7365490884872157 + }, "difference": { - "max": 0.7230514919507499, + "n": 5, "mean": 0.6731908103073884, + "std": 0.05553767153675633, "min": 0.5999371340636104, - "n": 5, - "std": 0.05553767153675633 + "max": 0.7230514919507499 }, - "label": "ClimaX", - "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.15408504007123774, + "n": 5, "mean": 0.1184503314534481, + "std": 0.04506675919516373, "min": 0.06670624319778372, - "n": 5, - "std": 0.04506675919516373 + "max": 0.15408504007123774 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6312315933404747 + "value": 0.011490700169087834, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7230353684802844 + "value": 0.01308555880921415, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.5999371340636104 + "value": 0.014244146704560834, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7230514919507499 + "value": 0.013497596536465794, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6886984637018222 + "value": 0.010293697838827224, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.06670624319778372 + "value": 0.3179193199949169, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.15368556446027506 + "value": 0.36399388919405107, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.07179059067972693 + "value": 0.3208182317552862, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.15408504007123774 + "value": 0.36524448333846904, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.14598421885821708 + "value": 0.34872837593596695, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.011490700169087834 + "value": 0.6427222935095626, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.01308555880921415 + "value": 0.7361209272894985, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.014244146704560834 + "value": 0.6141812807681712, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.013497596536465794 + "value": 0.7365490884872157, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.010293697838827224 + "value": 0.6989921615406495, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3179193199949169 + "value": 0.6312315933404747, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.36399388919405107 + "value": 0.7230353684802844, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3208182317552862 + "value": 0.5999371340636104, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.36524448333846904 + "value": 0.7230514919507499, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.34872837593596695 + "value": 0.6886984637018222, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6427222935095626 + "value": 0.06670624319778372, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7361209272894985 + "value": 0.15368556446027506, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6141812807681712 + "value": 0.07179059067972693, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7365490884872157 + "value": 0.15408504007123774, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6989921615406495 + "value": 0.14598421885821708, + "path": "" } ] }, - "strict_f1": { - "max": 0.014244146704560834, - "mean": 0.012522340011631166, - "min": 0.010293697838827224, - "n": 5, - "std": 0.0016018805893483327 - }, "thresholds": [ 0.0006294738670532621, 0.0006776005131691771, @@ -8669,192 +8684,192 @@ 0.0014632082671483923, 0.0014882655838293205 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "climax", + "label": "ClimaX", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.010287276360038863, + "std": 0.002686481400804917, + "min": 0.006734646906365392, + "max": 0.013757803485310216 + }, "tolerance_f1": { - "max": 0.36524448333846904, - "mean": 0.34334086004373804, - "min": 0.3179193199949169, "n": 5, - "std": 0.022852052082273543 + "mean": 0.3021399507767809, + "std": 0.042856591284064, + "min": 0.24066482674710968, + "max": 0.35193063179186285 }, "union_f1": { - "max": 0.7365490884872157, - "mean": 0.6857131503190195, - "min": 0.6141812807681712, - "n": 5, - "std": 0.05537732130173235 - } - }, - { - "difference": { - "max": 0.6825790409197952, - "mean": 0.5903626491479447, - "min": 0.47288261881572224, "n": 5, - "std": 0.07589123629900595 + "mean": 0.6006499255079836, + "std": 0.07567424048289119, + "min": 0.48364281191756603, + "max": 0.691273953290052 + }, + "difference": { + "n": 5, + "mean": 0.5903626491479447, + "std": 0.07589123629900595, + "min": 0.47288261881572224, + "max": 0.6825790409197952 }, - "label": "ClimaX", - "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.12288311688311689, + "n": 5, "mean": 0.057441929499072356, + "std": 0.04134148942976998, "min": 0.020547928262213976, - "n": 5, - "std": 0.04134148942976998 + "max": 0.12288311688311689 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5984324485861181 + "value": 0.011488825936418005, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.47288261881572224 + "value": 0.010760193101843772, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6150773048223223 + "value": 0.013757803485310216, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.6825790409197952 + "value": 0.008694912370256934, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5828418325957657 + "value": 0.006734646906365392, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.030718614718614718 + "value": 0.30575189275627285, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.020547928262213976 + "value": 0.24066482674710968, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.041087198515769946 + "value": 0.32901058451573606, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.07197278911564625 + "value": 0.35193063179186285, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.12288311688311689 + "value": 0.2833418180729231, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.011488825936418005 + "value": 0.6099212745225361, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.010760193101843772 + "value": 0.48364281191756603, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.013757803485310216 + "value": 0.6288351083076325, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.008694912370256934 + "value": 0.691273953290052, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.006734646906365392 + "value": 0.5895764795021311, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.30575189275627285 + "value": 0.5984324485861181, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.24066482674710968 + "value": 0.47288261881572224, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.32901058451573606 + "value": 0.6150773048223223, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.35193063179186285 + "value": 0.6825790409197952, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2833418180729231 + "value": 0.5828418325957657, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6099212745225361 + "value": 0.030718614718614718, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.48364281191756603 + "value": 0.020547928262213976, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6288351083076325 + "value": 0.041087198515769946, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.691273953290052 + "value": 0.07197278911564625, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5895764795021311 + "value": 0.12288311688311689, + "path": "" } ] }, - "strict_f1": { - "max": 0.013757803485310216, - "mean": 0.010287276360038863, - "min": 0.006734646906365392, - "n": 5, - "std": 0.002686481400804917 - }, "thresholds": [ 0.0006820998141378855, 0.0009633156281329466, @@ -8862,3244 +8877,3244 @@ 0.0013755693906898965, 0.0015083511573617753 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "dlwp", + "label": "DLWP", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.001692736472985844, + "std": 0.0004188778823216006, + "min": 0.001008153911497155, + "max": 0.002078902199880332 + }, "tolerance_f1": { - "max": 0.35193063179186285, - "mean": 0.3021399507767809, - "min": 0.24066482674710968, "n": 5, - "std": 0.042856591284064 + "mean": 0.14914807478919873, + "std": 0.032446265301716536, + "min": 0.09460750931776095, + "max": 0.17299721297318826 }, "union_f1": { - "max": 0.691273953290052, - "mean": 0.6006499255079836, - "min": 0.48364281191756603, "n": 5, - "std": 0.07567424048289119 - } - }, - { + "mean": 0.281900673424129, + "std": 0.06965835259895045, + "min": 0.17657638185878205, + "max": 0.36723610814010926 + }, "difference": { - "max": 0.36515720594022893, + "n": 5, "mean": 0.2802079369511431, + "std": 0.06925745403651407, "min": 0.17556822794728488, - "n": 5, - "std": 0.06925745403651407 + "max": 0.36515720594022893 }, - "label": "DLWP", - "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.0589282622139765, + "n": 5, "mean": 0.04322077922077922, + "std": 0.015619298009606992, "min": 0.02625751391465677, - "n": 5, - "std": 0.015619298009606992 + "max": 0.0589282622139765 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.26339192421292845 + "value": 0.0016304484151205278, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3044358568583716 + "value": 0.001786130143937306, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.2924864697969017 + "value": 0.0019600476944939, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.17556822794728488 + "value": 0.001008153911497155, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.36515720594022893 + "value": 0.002078902199880332, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0589282622139765 + "value": 0.14442504878529014, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.057648732220160794 + "value": 0.16383974771641738, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.04519196042053185 + "value": 0.17299721297318826, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.02625751391465677 + "value": 0.09460750931776095, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.028077427334570193 + "value": 0.16987085515333694, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0016304484151205278 + "value": 0.265022372628049, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.001786130143937306 + "value": 0.3062219870023089, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0019600476944939 + "value": 0.2944465174913956, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.001008153911497155 + "value": 0.17657638185878205, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.002078902199880332 + "value": 0.36723610814010926, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.14442504878529014 + "value": 0.26339192421292845, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.16383974771641738 + "value": 0.3044358568583716, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.17299721297318826 + "value": 0.2924864697969017, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.09460750931776095 + "value": 0.17556822794728488, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.16987085515333694 + "value": 0.36515720594022893, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.265022372628049 + "value": 0.0589282622139765, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3062219870023089 + "value": 0.057648732220160794, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.2944465174913956 + "value": 0.04519196042053185, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.17657638185878205 + "value": 0.02625751391465677, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.36723610814010926 + "value": 0.028077427334570193, + "path": "" } ] }, - "strict_f1": { - "max": 0.002078902199880332, - "mean": 0.001692736472985844, - "min": 0.001008153911497155, - "n": 5, - "std": 0.0004188778823216006 - }, "thresholds": [ 0.08, 0.1 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "dlwp", + "label": "DLWP", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.01805388290899331, + "std": 0.004835410031272547, + "min": 0.011684287006420426, + "max": 0.024960845732184808 + }, "tolerance_f1": { - "max": 0.17299721297318826, - "mean": 0.14914807478919873, - "min": 0.09460750931776095, "n": 5, - "std": 0.032446265301716536 + "mean": 0.31723135564296057, + "std": 0.032922909485364145, + "min": 0.28520211660517514, + "max": 0.35323298220195054 }, "union_f1": { - "max": 0.36723610814010926, - "mean": 0.281900673424129, - "min": 0.17657638185878205, "n": 5, - "std": 0.06965835259895045 - } - }, - { + "mean": 0.5545957535300212, + "std": 0.05291976314318387, + "min": 0.47313209204823786, + "max": 0.6023409019864735 + }, "difference": { - "max": 0.5906566149800531, + "n": 5, "mean": 0.5365418706210278, + "std": 0.0547523973572839, "min": 0.45626799482364494, - "n": 5, - "std": 0.0547523973572839 + "max": 0.5906566149800531 }, - "label": "DLWP", - "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.16420797467102008, + "n": 5, "mean": 0.09400069258929455, + "std": 0.05080713313175061, "min": 0.04792965271593945, - "n": 5, - "std": 0.05080713313175061 + "max": 0.16420797467102008 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.45626799482364494 + "value": 0.016864097224592905, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.5668021940668209 + "value": 0.01972516815912186, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.5053802602935323 + "value": 0.024960845732184808, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.5636022889410879 + "value": 0.017035016422646553, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5906566149800531 + "value": 0.011684287006420426, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.06984763035519936 + "value": 0.28520211660517514, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.05733155238943306 + "value": 0.3500819147736832, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.04792965271593945 + "value": 0.35323298220195054, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.1306866528148808 + "value": 0.3101459291213085, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.16420797467102008 + "value": 0.2874938355126855, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.016864097224592905 + "value": 0.47313209204823786, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.01972516815912186 + "value": 0.5865273622259427, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.024960845732184808 + "value": 0.5303411060257172, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.017035016422646553 + "value": 0.5806373053637345, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.011684287006420426 + "value": 0.6023409019864735, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.28520211660517514 + "value": 0.45626799482364494, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3500819147736832 + "value": 0.5668021940668209, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.35323298220195054 + "value": 0.5053802602935323, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3101459291213085 + "value": 0.5636022889410879, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.2874938355126855 + "value": 0.5906566149800531, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.47313209204823786 + "value": 0.06984763035519936, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.5865273622259427 + "value": 0.05733155238943306, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.5303411060257172 + "value": 0.04792965271593945, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.5806373053637345 + "value": 0.1306866528148808, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.6023409019864735 + "value": 0.16420797467102008, + "path": "" } ] }, - "strict_f1": { - "max": 0.024960845732184808, - "mean": 0.01805388290899331, - "min": 0.011684287006420426, - "n": 5, - "std": 0.004835410031272547 - }, "thresholds": [ 0.05, 0.08, 0.1 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "dlwp", + "label": "DLWP", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.016110218166136593, + "std": 0.005998581011452884, + "min": 0.008571862589787548, + "max": 0.023191305534082125 + }, "tolerance_f1": { - "max": 0.35323298220195054, - "mean": 0.31723135564296057, - "min": 0.28520211660517514, "n": 5, - "std": 0.032922909485364145 + "mean": 0.27658069580542066, + "std": 0.05921597516408298, + "min": 0.2160234001333052, + "max": 0.3409571337826908 }, "union_f1": { - "max": 0.6023409019864735, - "mean": 0.5545957535300212, - "min": 0.47313209204823786, "n": 5, - "std": 0.05291976314318387 - } - }, - { + "mean": 0.4712685600792209, + "std": 0.08011144927032286, + "min": 0.3704561794666731, + "max": 0.57270947387435 + }, "difference": { - "max": 0.5537110636994692, + "n": 5, "mean": 0.45515834191308435, + "std": 0.07792651255459898, "min": 0.3591438068453628, - "n": 5, - "std": 0.07792651255459898 + "max": 0.5537110636994692 }, - "label": "DLWP", - "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.11266572672405263, + "n": 5, "mean": 0.049700455130107846, + "std": 0.03684905017300331, "min": 0.025880577817354308, - "n": 5, - "std": 0.03684905017300331 + "max": 0.11266572672405263 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3591438068453628 + "value": 0.011312372621310312, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.5537110636994692 + "value": 0.018998410174880763, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.486215877804802 + "value": 0.023191305534082125, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3946007462885974 + "value": 0.018477139910622206, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.48212021492719026 + "value": 0.008571862589787548, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.052691204115959234 + "value": 0.2160234001333052, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.02979989116453943 + "value": 0.3409571337826908, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.025880577817354308 + "value": 0.33689092573275314, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.02746487582863362 + "value": 0.2606560686639405, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.11266572672405263 + "value": 0.22837595071441363, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.011312372621310312 + "value": 0.3704561794666731, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.018998410174880763 + "value": 0.57270947387435, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.023191305534082125 + "value": 0.5094071833388841, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.018477139910622206 + "value": 0.41307788619921965, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.008571862589787548 + "value": 0.4906920775169778, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2160234001333052 + "value": 0.3591438068453628, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3409571337826908 + "value": 0.5537110636994692, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.33689092573275314 + "value": 0.486215877804802, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2606560686639405 + "value": 0.3946007462885974, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.22837595071441363 + "value": 0.48212021492719026, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3704561794666731 + "value": 0.052691204115959234, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.57270947387435 + "value": 0.02979989116453943, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.5094071833388841 + "value": 0.025880577817354308, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.41307788619921965 + "value": 0.02746487582863362, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4906920775169778 + "value": 0.11266572672405263, + "path": "" } ] }, - "strict_f1": { - "max": 0.023191305534082125, - "mean": 0.016110218166136593, - "min": 0.008571862589787548, - "n": 5, - "std": 0.005998581011452884 - }, "thresholds": [ 0.05, 0.08, 0.1 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "dlwp", + "label": "DLWP", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.015247655927156777, + "std": 0.008986797239635393, + "min": 0.007379212786746584, + "max": 0.028949691085613415 + }, "tolerance_f1": { - "max": 0.3409571337826908, - "mean": 0.27658069580542066, - "min": 0.2160234001333052, "n": 5, - "std": 0.05921597516408298 + "mean": 0.2094029351195231, + "std": 0.047970894595050655, + "min": 0.1600767886426284, + "max": 0.28953047944835925 }, "union_f1": { - "max": 0.57270947387435, - "mean": 0.4712685600792209, - "min": 0.3704561794666731, "n": 5, - "std": 0.08011144927032286 - } - }, - { + "mean": 0.3493011789060551, + "std": 0.0784709395452888, + "min": 0.2769459546244767, + "max": 0.4666657840911348 + }, "difference": { - "max": 0.44734760227295295, + "n": 5, "mean": 0.33405352297889834, + "std": 0.0787597726808095, "min": 0.26320327785336994, - "n": 5, - "std": 0.0787597726808095 + "max": 0.44734760227295295 }, - "label": "DLWP", - "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.04168893011750155, + "n": 5, "mean": 0.019198021026592453, + "std": 0.014678247188046134, "min": 0.0027860235003092145, - "n": 5, - "std": 0.014678247188046134 + "max": 0.04168893011750155 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2695667418377301 + "value": 0.007379212786746584, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.44734760227295295 + "value": 0.01931818181818182, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.26320327785336994 + "value": 0.028949691085613415, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.30932859219900066 + "value": 0.01226752201682564, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.38082140073143794 + "value": 0.008323671928416423, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.04168893011750155 + "value": 0.1600767886426284, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.010166975881261595 + "value": 0.28953047944835925, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0027860235003092145 + "value": 0.20397124965652888, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.021262213976499692 + "value": 0.19661692001462533, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.02008596165739023 + "value": 0.19681923783547378, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.007379212786746584 + "value": 0.2769459546244767, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.01931818181818182 + "value": 0.4666657840911348, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.028949691085613415 + "value": 0.29215296893898335, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.01226752201682564 + "value": 0.3215961142158263, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.008323671928416423 + "value": 0.3891450726598544, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1600767886426284 + "value": 0.2695667418377301, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.28953047944835925 + "value": 0.44734760227295295, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.20397124965652888 + "value": 0.26320327785336994, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.19661692001462533 + "value": 0.30932859219900066, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.19681923783547378 + "value": 0.38082140073143794, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2769459546244767 + "value": 0.04168893011750155, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.4666657840911348 + "value": 0.010166975881261595, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.29215296893898335 + "value": 0.0027860235003092145, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3215961142158263 + "value": 0.021262213976499692, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.3891450726598544 + "value": 0.02008596165739023, + "path": "" } ] }, - "strict_f1": { - "max": 0.028949691085613415, - "mean": 0.015247655927156777, - "min": 0.007379212786746584, - "n": 5, - "std": 0.008986797239635393 - }, "thresholds": [ 0.08, 0.1, 0.2 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fcn", + "label": "FCN", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.002829241015607336, + "std": 0.0008385182499278472, + "min": 0.002068681772914111, + "max": 0.00400312243549969 + }, "tolerance_f1": { - "max": 0.28953047944835925, - "mean": 0.2094029351195231, - "min": 0.1600767886426284, "n": 5, - "std": 0.047970894595050655 + "mean": 0.19506132405627552, + "std": 0.03341186919631261, + "min": 0.16083569292964747, + "max": 0.23211789308126832 }, "union_f1": { - "max": 0.4666657840911348, - "mean": 0.3493011789060551, - "min": 0.2769459546244767, - "n": 5, - "std": 0.0784709395452888 - } - }, - { - "difference": { - "max": 0.5017259192233794, - "mean": 0.39777501610341254, - "min": 0.2830040701913329, "n": 5, - "std": 0.09342259085449434 + "mean": 0.4006042571190199, + "std": 0.09370061309902981, + "min": 0.2863487755227932, + "max": 0.5057290416588791 + }, + "difference": { + "n": 5, + "mean": 0.39777501610341254, + "std": 0.09342259085449434, + "min": 0.2830040701913329, + "max": 0.5017259192233794 }, - "label": "FCN", - "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.03192430426716141, + "n": 5, "mean": 0.01520235003092146, + "std": 0.013446242178130542, "min": 0.0016064316635745208, - "n": 5, - "std": 0.013446242178130542 + "max": 0.03192430426716141 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.4822930267328773 + "value": 0.0026467855605157606, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3840989936961055 + "value": 0.002068681772914111, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.33775307067336774 + "value": 0.00208290997764682, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2830040701913329 + "value": 0.0033447053314602988, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5017259192233794 + "value": 0.00400312243549969, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.027048113790970933 + "value": 0.23211789308126832, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.03192430426716141 + "value": 0.19094441931639336, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.009495980210265924 + "value": 0.16485748333864078, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0016064316635745208 + "value": 0.16083569292964747, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.005936920222634508 + "value": 0.22655113161542761, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0026467855605157606 + "value": 0.48493981229339306, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.002068681772914111 + "value": 0.38616767546901964, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.00208290997764682 + "value": 0.33983598065101456, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0033447053314602988 + "value": 0.2863487755227932, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.00400312243549969 + "value": 0.5057290416588791, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.23211789308126832 + "value": 0.4822930267328773, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.19094441931639336 + "value": 0.3840989936961055, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.16485748333864078 + "value": 0.33775307067336774, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.16083569292964747 + "value": 0.2830040701913329, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.22655113161542761 + "value": 0.5017259192233794, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.48493981229339306 + "value": 0.027048113790970933, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.38616767546901964 + "value": 0.03192430426716141, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.33983598065101456 + "value": 0.009495980210265924, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2863487755227932 + "value": 0.0016064316635745208, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5057290416588791 + "value": 0.005936920222634508, + "path": "" } ] }, - "strict_f1": { - "max": 0.00400312243549969, - "mean": 0.002829241015607336, - "min": 0.002068681772914111, - "n": 5, - "std": 0.0008385182499278472 - }, "thresholds": [ 0.15, 0.2, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fcn", + "label": "FCN", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.016231351161484737, + "std": 0.00506423236804874, + "min": 0.012095158293562608, + "max": 0.024537870112874206 + }, "tolerance_f1": { - "max": 0.23211789308126832, - "mean": 0.19506132405627552, - "min": 0.16083569292964747, "n": 5, - "std": 0.03341186919631261 + "mean": 0.2937694425517297, + "std": 0.0276260699067859, + "min": 0.25572856051368525, + "max": 0.3296389749733903 }, "union_f1": { - "max": 0.5057290416588791, - "mean": 0.4006042571190199, - "min": 0.2863487755227932, "n": 5, - "std": 0.09370061309902981 - } - }, - { + "mean": 0.5430325295439387, + "std": 0.0740889720448565, + "min": 0.47261322844070225, + "max": 0.6562774730922674 + }, "difference": { - "max": 0.6396713821034217, + "n": 5, "mean": 0.5268011783824539, + "std": 0.0743888677938, "min": 0.46051807014713964, - "n": 5, - "std": 0.0743888677938 + "max": 0.6396713821034217 }, - "label": "FCN", - "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.07633570792520035, + "n": 5, "mean": 0.047855941426733946, + "std": 0.029409395848307603, "min": 0.01251113089937667, - "n": 5, - "std": 0.029409395848307603 + "max": 0.07633570792520035 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.5645980289087711 + "value": 0.01222529471692621, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.46051807014713964 + "value": 0.012095158293562608, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.48713449736377723 + "value": 0.024537870112874206, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.4820839133891601 + "value": 0.01569234169521494, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.6396713821034217 + "value": 0.016606090988845718, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.06537300880577818 + "value": 0.3028035236186743, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.06528643514395963 + "value": 0.25572856051368525, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.01251113089937667 + "value": 0.27999632336313063, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0197734243593549 + "value": 0.300679830289768, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.07633570792520035 + "value": 0.3296389749733903, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.01222529471692621 + "value": 0.5768233236256973, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.012095158293562608 + "value": 0.47261322844070225, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.024537870112874206 + "value": 0.5116723674766515, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.01569234169521494 + "value": 0.49777625508437506, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.016606090988845718 + "value": 0.6562774730922674, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3028035236186743 + "value": 0.5645980289087711, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.25572856051368525 + "value": 0.46051807014713964, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.27999632336313063 + "value": 0.48713449736377723, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.300679830289768 + "value": 0.4820839133891601, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.3296389749733903 + "value": 0.6396713821034217, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.5768233236256973 + "value": 0.06537300880577818, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.47261322844070225 + "value": 0.06528643514395963, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.5116723674766515 + "value": 0.01251113089937667, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.49777625508437506 + "value": 0.0197734243593549, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.6562774730922674 + "value": 0.07633570792520035, + "path": "" } ] }, - "strict_f1": { - "max": 0.024537870112874206, - "mean": 0.016231351161484737, - "min": 0.012095158293562608, - "n": 5, - "std": 0.00506423236804874 - }, "thresholds": [ 0.15, 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fcn", + "label": "FCN", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.011776711221769876, + "std": 0.005117894394726336, + "min": 0.006403642961329112, + "max": 0.019849146486701073 + }, "tolerance_f1": { - "max": 0.3296389749733903, - "mean": 0.2937694425517297, - "min": 0.25572856051368525, "n": 5, - "std": 0.0276260699067859 + "mean": 0.22421694517240046, + "std": 0.03980326313080014, + "min": 0.16176542623012913, + "max": 0.2724598040570218 }, "union_f1": { - "max": 0.6562774730922674, - "mean": 0.5430325295439387, - "min": 0.47261322844070225, "n": 5, - "std": 0.0740889720448565 - } - }, - { + "mean": 0.4345103901380763, + "std": 0.09251305768823673, + "min": 0.3108270369881256, + "max": 0.5655038878451817 + }, "difference": { - "max": 0.5525178187961205, + "n": 5, "mean": 0.4227336789163064, + "std": 0.0902513347968921, "min": 0.30442339402679647, - "n": 5, - "std": 0.0902513347968921 + "max": 0.5525178187961205 }, - "label": "FCN", - "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.049730384881765115, + "n": 5, "mean": 0.0272573463935886, + "std": 0.01635259738911405, "min": 0.00802908875037103, - "n": 5, - "std": 0.01635259738911405 + "max": 0.049730384881765115 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.4452328295737776 + "value": 0.008918001736248123, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.30442339402679647 + "value": 0.006403642961329112, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.4238672329193606 + "value": 0.019849146486701073, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.387627119265477 + "value": 0.010726695875509895, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5525178187961205 + "value": 0.012986069049061175, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.030024982685267636 + "value": 0.2270292295163525, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.03344711586029484 + "value": 0.16176542623012913, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.00802908875037103 + "value": 0.23420382391023928, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.015055159790244386 + "value": 0.2256264421482596, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.049730384881765115 + "value": 0.2724598040570218, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.008918001736248123 + "value": 0.4541508313100257, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.006403642961329112 + "value": 0.3108270369881256, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.019849146486701073 + "value": 0.4437163794060617, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.010726695875509895 + "value": 0.3983538151409869, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.012986069049061175 + "value": 0.5655038878451817, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2270292295163525 + "value": 0.4452328295737776, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.16176542623012913 + "value": 0.30442339402679647, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.23420382391023928 + "value": 0.4238672329193606, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2256264421482596 + "value": 0.387627119265477, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.2724598040570218 + "value": 0.5525178187961205, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.4541508313100257 + "value": 0.030024982685267636, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3108270369881256 + "value": 0.03344711586029484, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.4437163794060617 + "value": 0.00802908875037103, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3983538151409869 + "value": 0.015055159790244386, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5655038878451817 + "value": 0.049730384881765115, + "path": "" } ] }, - "strict_f1": { - "max": 0.019849146486701073, - "mean": 0.011776711221769876, - "min": 0.006403642961329112, - "n": 5, - "std": 0.005117894394726336 - }, "thresholds": [ 0.15, 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fcn", + "label": "FCN", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.009962019740560447, + "std": 0.004315216342666275, + "min": 0.004578192172408018, + "max": 0.016197783461210574 + }, "tolerance_f1": { - "max": 0.2724598040570218, - "mean": 0.22421694517240046, - "min": 0.16176542623012913, "n": 5, - "std": 0.03980326313080014 + "mean": 0.1697921962176252, + "std": 0.03937109759451842, + "min": 0.10554731176319206, + "max": 0.20219147661108805 }, "union_f1": { - "max": 0.5655038878451817, - "mean": 0.4345103901380763, - "min": 0.3108270369881256, "n": 5, - "std": 0.09251305768823673 - } - }, - { + "mean": 0.3408594825588104, + "std": 0.08261638767751367, + "min": 0.2167503101126067, + "max": 0.4415948106689657 + }, "difference": { - "max": 0.4298567373898511, + "n": 5, "mean": 0.3308974628182499, + "std": 0.07927520828147509, "min": 0.2121721179401987, - "n": 5, - "std": 0.07927520828147509 + "max": 0.4298567373898511 }, - "label": "FCN", - "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.01035930735930736, + "n": 5, "mean": 0.008367841682127397, + "std": 0.0023583622377585994, "min": 0.005085961657390229, - "n": 5, - "std": 0.0023583622377585994 + "max": 0.01035930735930736 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3226698082083355 + "value": 0.008704984497972811, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2121721179401987 + "value": 0.004578192172408018, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3647698181559645 + "value": 0.016197783461210574, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3250188323968997 + "value": 0.00859106529209622, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4298567373898511 + "value": 0.011738073279114613, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.009654916512059368 + "value": 0.1596269732904185, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.01035930735930736 + "value": 0.10554731176319206, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.005085961657390229 + "value": 0.19512358967789506, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.010080395794681509 + "value": 0.18647162974553239, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.006658627087198516 + "value": 0.20219147661108805, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.008704984497972811 + "value": 0.3313747927063083, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.004578192172408018 + "value": 0.2167503101126067, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.016197783461210574 + "value": 0.3809676016171751, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.00859106529209622 + "value": 0.33360989768899596, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.011738073279114613 + "value": 0.4415948106689657, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1596269732904185 + "value": 0.3226698082083355, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.10554731176319206 + "value": 0.2121721179401987, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.19512358967789506 + "value": 0.3647698181559645, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.18647162974553239 + "value": 0.3250188323968997, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.20219147661108805 + "value": 0.4298567373898511, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.3313747927063083 + "value": 0.009654916512059368, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2167503101126067 + "value": 0.01035930735930736, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3809676016171751 + "value": 0.005085961657390229, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.33360989768899596 + "value": 0.010080395794681509, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4415948106689657 + "value": 0.006658627087198516, + "path": "" } ] }, - "strict_f1": { - "max": 0.016197783461210574, - "mean": 0.009962019740560447, - "min": 0.004578192172408018, - "n": 5, - "std": 0.004315216342666275 - }, "thresholds": [ 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fengwu", + "label": "FengWu", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0026125750208171442, + "std": 0.0007567108180601367, + "min": 0.0018414421399496025, + "max": 0.003438698584843275 + }, "tolerance_f1": { - "max": 0.20219147661108805, - "mean": 0.1697921962176252, - "min": 0.10554731176319206, "n": 5, - "std": 0.03937109759451842 + "mean": 0.12004984556809643, + "std": 0.06023907872870804, + "min": 0.042906126190853267, + "max": 0.20002472134774485 }, "union_f1": { - "max": 0.4415948106689657, - "mean": 0.3408594825588104, - "min": 0.2167503101126067, "n": 5, - "std": 0.08261638767751367 - } - }, - { + "mean": 0.24102217940052184, + "std": 0.13629274822774434, + "min": 0.06919785857013828, + "max": 0.4405198598043062 + }, "difference": { - "max": 0.43726313068468925, + "n": 5, "mean": 0.23840960437970468, + "std": 0.13573634707074209, "min": 0.06735555938074993, - "n": 5, - "std": 0.13573634707074209 + "max": 0.43726313068468925 }, - "label": "FengWu", - "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.012518243661100804, + "n": 5, "mean": 0.004276685219542362, + "std": 0.004829522662220775, "min": 0.00042881880024737165, - "n": 5, - "std": 0.004829522662220775 + "max": 0.012518243661100804 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.18397565646547948 + "value": 0.00268370607028754, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2798111779709097 + "value": 0.003438698584843275, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.2236424973966951 + "value": 0.0018414421399496025, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.06735555938074993 + "value": 0.001842299189388357, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.43726313068468925 + "value": 0.0032567291196169464, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0016931354359925788 + "value": 0.09710793321083327, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.004433395176252319 + "value": 0.1569315132056949, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0023098330241187385 + "value": 0.10327893388535589, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.00042881880024737165 + "value": 0.042906126190853267, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.012518243661100804 + "value": 0.20002472134774485, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00268370607028754 + "value": 0.186659362535767, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.003438698584843275 + "value": 0.283249876555753, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0018414421399496025 + "value": 0.2254839395366447, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.001842299189388357 + "value": 0.06919785857013828, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0032567291196169464 + "value": 0.4405198598043062, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.09710793321083327 + "value": 0.18397565646547948, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.1569315132056949 + "value": 0.2798111779709097, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.10327893388535589 + "value": 0.2236424973966951, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.042906126190853267 + "value": 0.06735555938074993, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.20002472134774485 + "value": 0.43726313068468925, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.186659362535767 + "value": 0.0016931354359925788, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.283249876555753 + "value": 0.004433395176252319, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.2254839395366447 + "value": 0.0023098330241187385, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.06919785857013828 + "value": 0.00042881880024737165, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4405198598043062 + "value": 0.012518243661100804, + "path": "" } ] }, - "strict_f1": { - "max": 0.003438698584843275, - "mean": 0.0026125750208171442, - "min": 0.0018414421399496025, - "n": 5, - "std": 0.0007567108180601367 - }, "thresholds": [ 0.2, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fengwu", + "label": "FengWu", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.015694746589097253, + "std": 0.003591671570543508, + "min": 0.01267427122940431, + "max": 0.0205686630369026 + }, "tolerance_f1": { - "max": 0.20002472134774485, - "mean": 0.12004984556809643, - "min": 0.042906126190853267, "n": 5, - "std": 0.06023907872870804 + "mean": 0.16276279599467128, + "std": 0.03702437856278332, + "min": 0.12305221580797777, + "max": 0.21207047330588913 }, "union_f1": { - "max": 0.4405198598043062, - "mean": 0.24102217940052184, - "min": 0.06919785857013828, "n": 5, - "std": 0.13629274822774434 - } - }, - { + "mean": 0.3010546945713337, + "std": 0.05010267654001306, + "min": 0.22237658124637444, + "max": 0.3374691422247865 + }, "difference": { - "max": 0.31898912374476807, + "n": 5, "mean": 0.28535994798223646, + "std": 0.04769577870036795, "min": 0.20932175095916816, - "n": 5, - "std": 0.04769577870036795 + "max": 0.31898912374476807 }, - "label": "FengWu", - "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.010032650638171565, + "n": 5, "mean": 0.00600375977045612, + "std": 0.0030413161402819498, "min": 0.0030746017611556348, - "n": 5, - "std": 0.0030413161402819498 + "max": 0.010032650638171565 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.20932175095916816 + "value": 0.013054830287206266, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.31898912374476807 + "value": 0.01848001848001848, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.26699212025153246 + "value": 0.01267427122940431, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3168732218355133 + "value": 0.0205686630369026, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.3146235231202003 + "value": 0.013695949911954608, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0030746017611556348 + "value": 0.12305221580797777, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.008098347679825863 + "value": 0.21207047330588913, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0032452755515979025 + "value": 0.13288915578187116, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.005567923221529633 + "value": 0.1866312847963942, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.010032650638171565 + "value": 0.15917085028122405, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.013054830287206266 + "value": 0.22237658124637444, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.01848001848001848 + "value": 0.3374691422247865, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.01267427122940431 + "value": 0.27966639148093675, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0205686630369026 + "value": 0.3374418848724159, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.013695949911954608 + "value": 0.3283194730321549, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.12305221580797777 + "value": 0.20932175095916816, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.21207047330588913 + "value": 0.31898912374476807, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.13288915578187116 + "value": 0.26699212025153246, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.1866312847963942 + "value": 0.3168732218355133, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.15917085028122405 + "value": 0.3146235231202003, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.22237658124637444 + "value": 0.0030746017611556348, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3374691422247865 + "value": 0.008098347679825863, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.27966639148093675 + "value": 0.0032452755515979025, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3374418848724159 + "value": 0.005567923221529633, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.3283194730321549 + "value": 0.010032650638171565, + "path": "" } ] }, - "strict_f1": { - "max": 0.0205686630369026, - "mean": 0.015694746589097253, - "min": 0.01267427122940431, - "n": 5, - "std": 0.003591671570543508 - }, "thresholds": [ 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fengwu", + "label": "FengWu", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.012426852806462965, + "std": 0.005333227666358275, + "min": 0.005460750853242321, + "max": 0.02015411973918198 + }, "tolerance_f1": { - "max": 0.21207047330588913, - "mean": 0.16276279599467128, - "min": 0.12305221580797777, "n": 5, - "std": 0.03702437856278332 + "mean": 0.12950332830320144, + "std": 0.05605188948381181, + "min": 0.04684227229450968, + "max": 0.1840127014362662 }, "union_f1": { - "max": 0.3374691422247865, - "mean": 0.3010546945713337, - "min": 0.22237658124637444, "n": 5, - "std": 0.05010267654001306 - } - }, - { + "mean": 0.24185379631697046, + "std": 0.0868538356837431, + "min": 0.10984387215996587, + "max": 0.33432003556596124 + }, "difference": { - "max": 0.3141659158267793, + "n": 5, "mean": 0.2294269435105075, + "std": 0.0818631441187217, "min": 0.10438312130672354, - "n": 5, - "std": 0.0818631441187217 + "max": 0.3141659158267793 }, - "label": "FengWu", - "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.005720045513010785, + "n": 5, "mean": 0.002608835460571881, + "std": 0.001935467532779746, "min": 0.0004947066389630949, - "n": 5, - "std": 0.001935467532779746 + "max": 0.005720045513010785 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.19895488431057093 + "value": 0.011185682326621925, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.28149494747245746 + "value": 0.014059753954305799, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.24813584863600618 + "value": 0.011273957158962795, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3141659158267793 + "value": 0.02015411973918198, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.10438312130672354 + "value": 0.005460750853242321, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0019998515880083113 + "value": 0.11778287307219655, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.005720045513010785 + "value": 0.17969557482138315, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0019738794894627485 + "value": 0.1191832198916516, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0028556940734144652 + "value": 0.1840127014362662, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0004947066389630949 + "value": 0.04684227229450968, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.011185682326621925 + "value": 0.21014056663719285, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.014059753954305799 + "value": 0.29555470142676327, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.011273957158962795 + "value": 0.259409805794969, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.02015411973918198 + "value": 0.33432003556596124, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.005460750853242321 + "value": 0.10984387215996587, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.11778287307219655 + "value": 0.19895488431057093, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.17969557482138315 + "value": 0.28149494747245746, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.1191832198916516 + "value": 0.24813584863600618, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.1840127014362662 + "value": 0.3141659158267793, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.04684227229450968 + "value": 0.10438312130672354, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.21014056663719285 + "value": 0.0019998515880083113, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.29555470142676327 + "value": 0.005720045513010785, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.259409805794969 + "value": 0.0019738794894627485, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.33432003556596124 + "value": 0.0028556940734144652, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.10984387215996587 + "value": 0.0004947066389630949, + "path": "" } ] }, - "strict_f1": { - "max": 0.02015411973918198, - "mean": 0.012426852806462965, - "min": 0.005460750853242321, - "n": 5, - "std": 0.005333227666358275 - }, "thresholds": [ 0.2, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fengwu", + "label": "FengWu", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.01119211887721548, + "std": 0.005022919520207697, + "min": 0.004807692307692307, + "max": 0.018831348656881748 + }, "tolerance_f1": { - "max": 0.1840127014362662, - "mean": 0.12950332830320144, - "min": 0.04684227229450968, "n": 5, - "std": 0.05605188948381181 + "mean": 0.11950764890450612, + "std": 0.05074475609731675, + "min": 0.04393081279657125, + "max": 0.17388386719173746 }, "union_f1": { - "max": 0.33432003556596124, - "mean": 0.24185379631697046, - "min": 0.10984387215996587, "n": 5, - "std": 0.0868538356837431 - } - }, - { + "mean": 0.22785982657353704, + "std": 0.07911462942092182, + "min": 0.1067457375833951, + "max": 0.31802311631118385 + }, "difference": { - "max": 0.2991917676543021, + "n": 5, "mean": 0.21666770769632157, + "std": 0.07443767874640678, "min": 0.1019380452757028, - "n": 5, - "std": 0.07443767874640678 + "max": 0.2991917676543021 }, - "label": "FengWu", - "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.0035126777983920843, + "n": 5, "mean": 0.0015006802721088436, + "std": 0.0012055357942519695, "min": 0.0003116883116883117, - "n": 5, - "std": 0.0012055357942519695 + "max": 0.0035126777983920843 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.19412591498868878 + "value": 0.010309278350515464, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2536326799641239 + "value": 0.011695906432748539, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.2344501305987902 + "value": 0.010316368638239341, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2991917676543021 + "value": 0.018831348656881748, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.1019380452757028 + "value": 0.004807692307692307, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0010822510822510823 + "value": 0.11130870584431331, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0035126777983920843 + "value": 0.1579992352547558, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0010810142238713667 + "value": 0.11041562343515275, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0015157699443413728 + "value": 0.17388386719173746, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0003116883116883117 + "value": 0.04393081279657125, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.010309278350515464 + "value": 0.20443519333920424, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.011695906432748539 + "value": 0.26532858639687246, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.010316368638239341 + "value": 0.24476649923702953, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.018831348656881748 + "value": 0.31802311631118385, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.004807692307692307 + "value": 0.1067457375833951, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.11130870584431331 + "value": 0.19412591498868878, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.1579992352547558 + "value": 0.2536326799641239, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.11041562343515275 + "value": 0.2344501305987902, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.17388386719173746 + "value": 0.2991917676543021, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.04393081279657125 + "value": 0.1019380452757028, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.20443519333920424 + "value": 0.0010822510822510823, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.26532858639687246 + "value": 0.0035126777983920843, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.24476649923702953 + "value": 0.0010810142238713667, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.31802311631118385 + "value": 0.0015157699443413728, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.1067457375833951 + "value": 0.0003116883116883117, + "path": "" } ] }, - "strict_f1": { - "max": 0.018831348656881748, - "mean": 0.01119211887721548, - "min": 0.004807692307692307, - "n": 5, - "std": 0.005022919520207697 - }, "thresholds": [ 0.2, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fuxi", + "label": "FuXi", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0037736840271503577, + "std": 0.0012118699857535593, + "min": 0.0029424127784783522, + "max": 0.005897939994871356 + }, "tolerance_f1": { - "max": 0.17388386719173746, - "mean": 0.11950764890450612, - "min": 0.04393081279657125, "n": 5, - "std": 0.05074475609731675 + "mean": 0.21032274703934553, + "std": 0.04821145875766877, + "min": 0.13539446199130012, + "max": 0.2682493391647262 }, "union_f1": { - "max": 0.31802311631118385, - "mean": 0.22785982657353704, - "min": 0.1067457375833951, "n": 5, - "std": 0.07911462942092182 - } - }, - { + "mean": 0.37288797408747854, + "std": 0.09446975287636859, + "min": 0.22039566333970034, + "max": 0.4541140407970205 + }, "difference": { - "max": 0.45054053460571325, + "n": 5, "mean": 0.36911429006032814, + "std": 0.09432711805206372, "min": 0.217453250561222, - "n": 5, - "std": 0.09432711805206372 + "max": 0.45054053460571325 }, - "label": "FuXi", - "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.008687322201607915, + "n": 5, "mean": 0.004504539270253556, + "std": 0.00277275768973151, "min": 0.001522943722943723, - "n": 5, - "std": 0.00277275768973151 + "max": 0.008687322201607915 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.44532955766011967 + "value": 0.0033573873422136956, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3742624367300788 + "value": 0.003097173828881146, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.217453250561222 + "value": 0.0029424127784783522, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.35798567074450705 + "value": 0.005897939994871356, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.45054053460571325 + "value": 0.003573506191307238, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.005430797773654916 + "value": 0.2682493391647262, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.004230179344465059 + "value": 0.20441214417386547, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.001522943722943723 + "value": 0.13539446199130012, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0026514533085961658 + "value": 0.22566599998043468, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.008687322201607915 + "value": 0.21789178988640112, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0033573873422136956 + "value": 0.44868694500233336, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.003097173828881146 + "value": 0.37735961055895995, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0029424127784783522 + "value": 0.22039566333970034, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.005897939994871356 + "value": 0.3638836107393784, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.003573506191307238 + "value": 0.4541140407970205, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2682493391647262 + "value": 0.44532955766011967, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.20441214417386547 + "value": 0.3742624367300788, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.13539446199130012 + "value": 0.217453250561222, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.22566599998043468 + "value": 0.35798567074450705, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.21789178988640112 + "value": 0.45054053460571325, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.44868694500233336 + "value": 0.005430797773654916, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.37735961055895995 + "value": 0.004230179344465059, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.22039566333970034 + "value": 0.001522943722943723, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3638836107393784 + "value": 0.0026514533085961658, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4541140407970205 + "value": 0.008687322201607915, + "path": "" } ] }, - "strict_f1": { - "max": 0.005897939994871356, - "mean": 0.0037736840271503577, - "min": 0.0029424127784783522, - "n": 5, - "std": 0.0012118699857535593 - }, "thresholds": [ 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fuxi", + "label": "FuXi", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.020306854641425946, + "std": 0.006799890593691795, + "min": 0.012024257632789628, + "max": 0.030370370370370364 + }, "tolerance_f1": { - "max": 0.2682493391647262, - "mean": 0.21032274703934553, - "min": 0.13539446199130012, "n": 5, - "std": 0.04821145875766877 + "mean": 0.31894420698818526, + "std": 0.047331205244476655, + "min": 0.26086649919553234, + "max": 0.38200179460101663 }, "union_f1": { - "max": 0.4541140407970205, - "mean": 0.37288797408747854, - "min": 0.22039566333970034, "n": 5, - "std": 0.09446975287636859 - } - }, - { + "mean": 0.5393080442029143, + "std": 0.08382153402586862, + "min": 0.4073509199876191, + "max": 0.6304826850687859 + }, "difference": { - "max": 0.6138219771674631, + "n": 5, "mean": 0.5190011895614883, + "std": 0.08687800582865718, "min": 0.3862603850081952, - "n": 5, - "std": 0.08687800582865718 + "max": 0.6138219771674631 }, - "label": "FuXi", - "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.0665751459384585, + "n": 5, "mean": 0.029315326011675073, + "std": 0.026391672443609994, "min": 0.004068962105471455, - "n": 5, - "std": 0.026391672443609994 + "max": 0.0665751459384585 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.6138219771674631 + "value": 0.016660707901322847, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.530575061359281 + "value": 0.02138840232322301, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3862603850081952 + "value": 0.02109053497942387, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.49281035455029043 + "value": 0.030370370370370364, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5715381697222122 + "value": 0.012024257632789628, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0665751459384585 + "value": 0.38200179460101663, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.024220837043633127 + "value": 0.30180702036802187, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.007007519540912239 + "value": 0.26086649919553234, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.004068962105471455 + "value": 0.3499638832422191, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.04470416542990007 + "value": 0.30008183753413636, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.016660707901322847 + "value": 0.6304826850687859, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.02138840232322301 + "value": 0.551963463682504, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.02109053497942387 + "value": 0.4073509199876191, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.030370370370370364 + "value": 0.5231807249206608, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.012024257632789628 + "value": 0.5835624273550017, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.38200179460101663 + "value": 0.6138219771674631, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.30180702036802187 + "value": 0.530575061359281, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.26086649919553234 + "value": 0.3862603850081952, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3499638832422191 + "value": 0.49281035455029043, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.30008183753413636 + "value": 0.5715381697222122, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.6304826850687859 + "value": 0.0665751459384585, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.551963463682504 + "value": 0.024220837043633127, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.4073509199876191 + "value": 0.007007519540912239, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.5231807249206608 + "value": 0.004068962105471455, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5835624273550017 + "value": 0.04470416542990007, + "path": "" } ] }, - "strict_f1": { - "max": 0.030370370370370364, - "mean": 0.020306854641425946, - "min": 0.012024257632789628, - "n": 5, - "std": 0.006799890593691795 - }, "thresholds": [ 0.1, 0.15, 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fuxi", + "label": "FuXi", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.016542308314081426, + "std": 0.007315696478545316, + "min": 0.008170871559633027, + "max": 0.026973684210526316 + }, "tolerance_f1": { - "max": 0.38200179460101663, - "mean": 0.31894420698818526, - "min": 0.26086649919553234, "n": 5, - "std": 0.047331205244476655 + "mean": 0.24012808383234296, + "std": 0.05778381935942338, + "min": 0.1656142112861716, + "max": 0.3231296638237823 }, "union_f1": { - "max": 0.6304826850687859, - "mean": 0.5393080442029143, - "min": 0.4073509199876191, "n": 5, - "std": 0.08382153402586862 - } - }, - { + "mean": 0.4021395209358275, + "std": 0.09930723729972839, + "min": 0.24700654033204764, + "max": 0.48397755354947186 + }, "difference": { - "max": 0.45700386933894555, + "n": 5, "mean": 0.38559721262174607, + "std": 0.09774430533636969, "min": 0.23030507895417707, - "n": 5, - "std": 0.09774430533636969 + "max": 0.45700386933894555 }, - "label": "FuXi", - "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.01593821114079351, + "n": 5, "mean": 0.00519664588898783, + "std": 0.006074020737200343, "min": 0.0016448995745522905, - "n": 5, - "std": 0.006074020737200343 + "max": 0.01593821114079351 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.34740284065122157 + "value": 0.011389521640091117, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.43990346446961154 + "value": 0.019476002782286114, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.23030507895417707 + "value": 0.016701461377870565, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.45700386933894555 + "value": 0.026973684210526316, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.45337080969477445 + "value": 0.008170871559633027, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0019404867913327397 + "value": 0.2251054852320675, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0040170179083803305 + "value": 0.26166119072060445, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0016448995745522905 + "value": 0.1656142112861716, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.002442614029880281 + "value": 0.3231296638237823, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.01593821114079351 + "value": 0.22512986809908905, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.011389521640091117 + "value": 0.35879236229131267, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.019476002782286114 + "value": 0.45937946725189766, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.016701461377870565 + "value": 0.24700654033204764, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.026973684210526316 + "value": 0.48397755354947186, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.008170871559633027 + "value": 0.4615416812544075, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2251054852320675 + "value": 0.34740284065122157, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.26166119072060445 + "value": 0.43990346446961154, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.1656142112861716 + "value": 0.23030507895417707, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3231296638237823 + "value": 0.45700386933894555, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.22512986809908905 + "value": 0.45337080969477445, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.35879236229131267 + "value": 0.0019404867913327397, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.45937946725189766 + "value": 0.0040170179083803305, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.24700654033204764 + "value": 0.0016448995745522905, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.48397755354947186 + "value": 0.002442614029880281, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4615416812544075 + "value": 0.01593821114079351, + "path": "" } ] }, - "strict_f1": { - "max": 0.026973684210526316, - "mean": 0.016542308314081426, - "min": 0.008170871559633027, - "n": 5, - "std": 0.007315696478545316 - }, "thresholds": [ 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "fuxi", + "label": "FuXi", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.013646120927699812, + "std": 0.006773001125535398, + "min": 0.006516342573438146, + "max": 0.0239695995323005 + }, "tolerance_f1": { - "max": 0.3231296638237823, - "mean": 0.24012808383234296, - "min": 0.1656142112861716, "n": 5, - "std": 0.05778381935942338 + "mean": 0.21954844569166232, + "std": 0.058600538898278126, + "min": 0.15340459969134643, + "max": 0.30046102485294895 }, "union_f1": { - "max": 0.48397755354947186, - "mean": 0.4021395209358275, - "min": 0.24700654033204764, "n": 5, - "std": 0.09930723729972839 - } - }, - { + "mean": 0.3673143252863447, + "std": 0.10028910465565644, + "min": 0.23133715755118187, + "max": 0.4554233007850691 + }, "difference": { - "max": 0.43708558281057147, + "n": 5, "mean": 0.3536682043586449, + "std": 0.09922328477614482, "min": 0.21606998197866278, - "n": 5, - "std": 0.09922328477614482 + "max": 0.43708558281057147 }, - "label": "FuXi", - "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.011240568954854669, + "n": 5, "mean": 0.003620779220779221, + "std": 0.004346389540674709, "min": 0.0009029066171923315, - "n": 5, - "std": 0.004346389540674709 + "max": 0.011240568954854669 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2820322009484037 + "value": 0.008802816901408451, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.40169955480281794 + "value": 0.013674670058832883, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.21606998197866278 + "value": 0.015267175572519083, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.4314537012527686 + "value": 0.0239695995323005, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.43708558281057147 + "value": 0.006516342573438146, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0013902288188002473 + "value": 0.18265891116039673, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0031719233147804574 + "value": 0.25519274478475146, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0009029066171923315 + "value": 0.15340459969134643, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0013982683982683983 + "value": 0.30046102485294895, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.011240568954854669 + "value": 0.20602494796886814, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.008802816901408451 + "value": 0.29083501784981214, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.013674670058832883 + "value": 0.4153742248616508, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.015267175572519083 + "value": 0.23133715755118187, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0239695995323005 + "value": 0.4554233007850691, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.006516342573438146 + "value": 0.4436019253840096, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.18265891116039673 + "value": 0.2820322009484037, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.25519274478475146 + "value": 0.40169955480281794, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.15340459969134643 + "value": 0.21606998197866278, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.30046102485294895 + "value": 0.4314537012527686, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.20602494796886814 + "value": 0.43708558281057147, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.29083501784981214 + "value": 0.0013902288188002473, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.4153742248616508 + "value": 0.0031719233147804574, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.23133715755118187 + "value": 0.0009029066171923315, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.4554233007850691 + "value": 0.0013982683982683983, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4436019253840096 + "value": 0.011240568954854669, + "path": "" } ] }, - "strict_f1": { - "max": 0.0239695995323005, - "mean": 0.013646120927699812, - "min": 0.006516342573438146, - "n": 5, - "std": 0.006773001125535398 - }, "thresholds": [ 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24", + "label": "Pangu-Weather", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0006175979748444972, + "std": 6.723431511345116e-05, + "min": 0.0005395660023552084, + "max": 0.000687991383709005 + }, "tolerance_f1": { - "max": 0.30046102485294895, - "mean": 0.21954844569166232, - "min": 0.15340459969134643, "n": 5, - "std": 0.058600538898278126 + "mean": 0.08180329398071068, + "std": 0.013544748931105265, + "min": 0.06046181002685303, + "max": 0.09170559714407109 }, "union_f1": { - "max": 0.4554233007850691, - "mean": 0.3673143252863447, - "min": 0.23133715755118187, "n": 5, - "std": 0.10028910465565644 - } - }, - { + "mean": 0.22056244847642267, + "std": 0.03734327197007739, + "min": 0.16301936607754167, + "max": 0.2480828253168327 + }, "difference": { - "max": 0.24742507585853188, + "n": 5, "mean": 0.2199448505015782, + "std": 0.03728443961138173, "min": 0.1624675064035813, - "n": 5, - "std": 0.03728443961138173 + "max": 0.24742507585853188 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.8827594310451453, + "n": 5, "mean": 0.6338042300556587, + "std": 0.2507607336766575, "min": 0.21222844774273344, - "n": 5, - "std": 0.2507607336766575 + "max": 0.8827594310451453 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.1624675064035813 + "value": 0.0005518596739603732, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.24742507585853188 + "value": 0.0006577494583008349, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2016630331978699 + "value": 0.0005395660023552084, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.24307011894354136 + "value": 0.000687991383709005, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.24509851810436645 + "value": 0.0006508233558970649, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.21222844774273344 + "value": 0.06046181002685303, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7071789734075449 + "value": 0.09170559714407109, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.8827594310451453 + "value": 0.07610963330033767, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.655947804576376 + "value": 0.08961425169301492, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7109064935064935 + "value": 0.09112517773927671, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0005518596739603732 + "value": 0.16301936607754167, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0006577494583008349 + "value": 0.2480828253168327, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0005395660023552084 + "value": 0.20220259920022512, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.000687991383709005 + "value": 0.24375811032725037, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0006508233558970649 + "value": 0.2457493414602635, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.06046181002685303 + "value": 0.1624675064035813, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.09170559714407109 + "value": 0.24742507585853188, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.07610963330033767 + "value": 0.2016630331978699, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.08961425169301492 + "value": 0.24307011894354136, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.09112517773927671 + "value": 0.24509851810436645, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.16301936607754167 + "value": 0.21222844774273344, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2480828253168327 + "value": 0.7071789734075449, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.20220259920022512 + "value": 0.8827594310451453, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.24375811032725037 + "value": 0.655947804576376, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2457493414602635 + "value": 0.7109064935064935, + "path": "" } ] }, - "strict_f1": { - "max": 0.000687991383709005, - "mean": 0.0006175979748444972, - "min": 0.0005395660023552084, - "n": 5, - "std": 6.723431511345116e-05 - }, "thresholds": [ 0.00024381681578233838, 0.0002703598001971841, @@ -12107,192 +12122,192 @@ 0.0002784867538139224, 0.00027882601716555655 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24", + "label": "Pangu-Weather", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.010646547680643631, + "std": 0.009091660121962298, + "min": 0.005639067545640032, + "max": 0.026612927125626056 + }, "tolerance_f1": { - "max": 0.09170559714407109, - "mean": 0.08180329398071068, - "min": 0.06046181002685303, "n": 5, - "std": 0.013544748931105265 + "mean": 0.16674283792931957, + "std": 0.04706476664025482, + "min": 0.1110438397786047, + "max": 0.24179585238367046 }, "union_f1": { - "max": 0.2480828253168327, - "mean": 0.22056244847642267, - "min": 0.16301936607754167, "n": 5, - "std": 0.03734327197007739 - } - }, - { + "mean": 0.3913796290737717, + "std": 0.07015617234020738, + "min": 0.2883417650922405, + "max": 0.4864922341150057 + }, "difference": { - "max": 0.4598793069893796, + "n": 5, "mean": 0.3807330813931281, + "std": 0.06491382662211008, "min": 0.27868986084470576, - "n": 5, - "std": 0.06491382662211008 + "max": 0.4598793069893796 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9213416444048679, + "n": 5, "mean": 0.6241896705253784, + "std": 0.41065405269695054, "min": 0.08085485307212822, - "n": 5, - "std": 0.41065405269695054 + "max": 0.9213416444048679 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4598793069893796 + "value": 0.026612927125626056, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.38914191998978065 + "value": 0.005648734389013078, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3868339716949791 + "value": 0.00568010509540423, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.27868986084470576 + "value": 0.009651904247534756, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.38912034744679525 + "value": 0.005639067545640032, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.08085485307212822 + "value": 0.24179585238367046, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9213416444048679 + "value": 0.1601110414083686, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.9162387454239636 + "value": 0.16063421440737302, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.28134213911150685 + "value": 0.1110438397786047, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9211709706144257 + "value": 0.16012924166858108, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.026612927125626056 + "value": 0.4864922341150057, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.005648734389013078 + "value": 0.39479065437879374, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.00568010509540423 + "value": 0.3925140767903833, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.009651904247534756 + "value": 0.2883417650922405, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.005639067545640032 + "value": 0.3947594149924353, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.24179585238367046 + "value": 0.4598793069893796, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1601110414083686 + "value": 0.38914191998978065, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.16063421440737302 + "value": 0.3868339716949791, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.1110438397786047 + "value": 0.27868986084470576, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16012924166858108 + "value": 0.38912034744679525, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4864922341150057 + "value": 0.08085485307212822, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.39479065437879374 + "value": 0.9213416444048679, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3925140767903833 + "value": 0.9162387454239636, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2883417650922405 + "value": 0.28134213911150685, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.3947594149924353 + "value": 0.9211709706144257, + "path": "" } ] }, - "strict_f1": { - "max": 0.026612927125626056, - "mean": 0.010646547680643631, - "min": 0.005639067545640032, - "n": 5, - "std": 0.009091660121962298 - }, "thresholds": [ 0.0002431461471132934, 0.0002697413437999785, @@ -12300,192 +12315,192 @@ 0.0002780304348561913, 0.00028155450127087533 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24", + "label": "Pangu-Weather", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.007206875839483156, + "std": 0.010865089659576754, + "min": 0.0011827321111768183, + "max": 0.02660504251473943 + }, "tolerance_f1": { - "max": 0.24179585238367046, - "mean": 0.16674283792931957, - "min": 0.1110438397786047, "n": 5, - "std": 0.04706476664025482 + "mean": 0.10446479485399655, + "std": 0.08187646601609166, + "min": 0.019439944457301548, + "max": 0.2411933776075557 }, "union_f1": { - "max": 0.4864922341150057, - "mean": 0.3913796290737717, - "min": 0.2883417650922405, "n": 5, - "std": 0.07015617234020738 - } - }, - { + "mean": 0.24123553210507903, + "std": 0.1613799726515661, + "min": 0.029367206619275145, + "max": 0.4840644650115095 + }, "difference": { - "max": 0.4574594224967701, + "n": 5, "mean": 0.2340286562655959, + "std": 0.15198406003212697, "min": 0.02818447450809833, - "n": 5, - "std": 0.15198406003212697 + "max": 0.4574594224967701 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9565152864351439, + "n": 5, "mean": 0.5820859305431878, + "std": 0.5127551339472974, "min": 0.0007742158899772435, - "n": 5, - "std": 0.5127551339472974 + "max": 0.9565152864351439 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4574594224967701 + "value": 0.02660504251473943, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2284351905540453 + "value": 0.00275028568108307, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2276584353760274 + "value": 0.002750953470140531, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.02818447450809833 + "value": 0.0011827321111768183, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2284057583930383 + "value": 0.0027453654202759313, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.04042742653606411 + "value": 0.2411933776075557, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9565152864351439 + "value": 0.08722255267001126, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.9562827743148313 + "value": 0.08724283500251707, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0007742158899772435 + "value": 0.019439944457301548, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9564299495399228 + "value": 0.08722526453259725, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.02660504251473943 + "value": 0.4840644650115095, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00275028568108307 + "value": 0.23118547623512836, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.002750953470140531 + "value": 0.23040938884616793, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0011827321111768183 + "value": 0.029367206619275145, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0027453654202759313 + "value": 0.23115112381331424, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.2411933776075557 + "value": 0.4574594224967701, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.08722255267001126 + "value": 0.2284351905540453, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.08724283500251707 + "value": 0.2276584353760274, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.019439944457301548 + "value": 0.02818447450809833, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.08722526453259725 + "value": 0.2284057583930383, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4840644650115095 + "value": 0.04042742653606411, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.23118547623512836 + "value": 0.9565152864351439, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.23040938884616793 + "value": 0.9562827743148313, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.029367206619275145 + "value": 0.0007742158899772435, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.23115112381331424 + "value": 0.9564299495399228, + "path": "" } ] }, - "strict_f1": { - "max": 0.02660504251473943, - "mean": 0.007206875839483156, - "min": 0.0011827321111768183, - "n": 5, - "std": 0.010865089659576754 - }, "thresholds": [ 0.00024276715703308582, 0.0002697413437999785, @@ -12493,191 +12508,191 @@ 0.0002780304348561913, 0.000285720539977774 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24", + "label": "Pangu-Weather", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.006615089943495571, + "std": 0.011136980463397231, + "min": 0.0014589084597825474, + "max": 0.026530371070668874 + }, "tolerance_f1": { - "max": 0.2411933776075557, - "mean": 0.10446479485399655, - "min": 0.019439944457301548, "n": 5, - "std": 0.08187646601609166 + "mean": 0.09076165157666524, + "std": 0.08204539789375309, + "min": 0.031681254572138745, + "max": 0.23561614605975847 }, "union_f1": { - "max": 0.4840644650115095, - "mean": 0.24123553210507903, - "min": 0.029367206619275145, "n": 5, - "std": 0.1613799726515661 - } - }, - { + "mean": 0.21219018441019574, + "std": 0.1438368307990196, + "min": 0.09062518509001295, + "max": 0.4621544722787787 + }, "difference": { - "max": 0.43562410120810985, + "n": 5, "mean": 0.20557509446670016, + "std": 0.13312438976032662, "min": 0.08847200309630879, - "n": 5, - "std": 0.13312438976032662 + "max": 0.43562410120810985 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9827272727272728, + "n": 5, "mean": 0.6610079158936302, + "std": 0.4509300379554766, "min": 0.0202152133580705, - "n": 5, - "std": 0.4509300379554766 + "max": 0.9827272727272728 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.43562410120810985 + "value": 0.026530371070668874, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.16739547174950506 + "value": 0.0014589084597825474, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.16835043410148648 + "value": 0.0014680041534393378, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.08847200309630879 + "value": 0.0021531819937041672, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16803346217809073 + "value": 0.0014649840398829276, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0202152133580705 + "value": 0.23561614605975847, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9827272727272728 + "value": 0.06196203978961812, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.9766338899196042 + "value": 0.062336581960243484, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.34681447124304265 + "value": 0.031681254572138745, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9786487322201608 + "value": 0.062212235501567396, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.026530371070668874 + "value": 0.4621544722787787, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0014589084597825474 + "value": 0.1688543802092876, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0014680041534393378 + "value": 0.16981843825492582, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0021531819937041672 + "value": 0.09062518509001295, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0014649840398829276 + "value": 0.16949844621797366, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.23561614605975847 + "value": 0.43562410120810985, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.06196203978961812 + "value": 0.16739547174950506, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.062336581960243484 + "value": 0.16835043410148648, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.031681254572138745 + "value": 0.08847200309630879, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.062212235501567396 + "value": 0.16803346217809073, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4621544722787787 + "value": 0.0202152133580705, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1688543802092876 + "value": 0.9827272727272728, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.16981843825492582 + "value": 0.9766338899196042, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.09062518509001295 + "value": 0.34681447124304265, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", - "seed": 123, - "value": 0.16949844621797366 - } - ] - }, - "strict_f1": { - "max": 0.026530371070668874, - "mean": 0.006615089943495571, - "min": 0.0014589084597825474, - "n": 5, - "std": 0.011136980463397231 + "seed": 123, + "value": 0.9786487322201608, + "path": "" + } + ] }, "thresholds": [ 0.0002, @@ -12686,1719 +12701,1719 @@ 0.00027438640245236456, 0.00028135377215221524 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24_h24", + "label": "Pangu-Weather", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.002839244726016104, + "std": 0.0014055201553382893, + "min": 0.0015570260801868433, + "max": 0.00458918723078781 + }, "tolerance_f1": { - "max": 0.23561614605975847, - "mean": 0.09076165157666524, - "min": 0.031681254572138745, "n": 5, - "std": 0.08204539789375309 + "mean": 0.11344735239772843, + "std": 0.05215606001713916, + "min": 0.04422040571422221, + "max": 0.17658327178527805 }, "union_f1": { - "max": 0.4621544722787787, - "mean": 0.21219018441019574, - "min": 0.09062518509001295, "n": 5, - "std": 0.1438368307990196 - } - }, - { + "mean": 0.26579704586756064, + "std": 0.12358583427828769, + "min": 0.1360311898534274, + "max": 0.4448076750202692 + }, "difference": { - "max": 0.44278198559528875, + "n": 5, "mean": 0.26295780114154454, + "std": 0.12366414839800409, "min": 0.13447416377324056, - "n": 5, - "std": 0.12366414839800409 + "max": 0.44278198559528875 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.015876932591218307, + "n": 5, "mean": 0.004966580086580087, + "std": 0.006165207367586338, "min": 0.0013461966604823748, - "n": 5, - "std": 0.006165207367586338 + "max": 0.015876932591218307 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1610876335708797 + "value": 0.00412434125105018, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2698875125828892 + "value": 0.0018999796430752525, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.44278198559528875 + "value": 0.002025689424980434, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3065577101854246 + "value": 0.00458918723078781, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.13447416377324056 + "value": 0.0015570260801868433, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0013768707482993197 + "value": 0.08503531719948608, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0034029684601113172 + "value": 0.11199020227536667, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.015876932591218307 + "value": 0.17658327178527805, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0028299319727891157 + "value": 0.14940756501428915, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0013461966604823748 + "value": 0.04422040571422221, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00412434125105018 + "value": 0.1652119748219299, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0018999796430752525 + "value": 0.27178749222596443, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.002025689424980434 + "value": 0.4448076750202692, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.00458918723078781 + "value": 0.31114689741621243, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0015570260801868433 + "value": 0.1360311898534274, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.08503531719948608 + "value": 0.1610876335708797, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.11199020227536667 + "value": 0.2698875125828892, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.17658327178527805 + "value": 0.44278198559528875, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.14940756501428915 + "value": 0.3065577101854246, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.04422040571422221 + "value": 0.13447416377324056, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1652119748219299 + "value": 0.0013768707482993197, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.27178749222596443 + "value": 0.0034029684601113172, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.4448076750202692 + "value": 0.015876932591218307, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.31114689741621243 + "value": 0.0028299319727891157, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.1360311898534274 + "value": 0.0013461966604823748, + "path": "" } ] }, - "strict_f1": { - "max": 0.00458918723078781, - "mean": 0.002839244726016104, - "min": 0.0015570260801868433, - "n": 5, - "std": 0.0014055201553382893 - }, "thresholds": [ 0.1, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24_h24", + "label": "Pangu-Weather", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.010727828314232248, + "std": 0.0030802455090206595, + "min": 0.006323656223052602, + "max": 0.014334527877323108 + }, "tolerance_f1": { - "max": 0.17658327178527805, - "mean": 0.11344735239772843, - "min": 0.04422040571422221, "n": 5, - "std": 0.05215606001713916 + "mean": 0.16440632872776462, + "std": 0.04420834148717055, + "min": 0.1236361475439318, + "max": 0.23786880092608761 }, "union_f1": { - "max": 0.4448076750202692, - "mean": 0.26579704586756064, - "min": 0.1360311898534274, "n": 5, - "std": 0.12358583427828769 - } - }, - { + "mean": 0.36002559619997526, + "std": 0.11617972461830063, + "min": 0.26905489411020966, + "max": 0.5563138131518957 + }, "difference": { - "max": 0.5419792852745725, + "n": 5, "mean": 0.34929776788574307, + "std": 0.1137924578244252, "min": 0.2627312378871571, - "n": 5, - "std": 0.1137924578244252 + "max": 0.5419792852745725 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.02707034728406055, + "n": 5, "mean": 0.014698228950232511, + "std": 0.008898365132157035, "min": 0.00599584446423271, - "n": 5, - "std": 0.008898365132157035 + "max": 0.02707034728406055 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.27422409940357173 + "value": 0.00961323496534764, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2627312378871571 + "value": 0.006323656223052602, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.5419792852745725 + "value": 0.014334527877323108, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3107866198278387 + "value": 0.012796587576646227, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.35676759703557515 + "value": 0.010571134928791661, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.019518650440288908 + "value": 0.13752896018113672, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.00599584446423271 + "value": 0.1236361475439318, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.02707034728406055 + "value": 0.23786880092608761, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.006668645493222519 + "value": 0.15839502058327895, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.014237657069357871 + "value": 0.16460271440438795, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00961323496534764 + "value": 0.2838373343689194, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.006323656223052602 + "value": 0.26905489411020966, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.014334527877323108 + "value": 0.5563138131518957, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.012796587576646227 + "value": 0.3235832074044849, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.010571134928791661 + "value": 0.3673387319643668, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.13752896018113672 + "value": 0.27422409940357173, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.1236361475439318 + "value": 0.2627312378871571, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.23786880092608761 + "value": 0.5419792852745725, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.15839502058327895 + "value": 0.3107866198278387, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.16460271440438795 + "value": 0.35676759703557515, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2838373343689194 + "value": 0.019518650440288908, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.26905489411020966 + "value": 0.00599584446423271, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.5563138131518957 + "value": 0.02707034728406055, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3235832074044849 + "value": 0.006668645493222519, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.3673387319643668 + "value": 0.014237657069357871, + "path": "" } ] }, - "strict_f1": { - "max": 0.014334527877323108, - "mean": 0.010727828314232248, - "min": 0.006323656223052602, - "n": 5, - "std": 0.0030802455090206595 - }, "thresholds": [ 0.1, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24_h24", + "label": "Pangu-Weather", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.006807753305469981, + "std": 0.0036980018094160414, + "min": 0.002669336002669336, + "max": 0.010952623535404991 + }, "tolerance_f1": { - "max": 0.23786880092608761, - "mean": 0.16440632872776462, - "min": 0.1236361475439318, "n": 5, - "std": 0.04420834148717055 + "mean": 0.10945088912540306, + "std": 0.06122864367998089, + "min": 0.03347640061525648, + "max": 0.19299696406724362 }, "union_f1": { - "max": 0.5563138131518957, - "mean": 0.36002559619997526, - "min": 0.26905489411020966, "n": 5, - "std": 0.11617972461830063 - } - }, - { + "mean": 0.2511409611173737, + "std": 0.14418045901223492, + "min": 0.12011082544991639, + "max": 0.478233072358786 + }, "difference": { - "max": 0.467280448823381, + "n": 5, "mean": 0.24433320781190368, + "std": 0.14096512937898192, "min": 0.11744148944724705, - "n": 5, - "std": 0.14096512937898192 + "max": 0.467280448823381 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.018105026219451866, + "n": 5, "mean": 0.006657019887206886, + "std": 0.006467521103800908, "min": 0.002389433066191748, - "n": 5, - "std": 0.006467521103800908 + "max": 0.018105026219451866 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.14003318233886966 + "value": 0.005579507988840983, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2112720683625158 + "value": 0.004407051282051282, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.467280448823381 + "value": 0.010952623535404991, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.285638850087505 + "value": 0.010430247718383311, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.11744148944724705 + "value": 0.002669336002669336, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0035594142673394677 + "value": 0.07891517659613233, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.004856782428020184 + "value": 0.09804613935969869, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.018105026219451866 + "value": 0.19299696406724362, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.004374443455031167 + "value": 0.14381976498868418, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.002389433066191748 + "value": 0.03347640061525648, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.005579507988840983 + "value": 0.14561269032771065, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.004407051282051282 + "value": 0.2156791196445671, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.010952623535404991 + "value": 0.478233072358786, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.010430247718383311 + "value": 0.2960690978058883, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.002669336002669336 + "value": 0.12011082544991639, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.07891517659613233 + "value": 0.14003318233886966, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.09804613935969869 + "value": 0.2112720683625158, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.19299696406724362 + "value": 0.467280448823381, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.14381976498868418 + "value": 0.285638850087505, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.03347640061525648 + "value": 0.11744148944724705, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.14561269032771065 + "value": 0.0035594142673394677, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.2156791196445671 + "value": 0.004856782428020184, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.478233072358786 + "value": 0.018105026219451866, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2960690978058883 + "value": 0.004374443455031167, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.12011082544991639 + "value": 0.002389433066191748, + "path": "" } ] }, - "strict_f1": { - "max": 0.010952623535404991, - "mean": 0.006807753305469981, - "min": 0.002669336002669336, - "n": 5, - "std": 0.0036980018094160414 - }, "thresholds": [ 0.1, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu24_h24", + "label": "Pangu-Weather", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.004657283091657516, + "std": 0.002873310351989516, + "min": 0.002313475997686524, + "max": 0.0094525403702245 + }, "tolerance_f1": { - "max": 0.19299696406724362, - "mean": 0.10945088912540306, - "min": 0.03347640061525648, "n": 5, - "std": 0.06122864367998089 + "mean": 0.08713410482710718, + "std": 0.03539578269908524, + "min": 0.036969834429575875, + "max": 0.1330139941503748 }, "union_f1": { - "max": 0.478233072358786, - "mean": 0.2511409611173737, - "min": 0.12011082544991639, "n": 5, - "std": 0.14418045901223492 - } - }, - { + "mean": 0.209055617580378, + "std": 0.08553820555850469, + "min": 0.12157774813170726, + "max": 0.3168721772329861 + }, "difference": { - "max": 0.31356600176953114, + "n": 5, "mean": 0.2043983344887205, + "std": 0.08451922309197234, "min": 0.11926427213402073, - "n": 5, - "std": 0.08451922309197234 + "max": 0.31356600176953114 }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather_h24", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.004520098948670378, + "n": 5, "mean": 0.0027991341991341996, + "std": 0.0012806432499773903, "min": 0.0014211502782931354, - "n": 5, - "std": 0.0012806432499773903 + "max": 0.004520098948670378 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.13031562548755435 + "value": 0.0051329911339244045, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.19289957921827447 + "value": 0.003081232492997199, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.31356600176953114 + "value": 0.003306175463454953, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2659461938342217 + "value": 0.0094525403702245, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.11926427213402073 + "value": 0.002313475997686524, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0019332096474953618 + "value": 0.07335569213391166, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0036982065553494126 + "value": 0.0918614020950846, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.004520098948670378 + "value": 0.10046960132658891, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.002423005565862709 + "value": 0.1330139941503748, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0014211502782931354 + "value": 0.036969834429575875, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0051329911339244045 + "value": 0.13544861662147875, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.003081232492997199 + "value": 0.19598081171127166, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.003306175463454953 + "value": 0.3168721772329861, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0094525403702245 + "value": 0.2753987342044462, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.002313475997686524 + "value": 0.12157774813170726, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.07335569213391166 + "value": 0.13031562548755435, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0918614020950846 + "value": 0.19289957921827447, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.10046960132658891 + "value": 0.31356600176953114, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.1330139941503748 + "value": 0.2659461938342217, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.036969834429575875 + "value": 0.11926427213402073, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.13544861662147875 + "value": 0.0019332096474953618, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.19598081171127166 + "value": 0.0036982065553494126, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3168721772329861 + "value": 0.004520098948670378, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2753987342044462 + "value": 0.002423005565862709, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.12157774813170726 + "value": 0.0014211502782931354, + "path": "" } ] }, - "strict_f1": { - "max": 0.0094525403702245, - "mean": 0.004657283091657516, - "min": 0.002313475997686524, - "n": 5, - "std": 0.002873310351989516 - }, "thresholds": [ 0.15, 0.3, 0.5 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu6", + "label": "Pangu-Weather", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0027554617131344606, + "std": 0.0010888849544285033, + "min": 0.0014740203906154037, + "max": 0.003978746443051044 + }, "tolerance_f1": { - "max": 0.1330139941503748, - "mean": 0.08713410482710718, - "min": 0.036969834429575875, "n": 5, - "std": 0.03539578269908524 + "mean": 0.17090859723170176, + "std": 0.04047686720194535, + "min": 0.12218430629860097, + "max": 0.2264875300093167 }, "union_f1": { - "max": 0.3168721772329861, - "mean": 0.209055617580378, - "min": 0.12157774813170726, "n": 5, - "std": 0.08553820555850469 - } - }, - { + "mean": 0.35638584510932614, + "std": 0.09032661075096597, + "min": 0.22268222965661022, + "max": 0.46795172614222125 + }, "difference": { - "max": 0.4653493031095141, + "n": 5, "mean": 0.3536303833961917, + "std": 0.0907743328516622, "min": 0.21893843540252053, - "n": 5, - "std": 0.0907743328516622 + "max": 0.4653493031095141 }, - "label": "Pangu-Weather", - "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.029129251700680273, + "n": 5, "mean": 0.010800964749536178, + "std": 0.011307793167633017, "min": 0.0012771799628942485, - "n": 5, - "std": 0.011307793167633017 + "max": 0.029129251700680273 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.21893843540252053 + "value": 0.0037437942540896885, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3981469865854522 + "value": 0.0019783244452090142, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3350135537819443 + "value": 0.0014740203906154037, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.35070363810152716 + "value": 0.003978746443051044, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.4653493031095141 + "value": 0.0026024230327071516, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0012771799628942485 + "value": 0.12218430629860097, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.01413716759431045 + "value": 0.1809729686535962, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.00479208410636982 + "value": 0.14188194761227685, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.004669140383426098 + "value": 0.18301623358471802, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.029129251700680273 + "value": 0.2264875300093167, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0037437942540896885 + "value": 0.22268222965661022, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.0019783244452090142 + "value": 0.40012531103066123, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0014740203906154037 + "value": 0.3364875741725597, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.003978746443051044 + "value": 0.3546823845445782, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.0026024230327071516 + "value": 0.46795172614222125, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.12218430629860097 + "value": 0.21893843540252053, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.1809729686535962 + "value": 0.3981469865854522, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.14188194761227685 + "value": 0.3350135537819443, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.18301623358471802 + "value": 0.35070363810152716, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.2264875300093167 + "value": 0.4653493031095141, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.22268222965661022 + "value": 0.0012771799628942485, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.40012531103066123 + "value": 0.01413716759431045, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3364875741725597 + "value": 0.00479208410636982, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3546823845445782 + "value": 0.004669140383426098, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.46795172614222125 + "value": 0.029129251700680273, + "path": "" } ] }, + "thresholds": [ + 0.15, + 0.2, + 0.3 + ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu6", + "label": "Pangu-Weather", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, "strict_f1": { - "max": 0.003978746443051044, - "mean": 0.0027554617131344606, - "min": 0.0014740203906154037, "n": 5, - "std": 0.0010888849544285033 + "mean": 0.013655691986016547, + "std": 0.003064288092894302, + "min": 0.010515247108307044, + "max": 0.017845192951148784 }, - "thresholds": [ - 0.15, - 0.2, - 0.3 - ], "tolerance_f1": { - "max": 0.2264875300093167, - "mean": 0.17090859723170176, - "min": 0.12218430629860097, "n": 5, - "std": 0.04047686720194535 + "mean": 0.2222219561322302, + "std": 0.068612974451766, + "min": 0.12664654352782623, + "max": 0.3098064830268362 }, "union_f1": { - "max": 0.46795172614222125, - "mean": 0.35638584510932614, - "min": 0.22268222965661022, "n": 5, - "std": 0.09032661075096597 - } - }, - { + "mean": 0.4342335956184513, + "std": 0.13238331958267657, + "min": 0.22723717713750308, + "max": 0.5952835470121373 + }, "difference": { - "max": 0.5794332814358789, + "n": 5, "mean": 0.42057790363243475, + "std": 0.13059925591392604, "min": 0.21672193002919604, - "n": 5, - "std": 0.13059925591392604 + "max": 0.5794332814358789 }, - "label": "Pangu-Weather", - "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.056067576926882357, + "n": 5, "mean": 0.02054862966261007, + "std": 0.02189321852493811, "min": 0.004447412684278223, - "n": 5, - "std": 0.02189321852493811 + "max": 0.056067576926882357 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.21672193002919604 + "value": 0.010515247108307044, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.4579044245182327 + "value": 0.011848926191063935, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.42661251656646776 + "value": 0.012218828103304637, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.4222173656123986 + "value": 0.017845192951148784, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5794332814358789 + "value": 0.015850265576258328, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.004447412684278223 + "value": 0.12664654352782623, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.027451271396062134 + "value": 0.23514322359982898, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.006297615514000198 + "value": 0.1893747086942956, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.008479271791827446 + "value": 0.2501388218123639, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.056067576926882357 + "value": 0.3098064830268362, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.010515247108307044 + "value": 0.22723717713750308, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.011848926191063935 + "value": 0.46975335070929664, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.012218828103304637 + "value": 0.4388313446697724, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.017845192951148784 + "value": 0.44006255856354737, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.015850265576258328 + "value": 0.5952835470121373, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.12664654352782623 + "value": 0.21672193002919604, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.23514322359982898 + "value": 0.4579044245182327, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.1893747086942956 + "value": 0.42661251656646776, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2501388218123639 + "value": 0.4222173656123986, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.3098064830268362 + "value": 0.5794332814358789, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.22723717713750308 + "value": 0.004447412684278223, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.46975335070929664 + "value": 0.027451271396062134, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.4388313446697724 + "value": 0.006297615514000198, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.44006255856354737 + "value": 0.008479271791827446, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5952835470121373 + "value": 0.056067576926882357, + "path": "" } ] }, - "strict_f1": { - "max": 0.017845192951148784, - "mean": 0.013655691986016547, - "min": 0.010515247108307044, - "n": 5, - "std": 0.003064288092894302 - }, "thresholds": [ 0.15, 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu6", + "label": "Pangu-Weather", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.010931248966985132, + "std": 0.003535277885116421, + "min": 0.00788436268068331, + "max": 0.016403526758253024 + }, "tolerance_f1": { - "max": 0.3098064830268362, - "mean": 0.2222219561322302, - "min": 0.12664654352782623, "n": 5, - "std": 0.068612974451766 + "mean": 0.18933650687857217, + "std": 0.0593294088865109, + "min": 0.1079162350962175, + "max": 0.25501453744070673 }, "union_f1": { - "max": 0.5952835470121373, - "mean": 0.4342335956184513, - "min": 0.22723717713750308, "n": 5, - "std": 0.13238331958267657 - } - }, - { + "mean": 0.3853250982462472, + "std": 0.1172207993601839, + "min": 0.19681189159430446, + "max": 0.5172553944576963 + }, "difference": { - "max": 0.5050483949122122, + "n": 5, "mean": 0.3743938492792621, + "std": 0.11526082075130117, "min": 0.18892752891362113, - "n": 5, - "std": 0.11526082075130117 + "max": 0.5050483949122122 }, - "label": "Pangu-Weather", - "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.03677772830711388, + "n": 5, "mean": 0.014029385574354408, + "std": 0.014738630997131075, "min": 0.0033887404768972, - "n": 5, - "std": 0.014738630997131075 + "max": 0.03677772830711388 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.18892752891362113 + "value": 0.00788436268068331, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.37162889777394226 + "value": 0.008011413520632135, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.4013551577246312 + "value": 0.010149942329873126, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.40500926707190377 + "value": 0.016403526758253024, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5050483949122122 + "value": 0.012206999545484059, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0033887404768972 + "value": 0.1079162350962175, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.021221678044919364 + "value": 0.18336249012390426, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.004044226773523301 + "value": 0.1623425297604306, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.004714554269318294 + "value": 0.23804674197160172, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.03677772830711388 + "value": 0.25501453744070673, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00788436268068331 + "value": 0.19681189159430446, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.008011413520632135 + "value": 0.3796403112945744, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.010149942329873126 + "value": 0.41150510005450436, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.016403526758253024 + "value": 0.4214127938301568, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.012206999545484059 + "value": 0.5172553944576963, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1079162350962175 + "value": 0.18892752891362113, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.18336249012390426 + "value": 0.37162889777394226, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.1623425297604306 + "value": 0.4013551577246312, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.23804674197160172 + "value": 0.40500926707190377, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.25501453744070673 + "value": 0.5050483949122122, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.19681189159430446 + "value": 0.0033887404768972, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3796403112945744 + "value": 0.021221678044919364, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.41150510005450436 + "value": 0.004044226773523301, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.4214127938301568 + "value": 0.004714554269318294, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.5172553944576963 + "value": 0.03677772830711388, + "path": "" } ] }, - "strict_f1": { - "max": 0.016403526758253024, - "mean": 0.010931248966985132, - "min": 0.00788436268068331, - "n": 5, - "std": 0.003535277885116421 - }, "thresholds": [ 0.15, 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "pangu6", + "label": "Pangu-Weather", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.008844166237577134, + "std": 0.0036010444504776733, + "min": 0.005117388748660143, + "max": 0.01471129091577786 + }, "tolerance_f1": { - "max": 0.25501453744070673, - "mean": 0.18933650687857217, - "min": 0.1079162350962175, "n": 5, - "std": 0.0593294088865109 + "mean": 0.17017232303990823, + "std": 0.05485942193604147, + "min": 0.09937369519832985, + "max": 0.226729727407505 }, "union_f1": { - "max": 0.5172553944576963, - "mean": 0.3853250982462472, - "min": 0.19681189159430446, "n": 5, - "std": 0.1172207993601839 - } - }, - { + "mean": 0.3456881777429305, + "std": 0.10293157698741554, + "min": 0.18187865016757276, + "max": 0.45415946247619876 + }, "difference": { - "max": 0.44522188799641144, + "n": 5, "mean": 0.3368440115053533, + "std": 0.10133432810494146, "min": 0.17487748330642924, - "n": 5, - "std": 0.10133432810494146 + "max": 0.44522188799641144 }, - "label": "Pangu-Weather", - "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.026268398268398267, + "n": 5, "mean": 0.010103277674706247, + "std": 0.011083627907574344, "min": 0.001932591218305504, - "n": 5, - "std": 0.011083627907574344 + "max": 0.026268398268398267 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.17487748330642924 + "value": 0.0070011668611435224, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.3180949408491859 + "value": 0.005117388748660143, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.36502027762951006 + "value": 0.008453410182516812, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3810054677452299 + "value": 0.01471129091577786, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.44522188799641144 + "value": 0.008937574479787332, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.001932591218305504 + "value": 0.09937369519832985, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.017168212739641312 + "value": 0.14938342628742876, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.0025015460729746444 + "value": 0.15039592221466835, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.0026456400742115027 + "value": 0.2249788440916092, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.026268398268398267 + "value": 0.226729727407505, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0070011668611435224 + "value": 0.18187865016757276, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.005117388748660143 + "value": 0.323212329597846, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.008453410182516812 + "value": 0.3734736878120269, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.01471129091577786 + "value": 0.3957167586610078, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.008937574479787332 + "value": 0.45415946247619876, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.09937369519832985 + "value": 0.17487748330642924, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.14938342628742876 + "value": 0.3180949408491859, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.15039592221466835 + "value": 0.36502027762951006, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.2249788440916092 + "value": 0.3810054677452299, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.226729727407505 + "value": 0.44522188799641144, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.18187865016757276 + "value": 0.001932591218305504, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 17, - "value": 0.323212329597846 + "value": 0.017168212739641312, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 37, - "value": 0.3734736878120269 + "value": 0.0025015460729746444, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 57, - "value": 0.3957167586610078 + "value": 0.0026456400742115027, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 97, - "value": 0.45415946247619876 + "value": 0.026268398268398267, + "path": "" } ] }, - "strict_f1": { - "max": 0.01471129091577786, - "mean": 0.008844166237577134, - "min": 0.005117388748660143, - "n": 5, - "std": 0.0036010444504776733 - }, "thresholds": [ 0.15, 0.2, 0.3 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "prithvi_wxc", + "label": "Prithvi-WxC", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0005524855672037344, + "std": 3.864412600940357e-05, + "min": 0.0005057825179019543, + "max": 0.0006040339888888368 + }, "tolerance_f1": { - "max": 0.226729727407505, - "mean": 0.17017232303990823, - "min": 0.09937369519832985, "n": 5, - "std": 0.05485942193604147 + "mean": 0.07164938383885015, + "std": 0.006556565946161047, + "min": 0.06182533987250242, + "max": 0.07892160427759505 }, "union_f1": { - "max": 0.45415946247619876, - "mean": 0.3456881777429305, - "min": 0.18187865016757276, "n": 5, - "std": 0.10293157698741554 - } - }, - { + "mean": 0.20185302299729843, + "std": 0.018299428665013905, + "min": 0.17269452770359542, + "max": 0.22218157532367563 + }, "difference": { - "max": 0.2215775413347868, + "n": 5, "mean": 0.20130053743009468, + "std": 0.018296989279232255, "min": 0.17211673370575886, - "n": 5, - "std": 0.018296989279232255 + "max": 0.2215775413347868 }, - "label": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9588571428571429, + "n": 5, "mean": 0.6197105751391466, + "std": 0.30910050227102126, "min": 0.22581076066790354, - "n": 5, - "std": 0.30910050227102126 + "max": 0.9588571428571429 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.2215775413347868 + "value": 0.0006040339888888368, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.20983886056574044 + "value": 0.000532941187181528, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.17211673370575886 + "value": 0.0005777939978365554, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.20276874034790468 + "value": 0.0005418761442097979, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.20020081119628272 + "value": 0.0005057825179019543, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5006159554730983 + "value": 0.07892160427759505, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9099814471243043 + "value": 0.07586161173658511, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.22581076066790354 + "value": 0.06182533987250242, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5032875695732839 + "value": 0.06950848282026648, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9588571428571429 + "value": 0.07212988048730175, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0006040339888888368 + "value": 0.22218157532367563, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.000532941187181528 + "value": 0.21037180175292197, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0005777939978365554 + "value": 0.17269452770359542, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0005418761442097979 + "value": 0.2033106164921145, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0005057825179019543 + "value": 0.20070659371418467, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.07892160427759505 + "value": 0.2215775413347868, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.07586161173658511 + "value": 0.20983886056574044, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.06182533987250242 + "value": 0.17211673370575886, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.06950848282026648 + "value": 0.20276874034790468, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.07212988048730175 + "value": 0.20020081119628272, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.22218157532367563 + "value": 0.5006159554730983, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.21037180175292197 + "value": 0.9099814471243043, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.17269452770359542 + "value": 0.22581076066790354, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2033106164921145 + "value": 0.5032875695732839, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.20070659371418467 + "value": 0.9588571428571429, + "path": "" } ] }, - "strict_f1": { - "max": 0.0006040339888888368, - "mean": 0.0005524855672037344, - "min": 0.0005057825179019543, - "n": 5, - "std": 3.864412600940357e-05 - }, "thresholds": [ 0.0002606518683023751, 0.00028123805532231927, @@ -14406,385 +14421,385 @@ 0.00028816089616157115, 0.00028831177041865885 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "prithvi_wxc", + "label": "Prithvi-WxC", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.01411939737783544, + "std": 0.011635188932059573, + "min": 0.005636060100166944, + "max": 0.027781926811053025 + }, "tolerance_f1": { - "max": 0.07892160427759505, - "mean": 0.07164938383885015, - "min": 0.06182533987250242, "n": 5, - "std": 0.006556565946161047 + "mean": 0.19263594340017406, + "std": 0.04501855438328491, + "min": 0.15977956000729543, + "max": 0.24416957191197788 }, "union_f1": { - "max": 0.22218157532367563, - "mean": 0.20185302299729843, - "min": 0.17269452770359542, "n": 5, - "std": 0.018299428665013905 - } - }, - { + "mean": 0.4257931093416185, + "std": 0.04549524292260173, + "min": 0.39407737525514647, + "max": 0.4925384357395131 + }, "difference": { - "max": 0.46663155596188977, + "n": 5, "mean": 0.4116737119637831, + "std": 0.034845536781985785, "min": 0.38844131515497954, - "n": 5, - "std": 0.034845536781985785 + "max": 0.46663155596188977 }, - "label": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9234194122885129, + "n": 5, "mean": 0.5741169486494508, + "std": 0.47898702418670663, "min": 0.013950727218759276, - "n": 5, - "std": 0.47898702418670663 + "max": 0.9234194122885129 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.38844131515497954 + "value": 0.005636060100166944, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.38844131515497954 + "value": 0.005636060100166944, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.46663155596188977 + "value": 0.02590687977762335, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4264130583920869 + "value": 0.027781926811053025, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.38844131515497954 + "value": 0.005636060100166944, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.9234194122885129 + "value": 0.15977956000729543, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9234194122885129 + "value": 0.15977956000729543, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.08637577916295637 + "value": 0.23967146506700618, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.013950727218759276 + "value": 0.24416957191197788, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9234194122885129 + "value": 0.15977956000729543, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.005636060100166944 + "value": 0.39407737525514647, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.005636060100166944 + "value": 0.39407737525514647, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.02590687977762335 + "value": 0.4925384357395131, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.027781926811053025 + "value": 0.4541949852031399, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.005636060100166944 + "value": 0.39407737525514647, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.15977956000729543 + "value": 0.38844131515497954, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.15977956000729543 + "value": 0.38844131515497954, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.23967146506700618 + "value": 0.46663155596188977, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.24416957191197788 + "value": 0.4264130583920869, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.15977956000729543 + "value": 0.38844131515497954, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.39407737525514647 + "value": 0.9234194122885129, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.39407737525514647 + "value": 0.9234194122885129, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4925384357395131 + "value": 0.08637577916295637, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4541949852031399 + "value": 0.013950727218759276, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.39407737525514647 + "value": 0.9234194122885129, + "path": "" } ] }, + "thresholds": [ + 0.00018521470337873325, + 0.0002438416049699299, + 0.0002606456328066997, + 0.0002606526104500517, + 0.0002883114793803543 + ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "prithvi_wxc", + "label": "Prithvi-WxC", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, "strict_f1": { - "max": 0.027781926811053025, - "mean": 0.01411939737783544, - "min": 0.005636060100166944, "n": 5, - "std": 0.011635188932059573 + "mean": 0.012376418057865393, + "std": 0.013201257527312249, + "min": 0.002747305898968793, + "max": 0.02774049217002237 }, - "thresholds": [ - 0.00018521470337873325, - 0.0002438416049699299, - 0.0002606456328066997, - 0.0002606526104500517, - 0.0002883114793803543 - ], "tolerance_f1": { - "max": 0.24416957191197788, - "mean": 0.19263594340017406, - "min": 0.15977956000729543, "n": 5, - "std": 0.04501855438328491 + "mean": 0.14878019275284607, + "std": 0.08442915398703313, + "min": 0.08713204456539597, + "max": 0.24342288270264262 }, "union_f1": { - "max": 0.4925384357395131, - "mean": 0.4257931093416185, - "min": 0.39407737525514647, "n": 5, - "std": 0.04549524292260173 - } - }, - { + "mean": 0.32691282462654847, + "std": 0.13208480223417152, + "min": 0.2309636203280157, + "max": 0.4900551948025372 + }, "difference": { - "max": 0.46415551438013897, + "n": 5, "mean": 0.31453640656868304, + "std": 0.11905339505419958, "min": 0.2282163144290469, - "n": 5, - "std": 0.11905339505419958 + "max": 0.46415551438013897 }, - "label": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9575541703769664, + "n": 5, "mean": 0.5845651528643514, + "std": 0.5108966939133424, "min": 0.006975363609379638, - "n": 5, - "std": 0.5108966939133424 + "max": 0.9575541703769664 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.2282163144290469 + "value": 0.002747305898968793, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2282163144290469 + "value": 0.002747305898968793, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.46415551438013897 + "value": 0.02589968042239822, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4238775751761356 + "value": 0.02774049217002237, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2282163144290469 + "value": 0.002747305898968793, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.9575541703769664 + "value": 0.08713204456539597, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9575541703769664 + "value": 0.08713204456539597, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.043187889581478185 + "value": 0.23908194736539978, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.006975363609379638 + "value": 0.24342288270264262, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9575541703769664 + "value": 0.08713204456539597, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.002747305898968793 + "value": 0.2309636203280157, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.002747305898968793 + "value": 0.2309636203280157, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.02589968042239822 + "value": 0.4900551948025372, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.02774049217002237 + "value": 0.451618067346158, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.002747305898968793 + "value": 0.2309636203280157, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.08713204456539597 + "value": 0.2282163144290469, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.08713204456539597 + "value": 0.2282163144290469, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.23908194736539978 + "value": 0.46415551438013897, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.24342288270264262 + "value": 0.4238775751761356, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.08713204456539597 + "value": 0.2282163144290469, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.2309636203280157 + "value": 0.9575541703769664, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2309636203280157 + "value": 0.9575541703769664, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4900551948025372 + "value": 0.043187889581478185, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.451618067346158 + "value": 0.006975363609379638, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2309636203280157 + "value": 0.9575541703769664, + "path": "" } ] }, - "strict_f1": { - "max": 0.02774049217002237, - "mean": 0.012376418057865393, - "min": 0.002747305898968793, - "n": 5, - "std": 0.013201257527312249 - }, "thresholds": [ 0.00018521470337873325, 0.0002438416049699299, @@ -14792,192 +14807,192 @@ 0.0002606526104500517, 0.0002883114793803543 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "prithvi_wxc", + "label": "Prithvi-WxC", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.011519506840078677, + "std": 0.013769946337526496, + "min": 0.001471035812113219, + "max": 0.027352941176470587 + }, "tolerance_f1": { - "max": 0.24342288270264262, - "mean": 0.14878019275284607, - "min": 0.08713204456539597, "n": 5, - "std": 0.08442915398703313 + "mean": 0.13151154173948754, + "std": 0.09455648302229831, + "min": 0.06246138809231819, + "max": 0.2365506772536198 }, "union_f1": { - "max": 0.4900551948025372, - "mean": 0.32691282462654847, - "min": 0.2309636203280157, "n": 5, - "std": 0.13208480223417152 - } - }, - { + "mean": 0.2813191893577128, + "std": 0.15286646962866243, + "min": 0.17013954201238674, + "max": 0.4676561751411253 + }, "difference": { - "max": 0.44182468955354215, + "n": 5, "mean": 0.2697996825176341, + "std": 0.13922359360588243, "min": 0.16866850620027352, - "n": 5, - "std": 0.13922359360588243 + "max": 0.44182468955354215 }, - "label": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9746196660482375, + "n": 5, "mean": 0.5897884972170686, + "std": 0.5269906679651803, "min": 0.0034879406307977735, - "n": 5, - "std": 0.5269906679651803 + "max": 0.9746196660482375 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.16866850620027352 + "value": 0.001471035812113219, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.16866850620027352 + "value": 0.001471035812113219, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.44182468955354215 + "value": 0.025831485587583147, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4011682044338078 + "value": 0.027352941176470587, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16866850620027352 + "value": 0.001471035812113219, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.9746196660482375 + "value": 0.06246138809231819, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9746196660482375 + "value": 0.06246138809231819, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.021595547309833023 + "value": 0.23362286716686329, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0034879406307977735 + "value": 0.2365506772536198, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9746196660482375 + "value": 0.06246138809231819, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.001471035812113219 + "value": 0.17013954201238674, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.001471035812113219 + "value": 0.17013954201238674, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.025831485587583147 + "value": 0.4676561751411253, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.027352941176470587 + "value": 0.4285211456102784, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.001471035812113219 + "value": 0.17013954201238674, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.06246138809231819 + "value": 0.16866850620027352, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.06246138809231819 + "value": 0.16866850620027352, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.23362286716686329 + "value": 0.44182468955354215, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2365506772536198 + "value": 0.4011682044338078, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.06246138809231819 + "value": 0.16866850620027352, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.17013954201238674 + "value": 0.9746196660482375, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.17013954201238674 + "value": 0.9746196660482375, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4676561751411253 + "value": 0.021595547309833023, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.4285211456102784 + "value": 0.0034879406307977735, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.17013954201238674 + "value": 0.9746196660482375, + "path": "" } ] }, - "strict_f1": { - "max": 0.027352941176470587, - "mean": 0.011519506840078677, - "min": 0.001471035812113219, - "n": 5, - "std": 0.013769946337526496 - }, "thresholds": [ 0.0002, 0.0002438417504890822, @@ -14985,951 +15000,951 @@ 0.0002606526104500517, 0.0002883114793803543 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "reference", + "label": "Reference", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.004546429694675032, + "std": 0.0014120999298872481, + "min": 0.0026426577586601383, + "max": 0.006568199246490847 + }, "tolerance_f1": { - "max": 0.2365506772536198, - "mean": 0.13151154173948754, - "min": 0.06246138809231819, "n": 5, - "std": 0.09455648302229831 + "mean": 0.29748392511772237, + "std": 0.012868439357804603, + "min": 0.2756667293290642, + "max": 0.30820726968158313 }, "union_f1": { - "max": 0.4676561751411253, - "mean": 0.2813191893577128, - "min": 0.17013954201238674, "n": 5, - "std": 0.15286646962866243 - } - }, - { + "mean": 0.5906557578801308, + "std": 0.02737158059714251, + "min": 0.5521558725617307, + "max": 0.6239373622157508 + }, "difference": { - "max": 0.6194077454910121, + "n": 5, "mean": 0.5861093281854558, + "std": 0.026945326610355286, "min": 0.5480197595988799, - "n": 5, - "std": 0.026945326610355286 + "max": 0.6194077454910121 }, - "label": "Reference", - "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.021380705009276437, + "n": 5, "mean": 0.016807866419294992, + "std": 0.0036836114267075385, "min": 0.012341125541125542, - "n": 5, - "std": 0.0036836114267075385 + "max": 0.021380705009276437 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5480197595988799 + "value": 0.004136112962850736, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.601348973225457 + "value": 0.006568199246490847, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6194077454910121 + "value": 0.0045296167247386755, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5755105867935725 + "value": 0.004855561780634761, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5862595758183573 + "value": 0.0026426577586601383, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.016084972170686457 + "value": 0.2985122125101588, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.014596289424860853 + "value": 0.29937890424270136, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.019636239950525665 + "value": 0.3056545098251043, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.012341125541125542 + "value": 0.30820726968158313, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.021380705009276437 + "value": 0.2756667293290642, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.004136112962850736 + "value": 0.5521558725617307, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.006568199246490847 + "value": 0.6079171724719479, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0045296167247386755 + "value": 0.6239373622157508, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.004855561780634761 + "value": 0.5803661485742072, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0026426577586601383 + "value": 0.5889022335770174, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.2985122125101588 + "value": 0.5480197595988799, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.29937890424270136 + "value": 0.601348973225457, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.3056545098251043 + "value": 0.6194077454910121, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.30820726968158313 + "value": 0.5755105867935725, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2756667293290642 + "value": 0.5862595758183573, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.5521558725617307 + "value": 0.016084972170686457, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.6079171724719479 + "value": 0.014596289424860853, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6239373622157508 + "value": 0.019636239950525665, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5803661485742072 + "value": 0.012341125541125542, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.5889022335770174 + "value": 0.021380705009276437, + "path": "" } ] }, - "strict_f1": { - "max": 0.006568199246490847, - "mean": 0.004546429694675032, - "min": 0.0026426577586601383, - "n": 5, - "std": 0.0014120999298872481 - }, "thresholds": [ 0.95 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "reference", + "label": "Reference", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.03560431762946002, + "std": 0.008809158358157916, + "min": 0.02462935239291061, + "max": 0.04873596787441818 + }, "tolerance_f1": { - "max": 0.30820726968158313, - "mean": 0.29748392511772237, - "min": 0.2756667293290642, "n": 5, - "std": 0.012868439357804603 + "mean": 0.3926165265132609, + "std": 0.014010934164100928, + "min": 0.3702273155255325, + "max": 0.4063154181985604 }, "union_f1": { - "max": 0.6239373622157508, - "mean": 0.5906557578801308, - "min": 0.5521558725617307, "n": 5, - "std": 0.02737158059714251 - } - }, - { + "mean": 0.7282799455103544, + "std": 0.025784171509335416, + "min": 0.7028883266246737, + "max": 0.7623590438272105 + }, "difference": { - "max": 0.7136230759527924, + "n": 5, "mean": 0.6926756278808944, + "std": 0.0199602553757671, "min": 0.6696707231715443, - "n": 5, - "std": 0.0199602553757671 + "max": 0.7136230759527924 }, - "label": "Reference", - "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.04860987434451371, + "n": 5, "mean": 0.030619372711981795, + "std": 0.010925458415897562, "min": 0.022801028989809045, - "n": 5, - "std": 0.010925458415897562 + "max": 0.04860987434451371 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6696707231715443 + "value": 0.03321760345312939, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7136230759527924 + "value": 0.04873596787441818, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7095179230119556 + "value": 0.038147512864494, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.6744974001959292 + "value": 0.0332911515623479, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6960690170722506 + "value": 0.02462935239291061, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.023750865736618186 + "value": 0.39555976109165375, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.02449292569506283 + "value": 0.40145293653054454, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.033442168793905214 + "value": 0.4063154181985604, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.022801028989809045 + "value": 0.3895272012200133, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.04860987434451371 + "value": 0.3702273155255325, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.03321760345312939 + "value": 0.7028883266246737, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.04873596787441818 + "value": 0.7623590438272105, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.038147512864494 + "value": 0.7476654358764496, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0332911515623479 + "value": 0.707788551758277, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.02462935239291061 + "value": 0.7206983694651612, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.39555976109165375 + "value": 0.6696707231715443, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.40145293653054454 + "value": 0.7136230759527924, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4063154181985604 + "value": 0.7095179230119556, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.3895272012200133 + "value": 0.6744974001959292, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.3702273155255325 + "value": 0.6960690170722506, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.7028883266246737 + "value": 0.023750865736618186, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7623590438272105 + "value": 0.02449292569506283, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7476654358764496 + "value": 0.033442168793905214, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.707788551758277 + "value": 0.022801028989809045, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7206983694651612 + "value": 0.04860987434451371, + "path": "" } ] }, - "strict_f1": { - "max": 0.04873596787441818, - "mean": 0.03560431762946002, - "min": 0.02462935239291061, - "n": 5, - "std": 0.008809158358157916 - }, "thresholds": [ 0.9, 0.95 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "reference", + "label": "Reference", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.03557511803103047, + "std": 0.008798648622457047, + "min": 0.02461746391852102, + "max": 0.0486915291328531 + }, "tolerance_f1": { - "max": 0.4063154181985604, - "mean": 0.3926165265132609, - "min": 0.3702273155255325, "n": 5, - "std": 0.014010934164100928 + "mean": 0.391664739051225, + "std": 0.013905680686091083, + "min": 0.369479820868587, + "max": 0.4052951105632299 }, "union_f1": { - "max": 0.7623590438272105, - "mean": 0.7282799455103544, - "min": 0.7028883266246737, "n": 5, - "std": 0.025784171509335416 - } - }, - { + "mean": 0.7252037371000112, + "std": 0.025669721937652358, + "min": 0.699798626339235, + "max": 0.7590968995963846 + }, "difference": { - "max": 0.7104053704635315, + "n": 5, "mean": 0.6896286190689807, + "std": 0.019887948249560174, "min": 0.6666121634162013, - "n": 5, - "std": 0.019887948249560174 + "max": 0.7104053704635315 }, - "label": "Reference", - "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.024304937172256853, + "n": 5, "mean": 0.015309686355990898, + "std": 0.005462729207948781, "min": 0.011400514494904522, - "n": 5, - "std": 0.005462729207948781 + "max": 0.024304937172256853 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6666121634162013 + "value": 0.033186462923033656, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7104053704635315 + "value": 0.0486915291328531, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7063341380484894 + "value": 0.0381213575591361, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.6715102385713686 + "value": 0.033258776621608484, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6932811848453125 + "value": 0.02461746391852102, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.011875432868309093 + "value": 0.3945515652074689, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.012246462847531414 + "value": 0.4004617740449798, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.016721084396952607 + "value": 0.4052951105632299, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.011400514494904522 + "value": 0.38853542457185924, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.024304937172256853 + "value": 0.369479820868587, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.033186462923033656 + "value": 0.699798626339235, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0486915291328531 + "value": 0.7590968995963846, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0381213575591361 + "value": 0.7444554956076256, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.033258776621608484 + "value": 0.7047690151929771, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.02461746391852102 + "value": 0.7178986487638336, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3945515652074689 + "value": 0.6666121634162013, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.4004617740449798 + "value": 0.7104053704635315, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.4052951105632299 + "value": 0.7063341380484894, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.38853542457185924 + "value": 0.6715102385713686, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.369479820868587 + "value": 0.6932811848453125, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.699798626339235 + "value": 0.011875432868309093, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7590968995963846 + "value": 0.012246462847531414, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7444554956076256 + "value": 0.016721084396952607, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7047690151929771 + "value": 0.011400514494904522, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7178986487638336 + "value": 0.024304937172256853, + "path": "" } ] }, + "thresholds": [ + 0.9, + 0.95 + ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "reference", + "label": "Reference", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, "strict_f1": { - "max": 0.0486915291328531, - "mean": 0.03557511803103047, - "min": 0.02461746391852102, "n": 5, - "std": 0.008798648622457047 + "mean": 0.035300192316067396, + "std": 0.008700007560989128, + "min": 0.024505093215452623, + "max": 0.04827336828783222 }, - "thresholds": [ - 0.9, - 0.95 - ], "tolerance_f1": { - "max": 0.4052951105632299, - "mean": 0.391664739051225, - "min": 0.369479820868587, "n": 5, - "std": 0.013905680686091083 + "mean": 0.3828491649394263, + "std": 0.012952356930876381, + "min": 0.36252633147981417, + "max": 0.39585180444145995 }, "union_f1": { - "max": 0.7590968995963846, - "mean": 0.7252037371000112, - "min": 0.699798626339235, "n": 5, - "std": 0.025669721937652358 - } - }, - { + "mean": 0.6972278451335353, + "std": 0.02466447592239507, + "min": 0.6717469182906158, + "max": 0.7294445251003491 + }, "difference": { - "max": 0.6811711568125168, + "n": 5, "mean": 0.6619276528174679, + "std": 0.019273149109644262, "min": 0.6388534040739275, - "n": 5, - "std": 0.019273149109644262 + "max": 0.6811711568125168 }, - "label": "Reference", - "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.012153370439084725, + "n": 5, "mean": 0.0076554112554112555, + "std": 0.0027315673026463, "min": 0.005700680272108844, - "n": 5, - "std": 0.0027315673026463 + "max": 0.012153370439084725 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6388534040739275 + "value": 0.032893514216688347, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.6811711568125168 + "value": 0.04827336828783222, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.677407206441351 + "value": 0.03787465940054495, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.6443640032843302 + "value": 0.03295432645981885, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6678424934752141 + "value": 0.024505093215452623, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.0059381570810142234 + "value": 0.3852239693382854, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.006123685837971553 + "value": 0.39128423640663973, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.008361162646876932 + "value": 0.39585180444145995, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.005700680272108844 + "value": 0.37935948303093225, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.012153370439084725 + "value": 0.36252633147981417, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.032893514216688347 + "value": 0.6717469182906158, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.04827336828783222 + "value": 0.7294445251003491, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.03787465940054495 + "value": 0.7152818658418959, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.03295432645981885 + "value": 0.6773183297441491, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.024505093215452623 + "value": 0.6923475866906666, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3852239693382854 + "value": 0.6388534040739275, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.39128423640663973 + "value": 0.6811711568125168, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.39585180444145995 + "value": 0.677407206441351, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.37935948303093225 + "value": 0.6443640032843302, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.36252633147981417 + "value": 0.6678424934752141, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.6717469182906158 + "value": 0.0059381570810142234, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7294445251003491 + "value": 0.006123685837971553, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7152818658418959 + "value": 0.008361162646876932, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.6773183297441491 + "value": 0.005700680272108844, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.6923475866906666 + "value": 0.012153370439084725, + "path": "" } ] }, - "strict_f1": { - "max": 0.04827336828783222, - "mean": 0.035300192316067396, - "min": 0.024505093215452623, - "n": 5, - "std": 0.008700007560989128 - }, "thresholds": [ 0.9, 0.95 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "stormcast", + "label": "StormCast", + "scope": "full_domain", + "scope_label": "global", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0006255534463925153, + "std": 0.00011899761411994709, + "min": 0.00043296285460247506, + "max": 0.0007179477702826634 + }, "tolerance_f1": { - "max": 0.39585180444145995, - "mean": 0.3828491649394263, - "min": 0.36252633147981417, "n": 5, - "std": 0.012952356930876381 + "mean": 0.08195132072062675, + "std": 0.021894994384989055, + "min": 0.04396525865017935, + "max": 0.09668169596405224 }, "union_f1": { - "max": 0.7294445251003491, - "mean": 0.6972278451335353, - "min": 0.6717469182906158, "n": 5, - "std": 0.02466447592239507 - } - }, - { + "mean": 0.2238166234389721, + "std": 0.05429431793170512, + "min": 0.12984783035809752, + "max": 0.2609912375807996 + }, "difference": { - "max": 0.26027328981051695, + "n": 5, "mean": 0.2231910699925796, + "std": 0.05417763955331957, "min": 0.12941486750349504, - "n": 5, - "std": 0.05417763955331957 + "max": 0.26027328981051695 }, - "label": "StormCast", - "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.7873692022263451, + "n": 5, "mean": 0.6065068645640074, + "std": 0.17489509665246694, "min": 0.3248534322820037, - "n": 5, - "std": 0.17489509665246694 + "max": 0.7873692022263451 }, - "scope": "full_domain", - "scope_label": "global", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.12941486750349504 + "value": 0.00043296285460247506, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.24941796232451194 + "value": 0.0006714969233599232, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.26027328981051695 + "value": 0.0007179477702826634, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2529130183198567 + "value": 0.0007136363665052921, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.22393621200451744 + "value": 0.0005917233172122231, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.3248534322820037 + "value": 0.04396525865017935, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.7037476808905381 + "value": 0.09264506128556889, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.62891280148423 + "value": 0.09668169596405224, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5876512059369202 + "value": 0.09390595738710937, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.7873692022263451 + "value": 0.08255863031622387, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.00043296285460247506 + "value": 0.12984783035809752, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0006714969233599232 + "value": 0.25008945924787185, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0007179477702826634 + "value": 0.2609912375807996, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.0007136363665052921 + "value": 0.253626654686362, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0005917233172122231 + "value": 0.22452793532172965, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.04396525865017935 + "value": 0.12941486750349504, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.09264506128556889 + "value": 0.24941796232451194, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.09668169596405224 + "value": 0.26027328981051695, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.09390595738710937 + "value": 0.2529130183198567, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.08255863031622387 + "value": 0.22393621200451744, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.12984783035809752 + "value": 0.3248534322820037, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.25008945924787185 + "value": 0.7037476808905381, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2609912375807996 + "value": 0.62891280148423, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.253626654686362 + "value": 0.5876512059369202, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.22452793532172965 + "value": 0.7873692022263451, + "path": "" } ] }, - "strict_f1": { - "max": 0.0007179477702826634, - "mean": 0.0006255534463925153, - "min": 0.00043296285460247506, - "n": 5, - "std": 0.00011899761411994709 - }, "thresholds": [ 0.0002731947461143136, 0.0002782046503853053, @@ -15937,192 +15952,192 @@ 0.0002892915508709848, 0.000301085616229102 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "stormcast", + "label": "StormCast", + "scope": "train_fire_top05pct", + "scope_label": "top 5%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.009572526967992595, + "std": 0.00801147738269429, + "min": 0.005636060100166944, + "max": 0.023876731839958566 + }, "tolerance_f1": { - "max": 0.09668169596405224, - "mean": 0.08195132072062675, - "min": 0.04396525865017935, "n": 5, - "std": 0.021894994384989055 + "mean": 0.15321899635692177, + "std": 0.05533660427697729, + "min": 0.10250537509480662, + "max": 0.23852766585861737 }, "union_f1": { - "max": 0.2609912375807996, - "mean": 0.2238166234389721, - "min": 0.12984783035809752, "n": 5, - "std": 0.05429431793170512 - } - }, - { + "mean": 0.36185682619841214, + "std": 0.09733119467598991, + "min": 0.25743196970586, + "max": 0.49125319073865004 + }, "difference": { - "max": 0.46737645889869145, + "n": 5, "mean": 0.35228429923041954, + "std": 0.09181595284950021, "min": 0.2505963038813504, - "n": 5, - "std": 0.09181595284950021 + "max": 0.46737645889869145 }, - "label": "StormCast", - "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9234194122885129, + "n": 5, "mean": 0.5760166221430691, + "std": 0.3529211324352006, "min": 0.09290590679726922, - "n": 5, - "std": 0.3529211324352006 + "max": 0.9234194122885129 }, - "scope": "train_fire_top05pct", - "scope_label": "top 5%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.46737645889869145 + "value": 0.023876731839958566, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.38914191998978065 + "value": 0.005648734389013078, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2505963038813504 + "value": 0.006835665824509611, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2658654982272957 + "value": 0.005865442686314773, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.38844131515497954 + "value": 0.005636060100166944, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.09290590679726922 + "value": 0.23852766585861737, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9213416444048679 + "value": 0.1601110414083686, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.43306619174829325 + "value": 0.10250537509480662, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.5093499554764025 + "value": 0.10517133941552075, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9234194122885129 + "value": 0.15977956000729543, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.023876731839958566 + "value": 0.49125319073865004, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.005648734389013078 + "value": 0.39479065437879374, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.006835665824509611 + "value": 0.25743196970586, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.005865442686314773 + "value": 0.2717309409136105, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.005636060100166944 + "value": 0.39407737525514647, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.23852766585861737 + "value": 0.46737645889869145, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.1601110414083686 + "value": 0.38914191998978065, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.10250537509480662 + "value": 0.2505963038813504, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.10517133941552075 + "value": 0.2658654982272957, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.15977956000729543 + "value": 0.38844131515497954, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.49125319073865004 + "value": 0.09290590679726922, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.39479065437879374 + "value": 0.9213416444048679, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.25743196970586 + "value": 0.43306619174829325, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2717309409136105 + "value": 0.5093499554764025, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.39407737525514647 + "value": 0.9234194122885129, + "path": "" } ] }, - "strict_f1": { - "max": 0.023876731839958566, - "mean": 0.009572526967992595, - "min": 0.005636060100166944, - "n": 5, - "std": 0.00801147738269429 - }, "thresholds": [ 0.0002716928720474243, 0.00027877578395418823, @@ -16130,192 +16145,192 @@ 0.0002883674460463226, 0.000301085616229102 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "stormcast", + "label": "StormCast", + "scope": "train_fire_top10pct", + "scope_label": "top 10%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.0072843788538849204, + "std": 0.009280155500075412, + "min": 0.002747305898968793, + "max": 0.023870550161812297 + }, "tolerance_f1": { - "max": 0.23852766585861737, - "mean": 0.15321899635692177, - "min": 0.10250537509480662, "n": 5, - "std": 0.05533660427697729 + "mean": 0.12666948350566662, + "std": 0.06328963590343438, + "min": 0.08713204456539597, + "max": 0.23793892088542526 }, "union_f1": { - "max": 0.49125319073865004, - "mean": 0.36185682619841214, - "min": 0.25743196970586, "n": 5, - "std": 0.09733119467598991 - } - }, - { + "mean": 0.30474835114719656, + "std": 0.1065272580858829, + "min": 0.2309636203280157, + "max": 0.4887624633565217 + }, "difference": { - "max": 0.4648919131947094, + "n": 5, "mean": 0.2974639722933117, + "std": 0.09749379807889277, "min": 0.2282163144290469, - "n": 5, - "std": 0.09749379807889277 + "max": 0.4648919131947094 }, - "label": "StormCast", - "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.9575541703769664, + "n": 5, "mean": 0.6807658058771149, + "std": 0.37389945609232794, "min": 0.04645295339863461, - "n": 5, - "std": 0.37389945609232794 + "max": 0.9575541703769664 }, - "scope": "train_fire_top10pct", - "scope_label": "top 10%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4648919131947094 + "value": 0.023870550161812297, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.2284351905540453 + "value": 0.00275028568108307, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2840988865146931 + "value": 0.00356800070444239, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.2816775567740636 + "value": 0.003485751823118051, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2282163144290469 + "value": 0.002747305898968793, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.04645295339863461 + "value": 0.23793892088542526, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.9565152864351439 + "value": 0.08722255267001126, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.7009498367468091 + "value": 0.1112312688602769, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7423567824280202 + "value": 0.10982263054722373, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.9575541703769664 + "value": 0.08713204456539597, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.023870550161812297 + "value": 0.4887624633565217, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.00275028568108307 + "value": 0.23118547623512836, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.00356800070444239 + "value": 0.2876668872191355, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.003485751823118051 + "value": 0.28516330859718164, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.002747305898968793 + "value": 0.2309636203280157, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.23793892088542526 + "value": 0.4648919131947094, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.08722255267001126 + "value": 0.2284351905540453, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.1112312688602769 + "value": 0.2840988865146931, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.10982263054722373 + "value": 0.2816775567740636, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.08713204456539597 + "value": 0.2282163144290469, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4887624633565217 + "value": 0.04645295339863461, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.23118547623512836 + "value": 0.9565152864351439, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.2876668872191355 + "value": 0.7009498367468091, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.28516330859718164 + "value": 0.7423567824280202, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.2309636203280157 + "value": 0.9575541703769664, + "path": "" } ] }, - "strict_f1": { - "max": 0.023870550161812297, - "mean": 0.0072843788538849204, - "min": 0.002747305898968793, - "n": 5, - "std": 0.009280155500075412 - }, "thresholds": [ 0.0002716928720474243, 0.00027818421949632466, @@ -16323,192 +16338,192 @@ 0.0002883674460463226, 0.000301085616229102 ], + "paths": [ + "", + "", + "", + "", + "" + ] + }, + { + "model_tag": "stormcast", + "label": "StormCast", + "scope": "train_fire_top20pct", + "scope_label": "top 20%", + "n": 5, + "strict_f1": { + "n": 5, + "mean": 0.005794988618784165, + "std": 0.009104127010779225, + "min": 0.0014637962799384197, + "max": 0.022074159907300116 + }, "tolerance_f1": { - "max": 0.23793892088542526, - "mean": 0.12666948350566662, - "min": 0.08713204456539597, "n": 5, - "std": 0.06328963590343438 + "mean": 0.10415728840460951, + "std": 0.07343651804127242, + "min": 0.062163327857870446, + "max": 0.23443811051591282 }, "union_f1": { - "max": 0.4887624633565217, - "mean": 0.30474835114719656, - "min": 0.2309636203280157, "n": 5, - "std": 0.1065272580858829 - } - }, - { + "mean": 0.2465979729410411, + "std": 0.12397339055627195, + "min": 0.16937256832950737, + "max": 0.4646063802629719 + }, "difference": { - "max": 0.4425322203556718, + "n": 5, "mean": 0.24080298432225694, + "std": 0.11498838284648957, "min": 0.16790877204956894, - "n": 5, - "std": 0.11498838284648957 + "max": 0.4425322203556718 }, - "label": "StormCast", - "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], "predicted_positive_rate": { - "max": 0.979443413729128, + "n": 5, "mean": 0.6783970315398886, + "std": 0.39241013872037767, "min": 0.020630797773654917, - "n": 5, - "std": 0.39241013872037767 + "max": 0.979443413729128 }, - "scope": "train_fire_top20pct", - "scope_label": "top 20%", "seed_values": { - "difference": [ + "strict_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4425322203556718 + "value": 0.022074159907300116, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.16790877204956894 + "value": 0.0014637962799384197, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.21164756227103257 + "value": 0.0020248273888312522, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.21401759488544256 + "value": 0.001948363237912615, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16790877204956894 + "value": 0.0014637962799384197, + "path": "" } ], - "predicted_positive_rate": [ + "tolerance_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.020630797773654917 + "value": 0.23443811051591282, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.979443413729128 + "value": 0.062163327857870446, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.6907606679035251 + "value": 0.08079366172039935, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.7217068645640075 + "value": 0.08122801407099456, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.979443413729128 + "value": 0.062163327857870446, + "path": "" } ], - "strict_f1": [ + "union_f1": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.022074159907300116 + "value": 0.4646063802629719, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.0014637962799384197 + "value": 0.16937256832950737, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.0020248273888312522 + "value": 0.21367238965986382, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.001948363237912615 + "value": 0.21596595812335517, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.0014637962799384197 + "value": 0.16937256832950737, + "path": "" } ], - "tolerance_f1": [ + "difference": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.23443811051591282 + "value": 0.4425322203556718, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.062163327857870446 + "value": 0.16790877204956894, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.08079366172039935 + "value": 0.21164756227103257, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.08122801407099456 + "value": 0.21401759488544256, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.062163327857870446 + "value": 0.16790877204956894, + "path": "" } ], - "union_f1": [ + "predicted_positive_rate": [ { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 1, - "value": 0.4646063802629719 + "value": 0.020630797773654917, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 7, - "value": 0.16937256832950737 + "value": 0.979443413729128, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 42, - "value": 0.21367238965986382 + "value": 0.6907606679035251, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 99, - "value": 0.21596595812335517 + "value": 0.7217068645640075, + "path": "" }, { - "path": "REDACTED_LOCAL_RESULT_PATH", "seed": 123, - "value": 0.16937256832950737 + "value": 0.979443413729128, + "path": "" } ] }, - "strict_f1": { - "max": 0.022074159907300116, - "mean": 0.005794988618784165, - "min": 0.0014637962799384197, - "n": 5, - "std": 0.009104127010779225 - }, "thresholds": [ 0.00021927931811660528, 0.0002497032401151955, @@ -16516,25 +16531,13 @@ 0.0002812395105138421, 0.00030111230444163084 ], - "tolerance_f1": { - "max": 0.23443811051591282, - "mean": 0.10415728840460951, - "min": 0.062163327857870446, - "n": 5, - "std": 0.07343651804127242 - }, - "union_f1": { - "max": 0.4646063802629719, - "mean": 0.2465979729410411, - "min": 0.16937256832950737, - "n": 5, - "std": 0.12397339055627195 - } + "paths": [ + "", + "", + "", + "", + "" + ] } - ], - "threshold_max_ppr": 0.999999, - "threshold_min_ppr": 1e-06, - "threshold_selector": "strict_f1", - "tolerance_combo": "t0_s8", - "union_combo": "t3_s8" + ] } diff --git a/artifacts/results/selection_regret_all_backbones_20260504.csv b/artifacts/results/selection_regret_all_backbones_20260504.csv deleted file mode 100644 index ea8a02982a8c66171b1344716be24df46cd272c8..0000000000000000000000000000000000000000 --- a/artifacts/results/selection_regret_all_backbones_20260504.csv +++ /dev/null @@ -1,25 +0,0 @@ -model_tag,label,scope,n,seeds,exact_regret_mean,exact_regret_std,tolerated_regret_mean,tolerated_regret_std,union_regret_mean,union_regret_std -reference,Reference,global,5,1 7 42 99 123,0.0,0.0,0.08783024981138902,0.09670495645481135,0.08783024981138902,0.09670495645481135 -reference,Reference,fire_prone,5,1 7 42 99 123,0.0,0.0,0.03402707057672223,0.032044658643147844,0.03402707057672223,0.032044658643147844 -prithvi_wxc,Prithvi-WxC,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -prithvi_wxc,Prithvi-WxC,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -aurora,Aurora,global,5,1 7 42 99 123,0.00020004882767231798,0.00026703384456332115,0.09851983041506818,0.1298781980037557,0.09851983041506818,0.1298781980037557 -aurora,Aurora,fire_prone,5,1 7 42 99 123,0.008202508825959588,0.01834136732088763,0.14391889430974364,0.32121904665016227,0.14391889430974364,0.32121904665016227 -climax,ClimaX,global,5,1 7 42 99 123,3.0287686240700486e-06,4.147312242167625e-06,0.0012959969982639485,0.0017746169760203706,0.0012959969982639485,0.0017746169760203706 -climax,ClimaX,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -stormcast,StormCast,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -stormcast,StormCast,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -pangu_weather,Pangu-Weather,global,5,1 7 42 99 123,0.00013033979247265275,0.0002685372203690466,0.048806713097574374,0.10733308684741971,0.048806713097574374,0.10733308684741971 -pangu_weather,Pangu-Weather,fire_prone,5,1 7 42 99 123,0.027875386332505546,0.02348779386900393,0.43111948243387105,0.39355644251497235,0.43111948243387105,0.39355644251497235 -dlwp,DLWP,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -dlwp,DLWP,fire_prone,5,1 7 42 99 123,0.0007702319787454587,0.0010995336594539604,0.043265915053601556,0.04332331365579739,0.043265915053601556,0.04332331365579739 -fcn,FCN,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -fcn,FCN,fire_prone,5,1 7 42 99 123,5.960229415004348e-06,1.3327478133443526e-05,0.011679805987441694,0.019872372458657642,0.011679805987441694,0.019872372458657642 -fengwu,FengWu,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -fengwu,FengWu,fire_prone,5,1 7 42 99 123,0.0006908222234409067,0.0011910586589384115,0.005222389249812243,0.0062394095558402415,0.005222389249812243,0.0062394095558402415 -fuxi,FuXi,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -fuxi,FuXi,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0010839188523199318,0.0017288780545672386,0.0010839188523199318,0.0017288780545672386 -pangu6,Pangu-Weather,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0 -pangu6,Pangu-Weather,fire_prone,5,1 7 42 99 123,0.0007280423771922354,0.001178746460551365,0.0018491271881979853,0.0032630386057089294,0.0018491271881979853,0.0032630386057089294 -alphaearth,AlphaEarth,global,5,1 7 42 99 123,0.0,0.0,0.1722171037486726,0.08849214830495522,0.1722171037486726,0.08849214830495522 -alphaearth,AlphaEarth,fire_prone,5,1 7 42 99 123,0.0,0.0,0.038803552655092256,0.0594825313313219,0.038803552655092256,0.0594825313313219 diff --git a/artifacts/results/selection_regret_all_backbones_20260504.json b/artifacts/results/selection_regret_all_backbones_20260504.json deleted file mode 100644 index 3fc2ab353adfc8eb74aaa156550b344da6ecd27c..0000000000000000000000000000000000000000 --- a/artifacts/results/selection_regret_all_backbones_20260504.json +++ /dev/null @@ -1,5196 +0,0 @@ -{ - "issues": [], - "notes": [ - "Reference global exact_regret has true zero displayed std; table omitted", - "Reference fire_prone exact_regret has true zero displayed std; table omitted", - "Prithvi-WxC global exact_regret has true zero displayed std; table omitted", - "Prithvi-WxC global tolerated_regret has true zero displayed std; table omitted", - "Prithvi-WxC global union_regret has true zero displayed std; table omitted", - "Prithvi-WxC fire_prone exact_regret has true zero displayed std; table omitted", - "Prithvi-WxC fire_prone tolerated_regret has true zero displayed std; table omitted", - "Prithvi-WxC fire_prone union_regret has true zero displayed std; table omitted", - "ClimaX fire_prone exact_regret has true zero displayed std; table omitted", - "ClimaX fire_prone tolerated_regret has true zero displayed std; table omitted", - "ClimaX fire_prone union_regret has true zero displayed std; table omitted", - "StormCast global exact_regret has true zero displayed std; table omitted", - "StormCast global tolerated_regret has true zero displayed std; table omitted", - "StormCast global union_regret has true zero displayed std; table omitted", - "StormCast fire_prone exact_regret has true zero displayed std; table omitted", - "StormCast fire_prone tolerated_regret has true zero displayed std; table omitted", - "StormCast fire_prone union_regret has true zero displayed std; table omitted", - "DLWP global exact_regret has true zero displayed std; table omitted", - "DLWP global tolerated_regret has true zero displayed std; table omitted", - "DLWP global union_regret has true zero displayed std; table omitted", - "FCN global exact_regret has true zero displayed std; table omitted", - "FCN global tolerated_regret has true zero displayed std; table omitted", - "FCN global union_regret has true zero displayed std; table omitted", - "FengWu global exact_regret has true zero displayed std; table omitted", - "FengWu global tolerated_regret has true zero displayed std; table omitted", - "FengWu global union_regret has true zero displayed std; table omitted", - "FuXi global exact_regret has true zero displayed std; table omitted", - "FuXi global tolerated_regret has true zero displayed std; table omitted", - "FuXi global union_regret has true zero displayed std; table omitted", - "FuXi fire_prone exact_regret has true zero displayed std; table omitted", - "Pangu-Weather global exact_regret has true zero displayed std; table omitted", - "Pangu-Weather global tolerated_regret has true zero displayed std; table omitted", - "Pangu-Weather global union_regret has true zero displayed std; table omitted", - "AlphaEarth global exact_regret has true zero displayed std; table omitted", - "AlphaEarth fire_prone exact_regret has true zero displayed std; table omitted" - ], - "rows": [ - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.004193290734824281", - "exact_ranking_score": "0.004193290734824281", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0056438127090301", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0056438127090301", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0006549130347299629", - "ranking_selected_val_pr_auc": "0.0007883068263073185", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.6337627266658229", - "tolerated_ranking_score": "0.40561891947698747", - "tolerated_regret": 0.22814380718883542, - "tolerated_test_gap": "0.22814380718883542", - "tolerated_val_decision_score": "0.6238973446702573", - "tolerated_val_gap": "0.1715500493543587", - "tolerated_val_ranking_score": "0.45234729531589857", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.6337627266658229", - "union_ranking_score": "0.40561891947698747", - "union_regret": 0.22814380718883542, - "union_test_gap": "0.22814380718883542", - "union_val_decision_score": "0.6238973446702573", - "union_val_gap": "0.1715500493543587", - "union_val_ranking_score": "0.45234729531589857" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.09715762273901808", - "exact_ranking_score": "0.09715762273901808", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.13462469733656174", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.13462469733656174", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.04390614036305261", - "ranking_selected_val_pr_auc": "0.05734457583552234", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7863702028280513", - "tolerated_ranking_score": "0.7863702028280513", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.758785818734687", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.758785818734687", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7863702028280513", - "union_ranking_score": "0.7863702028280513", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.758785818734687", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.758785818734687" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.006959088991986505", - "exact_ranking_score": "0.006959088991986505", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.006399069226294357", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.006399069226294357", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0007047712457371991", - "ranking_selected_val_pr_auc": "0.0007434343102653092", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.5604635792586619", - "tolerated_ranking_score": "0.4427625907752311", - "tolerated_regret": 0.11770098848343075, - "tolerated_test_gap": "0.11770098848343075", - "tolerated_val_decision_score": "0.5441785824390972", - "tolerated_val_gap": "0.056405166058839995", - "tolerated_val_ranking_score": "0.48777341638025723", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.5604635792586619", - "union_ranking_score": "0.4427625907752311", - "union_regret": 0.11770098848343075, - "union_test_gap": "0.11770098848343075", - "union_val_decision_score": "0.5441785824390972", - "union_val_gap": "0.056405166058839995", - "union_val_ranking_score": "0.48777341638025723" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.10531544957774468", - "exact_ranking_score": "0.10531544957774468", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.12267343485617596", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.12267343485617596", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.045809049876129763", - "ranking_selected_val_pr_auc": "0.05470980065861018", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7789089693560928", - "tolerated_ranking_score": "0.7789089693560928", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.749825256630312", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.749825256630312", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7789089693560928", - "union_ranking_score": "0.7789089693560928", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.749825256630312", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.749825256630312" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.004809747755451047", - "exact_ranking_score": "0.004809747755451047", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.004359099620457705", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.004359099620457705", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0005634701573991865", - "ranking_selected_val_pr_auc": "0.0005920938943373892", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.6812131506751973", - "tolerated_ranking_score": "0.4087444681515033", - "tolerated_regret": 0.272468682523694, - "tolerated_test_gap": "0.272468682523694", - "tolerated_val_decision_score": "0.7447325774946227", - "tolerated_val_gap": "0.3021550064919567", - "tolerated_val_ranking_score": "0.44257757100266604", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.6812131506751973", - "union_ranking_score": "0.4087444681515033", - "union_regret": 0.272468682523694, - "union_test_gap": "0.272468682523694", - "union_val_decision_score": "0.7447325774946227", - "union_val_gap": "0.3021550064919567", - "union_val_ranking_score": "0.44257757100266604" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.08702469619756958", - "exact_ranking_score": "0.08702469619756958", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.10515392597717052", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.10515392597717052", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.038083070948941686", - "ranking_selected_val_pr_auc": "0.04505214599515043", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.8461131676361712", - "tolerated_ranking_score": "0.7112901458230849", - "tolerated_regret": 0.13482302181308625, - "tolerated_test_gap": "0.13482302181308625", - "tolerated_val_decision_score": "0.806760850261024", - "tolerated_val_gap": "0.07815518654517495", - "tolerated_val_ranking_score": "0.728605663715849", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.8461131676361712", - "union_ranking_score": "0.7112901458230849", - "union_regret": 0.13482302181308625, - "union_test_gap": "0.13482302181308625", - "union_val_decision_score": "0.806760850261024", - "union_val_gap": "0.07815518654517495", - "union_val_ranking_score": "0.728605663715849" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.010460251046025104", - "exact_ranking_score": "0.010460251046025104", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.009547591069330199", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.009547591069330199", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.001005722733868245", - "ranking_selected_val_pr_auc": "0.0009777524224237678", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.6691395427484861", - "tolerated_ranking_score": "0.6184842128568402", - "tolerated_regret": 0.050655329891645895, - "tolerated_test_gap": "0.050655329891645895", - "tolerated_val_decision_score": "0.677404587086621", - "tolerated_val_gap": "0.011527156971746999", - "tolerated_val_ranking_score": "0.665877430114874", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.6691395427484861", - "union_ranking_score": "0.6184842128568402", - "union_regret": 0.050655329891645895, - "union_test_gap": "0.050655329891645895", - "union_val_decision_score": "0.677404587086621", - "union_val_gap": "0.011527156971746999", - "union_val_ranking_score": "0.665877430114874" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.12514898688915374", - "exact_ranking_score": "0.12514898688915374", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.14325685506435365", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.14325685506435365", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.05955397140893137", - "ranking_selected_val_pr_auc": "0.06058697421048699", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.8294499693616161", - "tolerated_ranking_score": "0.8294499693616161", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.8107581951083846", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.8107581951083846", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.8294499693616161", - "union_ranking_score": "0.8294499693616161", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.8107581951083846", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.8107581951083846" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.006780481898534931", - "exact_ranking_score": "0.006780481898534931", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.005964994898359627", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.005964994898359627", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0006577120081349608", - "ranking_selected_val_pr_auc": "0.0007340310681749519", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.5842714676652996", - "tolerated_ranking_score": "0.3921547570095426", - "tolerated_regret": 0.19211671065575697, - "tolerated_test_gap": "0.19211671065575697", - "tolerated_val_decision_score": "0.5663548190132099", - "tolerated_val_gap": "0.10549666519652506", - "tolerated_val_ranking_score": "0.4608581538166849", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.5842714676652996", - "union_ranking_score": "0.3921547570095426", - "union_regret": 0.19211671065575697, - "union_test_gap": "0.19211671065575697", - "union_val_decision_score": "0.5663548190132099", - "union_val_gap": "0.10549666519652506", - "union_val_ranking_score": "0.4608581538166849" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.10365251727541955", - "exact_ranking_score": "0.10365251727541955", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.12251941328731666", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.12251941328731666", - "family": "AlphaEarth", - "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0458102699699856", - "ranking_selected_val_pr_auc": "0.0554648890254577", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.8350245452333586", - "tolerated_ranking_score": "0.7758298037709835", - "tolerated_regret": 0.05919474146237502, - "tolerated_test_gap": "0.05919474146237502", - "tolerated_val_decision_score": "0.7736506356603177", - "tolerated_val_gap": "0.027586681737492236", - "tolerated_val_ranking_score": "0.7460639539228254", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.8350245452333586", - "union_ranking_score": "0.7758298037709835", - "union_regret": 0.05919474146237502, - "union_test_gap": "0.05919474146237502", - "union_val_decision_score": "0.7736506356603177", - "union_val_gap": "0.027586681737492236", - "union_val_ranking_score": "0.7460639539228254" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.00048497822606044473", - "exact_regret": 7.571921560175121e-06, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.00024254792826221397", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.23755358049655212", - "tolerated_regret": 0.0032399924956598714, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.23755358049655212", - "union_regret": 0.0032399924956598714 - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.0005311562430574265", - "exact_ranking_score": "0.0005311562430574265", - "exact_regret": 0.0, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_pr_auc": "0.00031683315961488916", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.23647112940979162", - "tolerated_regret": 0.004322443582420371, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.23647112940979162", - "union_regret": 0.004322443582420371 - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.04541038665445361", - "exact_ranking_score": "0.04541038665445361", - "exact_regret": 0.0, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_pr_auc": "0.03014151567817997", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7175172112337449", - "tolerated_regret": 0.0010152594893735323, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7175172112337449", - "union_regret": 0.0010152594893735323 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.00048497822606044473", - "exact_regret": 7.571921560175121e-06, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.0002876372030063385", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.240793572992212, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.0", - "union_regret": 0.240793572992212 - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.02613792907867929", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0", - "exact_regret": 0.0004925501476206198, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.00027660331739269843", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.23734400297937533", - "tolerated_regret": 0.0034495700128366613, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.23734400297937533", - "union_regret": 0.0034495700128366613 - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.0413500618483831", - "exact_ranking_score": "0.0413500618483831", - "exact_regret": 0.0, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.024802820904513342", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7184857293868923", - "tolerated_regret": 4.674133622617482e-05, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7184857293868923", - "union_regret": 4.674133622617482e-05 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0", - "exact_regret": 0.0004925501476206198, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.00024254792826221397", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.240793572992212, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.0", - "union_regret": 0.240793572992212 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.0", - "exact_regret": 0.04101254412979794, - "family": "Aurora", - "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.7185324707231184, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.0", - "union_regret": 0.7185324707231184 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.00048497822606044473", - "exact_regret": 7.571921560175121e-06, - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.0002543464414550104", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.23755358049655212", - "tolerated_regret": 0.0032399924956598714, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.23755358049655212", - "union_regret": 0.0032399924956598714 - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.02281272244151735", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00025952340213823634", - "ranking_selected_val_pr_auc": "0.00028511175967657057", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.024930358757410707", - "ranking_selected_val_pr_auc": "0.03159877521821671", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00023723426605001756", - "ranking_selected_val_pr_auc": "0.00023206733426223207", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.021516770872035896", - "ranking_selected_val_pr_auc": "0.023234418719582723", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.00048497822606044473", - "exact_regret": 7.571921560175121e-06, - "exact_test_gap": "7.571921560175121e-06", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "7.267025482891767e-06", - "exact_val_ranking_score": "0.00046544514931712863", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.00025423546642937565", - "ranking_selected_val_pr_auc": "0.00023120715224267713", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.23755358049655212", - "tolerated_regret": 0.0032399924956598714, - "tolerated_test_gap": "0.0032399924956598714", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.00299097714352789", - "tolerated_val_ranking_score": "0.2149459035878704", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.23755358049655212", - "union_regret": 0.0032399924956598714, - "union_test_gap": "0.0032399924956598714", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.00299097714352789", - "union_val_ranking_score": "0.2149459035878704" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.021317405351800135", - "ranking_selected_val_pr_auc": "0.02320587721933335", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.00048696535779102983", - "exact_ranking_score": "0.00048696535779102983", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.00046735226565276225", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.00046735226565276225", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0002340075376021003", - "ranking_selected_val_pr_auc": "0.00023166839498120457", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.2384111045733381", - "tolerated_ranking_score": "0.2384111045733381", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.21573180756095836", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.21573180756095836", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.2384111045733381", - "union_ranking_score": "0.2384111045733381", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.21573180756095836", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.21573180756095836" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "ClimaX", - "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.02099123219536693", - "ranking_selected_val_pr_auc": "0.023142108180759017", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.00487022180273714", - "exact_ranking_score": "0.00487022180273714", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0013194321729398599", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0013194321729398599", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0006257446338466172", - "ranking_selected_val_pr_auc": "0.00041081035639410466", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.38023285660836226", - "tolerated_ranking_score": "0.38023285660836226", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.40408253980815234", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.40408253980815234", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.38023285660836226", - "union_ranking_score": "0.38023285660836226", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.40408253980815234", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.40408253980815234" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.043506471331489265", - "exact_ranking_score": "0.043506471331489265", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04461624715984538", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04461624715984538", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.019747545663020845", - "ranking_selected_val_pr_auc": "0.019740435565559492", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7280364139105968", - "tolerated_ranking_score": "0.7280364139105968", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.7120828721033882", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.7120828721033882", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7280364139105968", - "union_ranking_score": "0.7280364139105968", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.7120828721033882", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.7120828721033882" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0025026814444047193", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0025026814444047193", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0007129763973023342", - "ranking_selected_val_pr_auc": "0.0004370316771032176", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3797689460796109", - "tolerated_ranking_score": "0.3797689460796109", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.4257234764346952", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.4257234764346952", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3797689460796109", - "union_ranking_score": "0.3797689460796109", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.4257234764346952", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.4257234764346952" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04249854057209574", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.001485996442297799, - "exact_test_gap": "0.001485996442297799", - "exact_val_decision_score": "0.04371409485637943", - "exact_val_gap": "0.0011821959321862904", - "exact_val_ranking_score": "0.04253189892419314", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.021118517500793188", - "ranking_selected_val_pr_auc": "0.02303236785318257", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7321459738801157", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.013613503156997275, - "tolerated_test_gap": "0.013613503156997275", - "tolerated_val_decision_score": "0.7128995538153613", - "tolerated_val_gap": "0.022224140467747677", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7321459738801157", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.013613503156997275, - "union_test_gap": "0.013613503156997275", - "union_val_decision_score": "0.7128995538153613", - "union_val_gap": "0.022224140467747677", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0020120724346076456", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0020120724346076456", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0006203713852571992", - "ranking_selected_val_pr_auc": "0.0004707444097300293", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3405125814370199", - "tolerated_ranking_score": "0.3405125814370199", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.474350305376108", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.474350305376108", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3405125814370199", - "union_ranking_score": "0.3405125814370199", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.474350305376108", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.474350305376108" - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.04637177602565815", - "exact_ranking_score": "0.04637177602565815", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.044021164021164016", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.044021164021164016", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.020762153794205103", - "ranking_selected_val_pr_auc": "0.020894537350314525", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7728400679088107", - "tolerated_ranking_score": "0.6606900017471591", - "tolerated_regret": 0.11215006616165157, - "tolerated_test_gap": "0.11215006616165157", - "tolerated_val_decision_score": "0.7122584178150692", - "tolerated_val_gap": "0.009110393702543962", - "tolerated_val_ranking_score": "0.7031480241125252", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7728400679088107", - "union_ranking_score": "0.6606900017471591", - "union_regret": 0.11215006616165157, - "union_test_gap": "0.11215006616165157", - "union_val_decision_score": "0.7122584178150692", - "union_val_gap": "0.009110393702543962", - "union_val_ranking_score": "0.7031480241125252" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0010056989607777405", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0010056989607777405", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0005264872646085452", - "ranking_selected_val_pr_auc": "0.00035284338324857376", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3432315705541329", - "tolerated_ranking_score": "0.3432315705541329", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3469162743670964", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3469162743670964", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3432315705541329", - "union_ranking_score": "0.3432315705541329", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3469162743670964", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3469162743670964" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.043377707581227436", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0023651634514294945, - "exact_test_gap": "0.0023651634514294945", - "exact_val_decision_score": "0.04303681559882578", - "exact_val_gap": "0.0005049166746326425", - "exact_val_ranking_score": "0.04253189892419314", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.018519739310339497", - "ranking_selected_val_pr_auc": "0.01953230111607746", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.762536895087842", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.044004424364723516, - "tolerated_test_gap": "0.044004424364723516", - "tolerated_val_decision_score": "0.7274867592832499", - "tolerated_val_gap": "0.036811345935636286", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.762536895087842", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.044004424364723516, - "union_test_gap": "0.044004424364723516", - "union_val_decision_score": "0.7274867592832499", - "union_val_gap": "0.036811345935636286", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.002074688796680498", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.002074688796680498", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0007477128447471452", - "ranking_selected_val_pr_auc": "0.00047085132311599663", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3979559626836394", - "tolerated_ranking_score": "0.3979559626836394", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.4277982050028173", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.4277982050028173", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3979559626836394", - "union_ranking_score": "0.3979559626836394", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.4277982050028173", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.4277982050028173" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04819843096725701", - "exact_ranking_score": "0.04819843096725701", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.05605272285813389", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.05605272285813389", - "family": "DLWP", - "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02136633936888583", - "ranking_selected_val_pr_auc": "0.024917289620763165", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7653346239087763", - "tolerated_ranking_score": "0.7187730423241409", - "tolerated_regret": 0.046561581584635414, - "tolerated_test_gap": "0.046561581584635414", - "tolerated_val_decision_score": "0.7238142274110074", - "tolerated_val_gap": "0.032313773339499696", - "tolerated_val_ranking_score": "0.6915004540715077", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7653346239087763", - "union_ranking_score": "0.7187730423241409", - "union_regret": 0.046561581584635414, - "union_test_gap": "0.046561581584635414", - "union_val_decision_score": "0.7238142274110074", - "union_val_gap": "0.032313773339499696", - "union_val_ranking_score": "0.6915004540715077" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0009319664492078285", - "exact_ranking_score": "0.0009319664492078285", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.004399472063352399", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.004399472063352399", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00037256097806901117", - "ranking_selected_val_pr_auc": "0.00045069184813668276", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.31167484413093016", - "tolerated_ranking_score": "0.31167484413093016", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3052875742065006", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3052875742065006", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.31167484413093016", - "union_ranking_score": "0.31167484413093016", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3052875742065006", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3052875742065006" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.042068766252528166", - "exact_ranking_score": "0.042068766252528166", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.043150583715087926", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.043150583715087926", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.01844011667219625", - "ranking_selected_val_pr_auc": "0.019762912264716534", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7182175622542595", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "-0.0003149084688589543", - "tolerated_val_decision_score": "0.6906967717983766", - "tolerated_val_gap": "2.135845076300935e-05", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7182175622542595", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "-0.0003149084688589543", - "union_val_decision_score": "0.6906967717983766", - "union_val_gap": "2.135845076300935e-05", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0006535947712418301", - "exact_ranking_score": "0.0006535947712418301", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0037359900373599006", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0037359900373599006", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.000387412312932838", - "ranking_selected_val_pr_auc": "0.0004364526611974742", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3096850885545486", - "tolerated_ranking_score": "0.3096850885545486", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2992790399255972", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2992790399255972", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3096850885545486", - "union_ranking_score": "0.3096850885545486", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2992790399255972", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2992790399255972" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04104234527687296", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 2.980114707502174e-05, - "exact_test_gap": "2.980114707502174e-05", - "exact_val_decision_score": "0.04257237303415807", - "exact_val_gap": "4.047410996493167e-05", - "exact_val_ranking_score": "0.04253189892419314", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.019943278646881796", - "ranking_selected_val_pr_auc": "0.0240285163314118", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7310509974227326", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.012518526699614174, - "tolerated_test_gap": "0.012518526699614174", - "tolerated_val_decision_score": "0.7058203125", - "tolerated_val_gap": "0.01514489915238637", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7310509974227326", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.012518526699614174, - "union_test_gap": "0.012518526699614174", - "union_val_decision_score": "0.7058203125", - "union_val_gap": "0.01514489915238637", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0007027406886858749", - "exact_ranking_score": "0.0007027406886858749", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0031868882312770315", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0031868882312770315", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00041063897933390575", - "ranking_selected_val_pr_auc": "0.00042489429428535383", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.2870596305028149", - "tolerated_ranking_score": "0.31987973649439366", - "tolerated_regret": 0.0, - "tolerated_test_gap": "-0.03282010599157875", - "tolerated_val_decision_score": "0.31159367647460456", - "tolerated_val_gap": "0.0054887421624861266", - "tolerated_val_ranking_score": "0.30610493431211844", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.2870596305028149", - "union_ranking_score": "0.31987973649439366", - "union_regret": 0.0, - "union_test_gap": "-0.03282010599157875", - "union_val_decision_score": "0.31159367647460456", - "union_val_gap": "0.0054887421624861266", - "union_val_ranking_score": "0.30610493431211844" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0414596444738876", - "exact_ranking_score": "0.0414596444738876", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04356664102845841", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04356664102845841", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.018030815062946615", - "ranking_selected_val_pr_auc": "0.018978807537809048", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7197180735022655", - "tolerated_ranking_score": "0.7197180735022655", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6914092197111065", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6914092197111065", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7197180735022655", - "union_ranking_score": "0.7197180735022655", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6914092197111065", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6914092197111065" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.001086071137659517", - "exact_ranking_score": "0.001086071137659517", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.003349460364719018", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.003349460364719018", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0003268363416054406", - "ranking_selected_val_pr_auc": "0.00039156287297214436", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3051941376005135", - "tolerated_ranking_score": "0.3051941376005135", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2948179787588662", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2948179787588662", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3051941376005135", - "union_ranking_score": "0.3051941376005135", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2948179787588662", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2948179787588662" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.040288313241635454", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "-0.0007242308881624873", - "exact_val_decision_score": "0.04306006752022422", - "exact_val_gap": "0.0005281685960310811", - "exact_val_ranking_score": "0.04253189892419314", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.02050876208409485", - "ranking_selected_val_pr_auc": "0.02590116599560572", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7644129739607127", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.045880503237594294, - "tolerated_test_gap": "0.045880503237594294", - "tolerated_val_decision_score": "0.6962583126352054", - "tolerated_val_gap": "0.005582899287591814", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7644129739607127", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.045880503237594294, - "union_test_gap": "0.045880503237594294", - "union_val_decision_score": "0.6962583126352054", - "union_val_gap": "0.005582899287591814", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0018159806295399514", - "exact_ranking_score": "0.0018159806295399514", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.003241491085899514", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.003241491085899514", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00038120453362995967", - "ranking_selected_val_pr_auc": "0.0004356191579614426", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3054145960271247", - "tolerated_ranking_score": "0.3054145960271247", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2930313844339525", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2930313844339525", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3054145960271247", - "union_ranking_score": "0.3054145960271247", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2930313844339525", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2930313844339525" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.042822140550172624", - "exact_ranking_score": "0.042822140550172624", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.043420983295568316", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.043420983295568316", - "family": "FCN", - "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.029098665712304895", - "ranking_selected_val_pr_auc": "0.032221638150467385", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.726408418760773", - "tolerated_ranking_score": "0.726408418760773", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6933831038489982", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6933831038489982", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.726408418760773", - "union_ranking_score": "0.726408418760773", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6933831038489982", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6933831038489982" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0031948881789137383", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0031948881789137383", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.000398077435365184", - "ranking_selected_val_pr_auc": "0.00047223676755013246", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.31071390711162444", - "tolerated_ranking_score": "0.31071390711162444", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3130246199713141", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3130246199713141", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.31071390711162444", - "union_ranking_score": "0.31071390711162444", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3130246199713141", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3130246199713141" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04452825597664091", - "exact_ranking_score": "0.04452825597664091", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0455414882958375", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0455414882958375", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.022736128443885992", - "ranking_selected_val_pr_auc": "0.021286665509494304", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7269980510116651", - "tolerated_ranking_score": "0.7269980510116651", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.7088464801269441", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.7088464801269441", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7269980510116651", - "union_ranking_score": "0.7269980510116651", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.7088464801269441", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.7088464801269441" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.003656307129798903", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.003656307129798903", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0004116035724473925", - "ranking_selected_val_pr_auc": "0.0004377390722357372", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3145618361221859", - "tolerated_ranking_score": "0.3145618361221859", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3129112248106385", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3129112248106385", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3145618361221859", - "union_ranking_score": "0.3145618361221859", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3129112248106385", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3129112248106385" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04171607200603644", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0007035278762385008, - "exact_test_gap": "0.0007035278762385008", - "exact_val_decision_score": "0.043619564341543335", - "exact_val_gap": "0.0010876654173501976", - "exact_val_ranking_score": "0.04253189892419314", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.020495144103834257", - "ranking_selected_val_pr_auc": "0.023246350918909517", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7251252524331628", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.006592781710044404, - "tolerated_test_gap": "0.006592781710044404", - "tolerated_val_decision_score": "0.7015002586652871", - "tolerated_val_gap": "0.01082484531767347", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7251252524331628", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.006592781710044404, - "union_test_gap": "0.006592781710044404", - "union_val_decision_score": "0.7015002586652871", - "union_val_gap": "0.01082484531767347", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.001528350909368791", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.001528350909368791", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00036987379624400454", - "ranking_selected_val_pr_auc": "0.00039852867365630293", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.31219058559732665", - "tolerated_ranking_score": "0.31219058559732665", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.31476872634310843", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.31476872634310843", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.31219058559732665", - "union_ranking_score": "0.31219058559732665", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.31476872634310843", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.31476872634310843" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04574838388861263", - "exact_ranking_score": "0.04574838388861263", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04613821138211382", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04613821138211382", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.021165185967854567", - "ranking_selected_val_pr_auc": "0.020671268187642432", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7265705731122933", - "tolerated_ranking_score": "0.7265705731122933", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.7074814597750377", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.7074814597750377", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7265705731122933", - "union_ranking_score": "0.7265705731122933", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.7074814597750377", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.7074814597750377" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0027012425715829285", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0027012425715829285", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00036963872610239895", - "ranking_selected_val_pr_auc": "0.00042054736055892957", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.30641904273669174", - "tolerated_ranking_score": "0.30641904273669174", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.30868675618673724", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.30868675618673724", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.30641904273669174", - "union_ranking_score": "0.30641904273669174", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.30868675618673724", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.30868675618673724" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.043763127370763974", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0027505832409660327, - "exact_test_gap": "0.0027505832409660327", - "exact_val_decision_score": "0.04404735872043984", - "exact_val_gap": "0.0015154597962467042", - "exact_val_ranking_score": "0.04253189892419314", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.016801706970978273", - "ranking_selected_val_pr_auc": "0.022509307119807316", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.722901134194411", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.004368663471292611, - "tolerated_test_gap": "0.004368663471292611", - "tolerated_val_decision_score": "0.7033268834654516", - "tolerated_val_gap": "0.01265147011783796", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.722901134194411", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.004368663471292611, - "union_test_gap": "0.004368663471292611", - "union_val_decision_score": "0.7033268834654516", - "union_val_gap": "0.01265147011783796", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0041356492969396195", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0041356492969396195", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00042782651526874734", - "ranking_selected_val_pr_auc": "0.0004271990954031648", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3111214819309595", - "tolerated_ranking_score": "0.3111214819309595", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.30674980181224504", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.30674980181224504", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3111214819309595", - "union_ranking_score": "0.3111214819309595", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.30674980181224504", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.30674980181224504" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04882017900732303", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04882017900732303", - "family": "FengWu", - "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02199779031689959", - "ranking_selected_val_pr_auc": "0.030768274290330223", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7336829717908426", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.015150501067724198, - "tolerated_test_gap": "0.015150501067724198", - "tolerated_val_decision_score": "0.6987796297897932", - "tolerated_val_gap": "0.008104216442179624", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7336829717908426", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.015150501067724198, - "union_test_gap": "0.015150501067724198", - "union_val_decision_score": "0.6987796297897932", - "union_val_gap": "0.008104216442179624", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0006866311182961118", - "exact_ranking_score": "0.0006866311182961118", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0008811590083408097", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0008811590083408097", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0003596048414560045", - "ranking_selected_val_pr_auc": "0.0004121169333774252", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.21516044416153002", - "tolerated_ranking_score": "0.3091927111996169", - "tolerated_regret": 0.0, - "tolerated_test_gap": "-0.09403226703808687", - "tolerated_val_decision_score": "0.3118144595732139", - "tolerated_val_gap": "0.001671993688703799", - "tolerated_val_ranking_score": "0.3101424658845101", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.21516044416153002", - "union_ranking_score": "0.3091927111996169", - "union_regret": 0.0, - "union_test_gap": "-0.09403226703808687", - "union_val_decision_score": "0.3118144595732139", - "union_val_gap": "0.001671993688703799", - "union_val_ranking_score": "0.3101424658845101" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.043196160341303", - "exact_ranking_score": "0.043196160341303", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04576053854784421", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04576053854784421", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.01904942261850253", - "ranking_selected_val_pr_auc": "0.020787588115504", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7261195534617713", - "tolerated_ranking_score": "0.7246636456247585", - "tolerated_regret": 0.001455907837012771, - "tolerated_test_gap": "0.001455907837012771", - "tolerated_val_decision_score": "0.7033365957825429", - "tolerated_val_gap": "0.003706992036648815", - "tolerated_val_ranking_score": "0.6996296037458941", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7261195534617713", - "union_ranking_score": "0.7246636456247585", - "union_regret": 0.001455907837012771, - "union_test_gap": "0.001455907837012771", - "union_val_decision_score": "0.7033365957825429", - "union_val_gap": "0.003706992036648815", - "union_val_ranking_score": "0.6996296037458941" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0009888839378897987", - "exact_ranking_score": "0.0009888839378897987", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0010117798317730948", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0010117798317730948", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00039999784430104664", - "ranking_selected_val_pr_auc": "0.00043073606570656576", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3096924811079745", - "tolerated_ranking_score": "0.3096924811079745", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.31039074285584656", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.31039074285584656", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3096924811079745", - "union_ranking_score": "0.3096924811079745", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.31039074285584656", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.31039074285584656" - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.018543768748295608", - "exact_ranking_score": "0.018543768748295608", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04600811907983762", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04600811907983762", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.021045021724179047", - "ranking_selected_val_pr_auc": "0.025380726482200633", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7224961571477053", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0039636864245868875, - "tolerated_test_gap": "0.0039636864245868875", - "tolerated_val_decision_score": "0.6975587357465372", - "tolerated_val_gap": "0.00688332239892353", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7224961571477053", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0039636864245868875, - "union_test_gap": "0.0039636864245868875", - "union_val_decision_score": "0.6975587357465372", - "union_val_gap": "0.00688332239892353", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0003188165529554294", - "exact_ranking_score": "0.0003188165529554294", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0011176098051633573", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0011176098051633573", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0003830459807690152", - "ranking_selected_val_pr_auc": "0.00042901589997107405", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.316668570748256", - "tolerated_ranking_score": "0.316668570748256", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3180456774666437", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3180456774666437", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.316668570748256", - "union_ranking_score": "0.316668570748256", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3180456774666437", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3180456774666437" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04603830266616599", - "exact_ranking_score": "0.04603830266616599", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04428715218026559", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04428715218026559", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.020363596550511454", - "ranking_selected_val_pr_auc": "0.020674507603092498", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7054843984273774", - "tolerated_ranking_score": "0.7054843984273774", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.72648949860566", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.72648949860566", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7054843984273774", - "union_ranking_score": "0.7054843984273774", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.72648949860566", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.72648949860566" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0008450613619815959", - "exact_ranking_score": "0.0008450613619815959", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0009152735719220856", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0009152735719220856", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00037118462783325537", - "ranking_selected_val_pr_auc": "0.0004157684867215522", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3062654921252336", - "tolerated_ranking_score": "0.3062654921252336", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.30677382687191324", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.30677382687191324", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3062654921252336", - "union_ranking_score": "0.3062654921252336", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.30677382687191324", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.30677382687191324" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.044572342126298965", - "exact_ranking_score": "0.044572342126298965", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04500911929271383", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04500911929271383", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.018717347905238046", - "ranking_selected_val_pr_auc": "0.020626878606552698", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7235687421646468", - "tolerated_ranking_score": "0.7235687421646468", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6980281690140845", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6980281690140845", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7235687421646468", - "union_ranking_score": "0.7235687421646468", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6980281690140845", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6980281690140845" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0007771032641515431", - "exact_ranking_score": "0.0007771032641515431", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0008510302291201489", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0008510302291201489", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0003758107890486513", - "ranking_selected_val_pr_auc": "0.0004065107971511574", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3108110703043797", - "tolerated_ranking_score": "0.3108110703043797", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3116979187141017", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3116979187141017", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3108110703043797", - "union_ranking_score": "0.3108110703043797", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3116979187141017", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3116979187141017" - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.030284377692970574", - "exact_ranking_score": "0.030284377692970574", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.051651931130758486", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.051651931130758486", - "family": "FuXi", - "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.02225455497934009", - "ranking_selected_val_pr_auc": "0.02778239852030042", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7183426482806338", - "tolerated_ranking_score": "0.7203102915557309", - "tolerated_regret": 0.0, - "tolerated_test_gap": "-0.0019676432750971307", - "tolerated_val_decision_score": "0.7124343844122053", - "tolerated_val_gap": "0.02087441927176159", - "tolerated_val_ranking_score": "0.6915599651404437", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7183426482806338", - "union_ranking_score": "0.7203102915557309", - "union_regret": 0.0, - "union_test_gap": "-0.0019676432750971307", - "union_val_decision_score": "0.7124343844122053", - "union_val_gap": "0.02087441927176159", - "union_val_ranking_score": "0.6915599651404437" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.00048497822606044473", - "exact_regret": 7.571921560175121e-06, - "exact_test_gap": "7.571921560175121e-06", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "7.267025482891767e-06", - "exact_val_ranking_score": "0.00046544514931712863", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.00024254792826221397", - "ranking_selected_val_pr_auc": "0.00023277674706246134", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.23755358049655212", - "tolerated_regret": 0.0032399924956598714, - "tolerated_test_gap": "0.0032399924956598714", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.00299097714352789", - "tolerated_val_ranking_score": "0.2149459035878704", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.23755358049655212", - "union_regret": 0.0032399924956598714, - "union_test_gap": "0.0032399924956598714", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.00299097714352789", - "union_val_ranking_score": "0.2149459035878704" - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.014813375233921266", - "ranking_selected_val_pr_auc": "0.02263545197493654", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.0005266170481980583", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 3.406690057743849e-05, - "exact_test_gap": "3.406690057743849e-05", - "exact_val_decision_score": "0.00047637431559568154", - "exact_val_gap": "3.6621407956611337e-06", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0003300754798792237", - "ranking_selected_val_pr_auc": "0.00024271959698351982", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "pixel MLP head", - "exact_decision_head_arch": "pixel_mlp", - "exact_decision_score": "0.045738515901060074", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.004725971771262133, - "exact_test_gap": "0.004725971771262133", - "exact_val_decision_score": "0.04344902498318937", - "exact_val_gap": "0.0009171260589962357", - "exact_val_ranking_score": "0.04253189892419314", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.0006100601402256502", - "exact_ranking_score": "0.0", - "exact_regret": 0.0006100601402256502, - "exact_test_gap": "0.0006100601402256502", - "exact_val_decision_score": "0.00048444832931275107", - "exact_val_gap": "0.00048444832931275107", - "exact_val_ranking_score": "0.0", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.00021817709863716954", - "ranking_selected_val_pr_auc": "0.00025116618856577995", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.240793572992212, - "tolerated_test_gap": "0.240793572992212", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.2179368807313983", - "tolerated_val_ranking_score": "0.0", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.0", - "union_regret": 0.240793572992212, - "union_test_gap": "0.240793572992212", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.2179368807313983", - "union_val_ranking_score": "0.0" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04813290893564165", - "exact_ranking_score": "0.0", - "exact_regret": 0.04813290893564165, - "exact_test_gap": "0.04813290893564165", - "exact_val_decision_score": "0.04370736036385822", - "exact_val_gap": "0.04370736036385822", - "exact_val_ranking_score": "0.0", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.02188826028256454", - "ranking_selected_val_pr_auc": "0.032209470774519654", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.7185324707231184, - "tolerated_test_gap": "0.7185324707231184", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.6906754133476136", - "tolerated_val_ranking_score": "0.0", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.0", - "union_regret": 0.7185324707231184, - "union_test_gap": "0.7185324707231184", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.6906754133476136", - "union_val_ranking_score": "0.0" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.00035527516330545857", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "-0.00013727498431516128", - "exact_val_decision_score": "0.0005024040170237669", - "exact_val_gap": "2.9691842223746458e-05", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00028045576992618193", - "ranking_selected_val_pr_auc": "0.00024387937757633073", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.04550550682582603", - "exact_ranking_score": "0.0", - "exact_regret": 0.04550550682582603, - "exact_test_gap": "0.04550550682582603", - "exact_val_decision_score": "0.04319677350880917", - "exact_val_gap": "0.04319677350880917", - "exact_val_ranking_score": "0.0", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.029883397445312192", - "ranking_selected_val_pr_auc": "0.02789152422119005", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.7185324707231184, - "tolerated_test_gap": "0.7185324707231184", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.6906754133476136", - "tolerated_val_ranking_score": "0.0", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.0", - "union_regret": 0.7185324707231184, - "union_test_gap": "0.7185324707231184", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.6906754133476136", - "union_val_ranking_score": "0.0" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004887421693148924", - "exact_ranking_score": "0.0004887421693148924", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004690575308283202", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004690575308283202", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0003408299850116487", - "ranking_selected_val_pr_auc": "0.00024250312441493891", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.229804399271568", - "tolerated_ranking_score": "0.23917716245227552", - "tolerated_regret": 0.0, - "tolerated_test_gap": "-0.00937276318070751", - "tolerated_val_decision_score": "0.2256312133013075", - "tolerated_val_gap": "0.009197267184311658", - "tolerated_val_ranking_score": "0.21643394611699585", - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.229804399271568", - "union_ranking_score": "0.23917716245227552", - "union_regret": 0.0, - "union_test_gap": "-0.00937276318070751", - "union_val_decision_score": "0.2256312133013075", - "union_val_gap": "0.009197267184311658", - "union_val_ranking_score": "0.21643394611699585" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.0", - "exact_regret": 0.04101254412979794, - "exact_test_gap": "0.04101254412979794", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.04253189892419314", - "exact_val_ranking_score": "0.0", - "family": "Pangu-Weather", - "model_tag": "pangu_weather", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.0", - "tolerated_regret": 0.7185324707231184, - "tolerated_test_gap": "0.7185324707231184", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.6906754133476136", - "tolerated_val_ranking_score": "0.0", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.0", - "union_regret": 0.7185324707231184, - "union_test_gap": "0.7185324707231184", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.6906754133476136", - "union_val_ranking_score": "0.0" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.004253509145044662", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.004253509145044662", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00042881385578730365", - "ranking_selected_val_pr_auc": "0.0004901027948814223", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.32066974948800614", - "tolerated_ranking_score": "0.32066974948800614", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3120952399779768", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3120952399779768", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.32066974948800614", - "union_ranking_score": "0.32066974948800614", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3120952399779768", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3120952399779768" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04418397717161179", - "exact_ranking_score": "0.04418397717161179", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04535556523362161", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04535556523362161", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.02328829578760066", - "ranking_selected_val_pr_auc": "0.02010310006243177", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7231388400090598", - "tolerated_ranking_score": "0.7231388400090598", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.698733466652633", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.698733466652633", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7231388400090598", - "union_ranking_score": "0.7231388400090598", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.698733466652633", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.698733466652633" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0042863266180882984", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0042863266180882984", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00043738577276574255", - "ranking_selected_val_pr_auc": "0.0004554894460132563", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.31812983009681495", - "tolerated_ranking_score": "0.31812983009681495", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3113263265858816", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3113263265858816", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.31812983009681495", - "union_ranking_score": "0.31812983009681495", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3113263265858816", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3113263265858816" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04194314737445728", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0009306032446593365, - "exact_test_gap": "0.0009306032446593365", - "exact_val_decision_score": "0.04314380355334118", - "exact_val_gap": "0.0006119046291480426", - "exact_val_ranking_score": "0.04253189892419314", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.02254610440161657", - "ranking_selected_val_pr_auc": "0.026053751384712866", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7261721555350364", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.007639684811918013, - "tolerated_test_gap": "0.007639684811918013", - "tolerated_val_decision_score": "0.7019436364687074", - "tolerated_val_gap": "0.011268223121093768", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7261721555350364", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.007639684811918013, - "union_test_gap": "0.007639684811918013", - "union_val_decision_score": "0.7019436364687074", - "union_val_gap": "0.011268223121093768", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0024802356223841263", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0024802356223841263", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00040651309469793043", - "ranking_selected_val_pr_auc": "0.0004045310999550776", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.32154249424354786", - "tolerated_ranking_score": "0.32154249424354786", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.31489341288382094", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.31489341288382094", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.32154249424354786", - "union_ranking_score": "0.32154249424354786", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.31489341288382094", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.31489341288382094" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0430065972893146", - "exact_ranking_score": "0.0430065972893146", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.044111302847282145", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.044111302847282145", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.021420904714594617", - "ranking_selected_val_pr_auc": "0.019667925605377917", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7241279907754397", - "tolerated_ranking_score": "0.7241279907754397", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6965685921876124", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6965685921876124", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7241279907754397", - "union_ranking_score": "0.7241279907754397", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6965685921876124", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6965685921876124" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.003894080996884735", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.003894080996884735", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.00038395193539280824", - "ranking_selected_val_pr_auc": "0.0005470853500668261", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.31120670593952293", - "tolerated_ranking_score": "0.31120670593952293", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.30691963220059004", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.30691963220059004", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.31120670593952293", - "union_ranking_score": "0.31120670593952293", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.30691963220059004", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.30691963220059004" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04372215277109978", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0027096086413018403, - "exact_test_gap": "0.0027096086413018403", - "exact_val_decision_score": "0.043845404352062356", - "exact_val_gap": "0.0013135054278692188", - "exact_val_ranking_score": "0.04253189892419314", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.018664183428008234", - "ranking_selected_val_pr_auc": "0.024661788918260808", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.719461376864001", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0009289061408825905, - "tolerated_test_gap": "0.0009289061408825905", - "tolerated_val_decision_score": "0.6947253251451462", - "tolerated_val_gap": "0.004049911797532557", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.719461376864001", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0009289061408825905, - "union_test_gap": "0.0009289061408825905", - "union_val_decision_score": "0.6947253251451462", - "union_val_gap": "0.004049911797532557", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0", - "exact_ranking_score": "0.0", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.002408671216378964", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.002408671216378964", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0004450373086921994", - "ranking_selected_val_pr_auc": "0.000447244495874373", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.3214547875801752", - "tolerated_ranking_score": "0.3214547875801752", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.3045294255465288", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.3045294255465288", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.3214547875801752", - "union_ranking_score": "0.3214547875801752", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.3045294255465288", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.3045294255465288" - }, - { - "exact_decision_head": "linear probe", - "exact_decision_head_arch": "linear", - "exact_decision_score": "0.0044004400440044", - "exact_ranking_score": "0.0044004400440044", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.06142857142857142", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.06142857142857142", - "family": "Pangu-Weather", - "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.024740444457605402", - "ranking_selected_val_pr_auc": "0.036676155849720546", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.719015307962107", - "tolerated_ranking_score": "0.7183382629739177", - "tolerated_regret": 0.0006770449881893237, - "tolerated_test_gap": "0.0006770449881893237", - "tolerated_val_decision_score": "0.6913348632305398", - "tolerated_val_gap": "0.0006380914321632014", - "tolerated_val_ranking_score": "0.6906967717983766", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.719015307962107", - "union_ranking_score": "0.7183382629739177", - "union_regret": 0.0006770449881893237, - "union_test_gap": "0.0006770449881893237", - "union_val_decision_score": "0.6913348632305398", - "union_val_gap": "0.0006380914321632014", - "union_val_ranking_score": "0.6906967717983766" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0002463357401629007", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0002463357401629007", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0002463357401629007", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0002463357401629007", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.00048497822606044473", - "exact_ranking_score": "0.00048497822606044473", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0002454330184381399", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.23755358049655212", - "tolerated_ranking_score": "0.23755358049655212", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.23755358049655212", - "union_ranking_score": "0.23755358049655212", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "family": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_pr_auc": "0.02093558282208589", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.017134572294714646", - "exact_ranking_score": "0.017134572294714646", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.002624341503354088", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7286109603326707", - "tolerated_ranking_score": "0.6186566066408326", - "tolerated_regret": 0.10995435369183815, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7286109603326707", - "union_ranking_score": "0.6186566066408326", - "union_regret": 0.10995435369183815 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.1611624834874505", - "exact_ranking_score": "0.1611624834874505", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.10204224118176683", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.8685988450931266", - "tolerated_ranking_score": "0.799032457577039", - "tolerated_regret": 0.06956638751608757, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.8685988450931266", - "union_ranking_score": "0.799032457577039", - "union_regret": 0.06956638751608757 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.019320660641944532", - "exact_ranking_score": "0.019320660641944532", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0025603887793133394", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.735221546471909", - "tolerated_ranking_score": "0.5009707461135111", - "tolerated_regret": 0.23425080035839785, - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.735221546471909", - "union_ranking_score": "0.5009707461135111", - "union_regret": 0.23425080035839785 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.19165378670788252", - "exact_ranking_score": "0.19165378670788252", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.11551882470432043", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "pixel MLP head", - "tolerated_decision_head_arch": "pixel_mlp", - "tolerated_decision_score": "0.8567215417854325", - "tolerated_ranking_score": "0.8066689866810086", - "tolerated_regret": 0.05005255510442386, - "union_decision_head": "pixel MLP head", - "union_decision_head_arch": "pixel_mlp", - "union_decision_score": "0.8567215417854325", - "union_ranking_score": "0.8066689866810086", - "union_regret": 0.05005255510442386 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.025515210991167808", - "exact_ranking_score": "0.025515210991167808", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0032729435045747413", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7239943741463363", - "tolerated_ranking_score": "0.7239943741463363", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7239943741463363", - "union_ranking_score": "0.7239943741463363", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.19225806451612906", - "exact_ranking_score": "0.19225806451612906", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.12048427320762313", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.8358151221553631", - "tolerated_ranking_score": "0.8358151221553631", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.8358151221553631", - "union_ranking_score": "0.8358151221553631", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.02287675150128682", - "exact_ranking_score": "0.02287675150128682", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0030317345997110698", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.5002193417313746", - "tolerated_ranking_score": "0.5002193417313746", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.5002193417313746", - "union_ranking_score": "0.5002193417313746", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.22885572139303484", - "exact_ranking_score": "0.22885572139303484", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.1323902067230726", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.8027807546359551", - "tolerated_ranking_score": "0.8027807546359551", - "tolerated_regret": 0.0, - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.8027807546359551", - "union_ranking_score": "0.8027807546359551", - "union_regret": 0.0 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.021876258220373104", - "exact_ranking_score": "0.021876258220373104", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.0031953098868323193", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.7647466876509988", - "tolerated_ranking_score": "0.6698005926442897", - "tolerated_regret": 0.09494609500670914, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.7647466876509988", - "union_ranking_score": "0.6698005926442897", - "union_regret": 0.09494609500670914 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.19477124183006533", - "exact_ranking_score": "0.19477124183006533", - "exact_regret": 0.0, - "family": "Reference", - "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_pr_auc": "0.11947246238005743", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "linear probe", - "tolerated_decision_head_arch": "linear", - "tolerated_decision_score": "0.9039923296574045", - "tolerated_ranking_score": "0.8534759193943048", - "tolerated_regret": 0.05051641026309972, - "union_decision_head": "linear probe", - "union_decision_head_arch": "linear", - "union_decision_score": "0.9039923296574045", - "union_ranking_score": "0.8534759193943048", - "union_regret": 0.05051641026309972 - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0002463357401629007", - "ranking_selected_val_pr_auc": "0.00023641196480702655", - "scope": "global", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 1, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0002463357401629007", - "ranking_selected_val_pr_auc": "0.00023641196480702655", - "scope": "global", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 123, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0002463357401629007", - "ranking_selected_val_pr_auc": "0.00023641196480702655", - "scope": "global", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 42, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004925501476206198", - "exact_ranking_score": "0.0004925501476206198", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004727121748000204", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004727121748000204", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0002463357401629007", - "ranking_selected_val_pr_auc": "0.00023641196480702655", - "scope": "global", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.240793572992212", - "tolerated_ranking_score": "0.240793572992212", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.2179368807313983", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.2179368807313983", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.240793572992212", - "union_ranking_score": "0.240793572992212", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.2179368807313983", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "linear probe", - "ranking_selected_head_arch": "linear", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 7, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.0004905319945803844", - "exact_ranking_score": "0.0004905319945803844", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.0004707752858483998", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.0004707752858483998", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "shallow spatial adapter", - "ranking_selected_head_arch": "shallow", - "ranking_selected_test_pr_auc": "0.0002463357401629007", - "ranking_selected_val_pr_auc": "0.00023641196480702655", - "scope": "global", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.2399251001974223", - "tolerated_ranking_score": "0.2399251001974223", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.21714067007331003", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.21714067007331003", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.2399251001974223", - "union_ranking_score": "0.2399251001974223", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.21714067007331003", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.21714067007331003" - }, - { - "exact_decision_head": "shallow spatial adapter", - "exact_decision_head_arch": "shallow", - "exact_decision_score": "0.04101254412979794", - "exact_ranking_score": "0.04101254412979794", - "exact_regret": 0.0, - "exact_test_gap": "0.0", - "exact_val_decision_score": "0.04253189892419314", - "exact_val_gap": "0.0", - "exact_val_ranking_score": "0.04253189892419314", - "family": "StormCast", - "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", - "ranking_selected_head": "pixel MLP head", - "ranking_selected_head_arch": "pixel_mlp", - "ranking_selected_test_pr_auc": "0.02093558282208589", - "ranking_selected_val_pr_auc": "0.0217280163599182", - "scope": "fire_prone", - "seed": 99, - "tolerated_decision_head": "shallow spatial adapter", - "tolerated_decision_head_arch": "shallow", - "tolerated_decision_score": "0.7185324707231184", - "tolerated_ranking_score": "0.7185324707231184", - "tolerated_regret": 0.0, - "tolerated_test_gap": "0.0", - "tolerated_val_decision_score": "0.6906754133476136", - "tolerated_val_gap": "0.0", - "tolerated_val_ranking_score": "0.6906754133476136", - "union_decision_head": "shallow spatial adapter", - "union_decision_head_arch": "shallow", - "union_decision_score": "0.7185324707231184", - "union_ranking_score": "0.7185324707231184", - "union_regret": 0.0, - "union_test_gap": "0.0", - "union_val_decision_score": "0.6906754133476136", - "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" - } - ], - "summary": [ - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Reference", - "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.23425080035839785, - "mean": 0.08783024981138902, - "min": 0.0, - "n": 5, - "std": 0.09670495645481135 - }, - "union_regret": { - "max": 0.23425080035839785, - "mean": 0.08783024981138902, - "min": 0.0, - "n": 5, - "std": 0.09670495645481135 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Reference", - "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.06956638751608757, - "mean": 0.03402707057672223, - "min": 0.0, - "n": 5, - "std": 0.032044658643147844 - }, - "union_regret": { - "max": 0.06956638751608757, - "mean": 0.03402707057672223, - "min": 0.0, - "n": 5, - "std": 0.032044658643147844 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Prithvi-WxC", - "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0004925501476206198, - "mean": 0.00020004882767231798, - "min": 0.0, - "n": 5, - "std": 0.00026703384456332115 - }, - "label": "Aurora", - "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.240793572992212, - "mean": 0.09851983041506818, - "min": 0.0032399924956598714, - "n": 5, - "std": 0.1298781980037557 - }, - "union_regret": { - "max": 0.240793572992212, - "mean": 0.09851983041506818, - "min": 0.0032399924956598714, - "n": 5, - "std": 0.1298781980037557 - } - }, - { - "exact_regret": { - "max": 0.04101254412979794, - "mean": 0.008202508825959588, - "min": 0.0, - "n": 5, - "std": 0.01834136732088763 - }, - "label": "Aurora", - "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.7185324707231184, - "mean": 0.14391889430974364, - "min": 0.0, - "n": 5, - "std": 0.32121904665016227 - }, - "union_regret": { - "max": 0.7185324707231184, - "mean": 0.14391889430974364, - "min": 0.0, - "n": 5, - "std": 0.32121904665016227 - } - }, - { - "exact_regret": { - "max": 7.571921560175121e-06, - "mean": 3.0287686240700486e-06, - "min": 0.0, - "n": 5, - "std": 4.147312242167625e-06 - }, - "label": "ClimaX", - "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0032399924956598714, - "mean": 0.0012959969982639485, - "min": 0.0, - "n": 5, - "std": 0.0017746169760203706 - }, - "union_regret": { - "max": 0.0032399924956598714, - "mean": 0.0012959969982639485, - "min": 0.0, - "n": 5, - "std": 0.0017746169760203706 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "ClimaX", - "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "StormCast", - "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "StormCast", - "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0006100601402256502, - "mean": 0.00013033979247265275, - "min": 0.0, - "n": 5, - "std": 0.0002685372203690466 - }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.240793572992212, - "mean": 0.048806713097574374, - "min": 0.0, - "n": 5, - "std": 0.10733308684741971 - }, - "union_regret": { - "max": 0.240793572992212, - "mean": 0.048806713097574374, - "min": 0.0, - "n": 5, - "std": 0.10733308684741971 - } - }, - { - "exact_regret": { - "max": 0.04813290893564165, - "mean": 0.027875386332505546, - "min": 0.0, - "n": 5, - "std": 0.02348779386900393 - }, - "label": "Pangu-Weather", - "model_tag": "pangu_weather", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.7185324707231184, - "mean": 0.43111948243387105, - "min": 0.0, - "n": 5, - "std": 0.39355644251497235 - }, - "union_regret": { - "max": 0.7185324707231184, - "mean": 0.43111948243387105, - "min": 0.0, - "n": 5, - "std": 0.39355644251497235 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "DLWP", - "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0023651634514294945, - "mean": 0.0007702319787454587, - "min": 0.0, - "n": 5, - "std": 0.0010995336594539604 - }, - "label": "DLWP", - "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.11215006616165157, - "mean": 0.043265915053601556, - "min": 0.0, - "n": 5, - "std": 0.04332331365579739 - }, - "union_regret": { - "max": 0.11215006616165157, - "mean": 0.043265915053601556, - "min": 0.0, - "n": 5, - "std": 0.04332331365579739 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FCN", - "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 2.980114707502174e-05, - "mean": 5.960229415004348e-06, - "min": 0.0, - "n": 5, - "std": 1.3327478133443526e-05 - }, - "label": "FCN", - "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.045880503237594294, - "mean": 0.011679805987441694, - "min": 0.0, - "n": 5, - "std": 0.019872372458657642 - }, - "union_regret": { - "max": 0.045880503237594294, - "mean": 0.011679805987441694, - "min": 0.0, - "n": 5, - "std": 0.019872372458657642 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FengWu", - "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0027505832409660327, - "mean": 0.0006908222234409067, - "min": 0.0, - "n": 5, - "std": 0.0011910586589384115 - }, - "label": "FengWu", - "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.015150501067724198, - "mean": 0.005222389249812243, - "min": 0.0, - "n": 5, - "std": 0.0062394095558402415 - }, - "union_regret": { - "max": 0.015150501067724198, - "mean": 0.005222389249812243, - "min": 0.0, - "n": 5, - "std": 0.0062394095558402415 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FuXi", - "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FuXi", - "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0039636864245868875, - "mean": 0.0010839188523199318, - "min": 0.0, - "n": 5, - "std": 0.0017288780545672386 - }, - "union_regret": { - "max": 0.0039636864245868875, - "mean": 0.0010839188523199318, - "min": 0.0, - "n": 5, - "std": 0.0017288780545672386 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Pangu-Weather", - "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "union_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - } - }, - { - "exact_regret": { - "max": 0.0027096086413018403, - "mean": 0.0007280423771922354, - "min": 0.0, - "n": 5, - "std": 0.001178746460551365 - }, - "label": "Pangu-Weather", - "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.007639684811918013, - "mean": 0.0018491271881979853, - "min": 0.0, - "n": 5, - "std": 0.0032630386057089294 - }, - "union_regret": { - "max": 0.007639684811918013, - "mean": 0.0018491271881979853, - "min": 0.0, - "n": 5, - "std": 0.0032630386057089294 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "AlphaEarth", - "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "global", - "scope_label": "\\(\\Omega=\\)global", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.272468682523694, - "mean": 0.1722171037486726, - "min": 0.050655329891645895, - "n": 5, - "std": 0.08849214830495522 - }, - "union_regret": { - "max": 0.272468682523694, - "mean": 0.1722171037486726, - "min": 0.050655329891645895, - "n": 5, - "std": 0.08849214830495522 - } - }, - { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "AlphaEarth", - "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], - "scope": "fire_prone", - "scope_label": "\\(\\Omega=\\)top 20\\%", - "seeds": [ - 1, - 7, - 42, - 99, - 123 - ], - "tolerated_regret": { - "max": 0.13482302181308625, - "mean": 0.038803552655092256, - "min": 0.0, - "n": 5, - "std": 0.0594825313313219 - }, - "union_regret": { - "max": 0.13482302181308625, - "mean": 0.038803552655092256, - "min": 0.0, - "n": 5, - "std": 0.0594825313313219 - } - } - ] -} diff --git a/artifacts/results/selection_regret_scope_sweep_20260505.csv b/artifacts/results/selection_regret_scope_sweep_20260505.csv index b2d3d4305cf14014566b7a7ab80542c3846b3fba..4911b95bc7663f961afe628d381f31e1fb57aa11 100644 --- a/artifacts/results/selection_regret_scope_sweep_20260505.csv +++ b/artifacts/results/selection_regret_scope_sweep_20260505.csv @@ -1,45 +1,45 @@ -model_tag,label,scope,scope_label,n,seeds,exact_regret_mean,exact_regret_std,exact_regret_min,exact_regret_max,tolerated_regret_mean,tolerated_regret_std,tolerated_regret_min,tolerated_regret_max,union_regret_mean,union_regret_std,union_regret_min,union_regret_max -reference,FireWx-FM ref.,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.07383089600948442,0.07453636071372995,0.0,0.17497107865629713,0.07383089600948442,0.07453636071372995,0.0,0.17497107865629713 -reference,FireWx-FM ref.,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.003663718115532055,0.006812231244812292,0.0,0.015676762201120686,0.003663718115532055,0.006812231244812292,0.0,0.015676762201120686 -reference,FireWx-FM ref.,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.012275489592085752,0.012665162001740834,0.0,0.02670880922526031,0.012275489592085752,0.012665162001740834,0.0,0.02670880922526031 -reference,FireWx-FM ref.,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.029384646387840017,0.02751315335001922,0.0,0.05675140203555318,0.029384646387840017,0.02751315335001922,0.0,0.05675140203555318 -prithvi_wxc,Prithvi-WxC,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -prithvi_wxc,Prithvi-WxC,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -prithvi_wxc,Prithvi-WxC,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -prithvi_wxc,Prithvi-WxC,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -aurora,Aurora,global,\(\Omega=\)global,5,1 7 42 99 123,0.00010153879814819402,0.00021861477435572763,0.0,0.0004925501476206198,0.04945471159670635,0.10697394238964528,0.0,0.240793572992212,0.04945471159670635,0.10697394238964528,0.0,0.240793572992212 -aurora,Aurora,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.023667154505540456,0.05292136630837888,0.0,0.11833577252770228,0.1542829840966487,0.34498724021162547,0.0,0.7714149204832434,0.1542829840966487,0.34498724021162547,0.0,0.7714149204832434 -aurora,Aurora,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.014651279478173606,0.032761256870543834,0.0,0.07325639739086803,0.1399343063221699,0.31290262132065055,0.0,0.6996715316108496,0.1399343063221699,0.31290262132065055,0.0,0.6996715316108496 -aurora,Aurora,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.008202508825959588,0.01834136732088763,0.0,0.04101254412979794,0.1437064941446237,0.32133748971555404,0.0,0.7185324707231184,0.1437064941446237,0.32133748971555404,0.0,0.7185324707231184 -climax,ClimaX,global,\(\Omega=\)global,5,1 7 42 99 123,3.0287686240700486e-06,4.147312242167625e-06,0.0,7.571921560175121e-06,0.0012959969982639485,0.0017746169760203706,0.0,0.0032399924956598714,0.0012959969982639485,0.0017746169760203706,0.0,0.0032399924956598714 -climax,ClimaX,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -climax,ClimaX,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -climax,ClimaX,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -stormcast,StormCast,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -stormcast,StormCast,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -stormcast,StormCast,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -stormcast,StormCast,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -dlwp,DLWP,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -dlwp,DLWP,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0048037709215293075,0.006217185202152866,0.0,0.015203005078871956,0.016716228534155796,0.016079313546074458,0.0,0.03305057342744666,0.016716228534155796,0.016079313546074458,0.0,0.03305057342744666 -dlwp,DLWP,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0017281632798742507,0.002514722758075371,0.0,0.005523780499856246,0.02846514801700826,0.026938012702643194,0.0,0.053927677500854476,0.02846514801700826,0.026938012702643194,0.0,0.053927677500854476 -dlwp,DLWP,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0007702319787454587,0.0010995336594539604,0.0,0.0023651634514294945,0.04463354681768479,0.04356064433532197,0.0,0.11215006616165157,0.04463354681768479,0.04356064433532197,0.0,0.11215006616165157 -fcn,FCN,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -fcn,FCN,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0006342898232943345,0.0009899554165032742,0.0,0.002257520679520411,0.004509624980300697,0.010070611656609236,0.0,0.022524473456150496,0.004509624980300697,0.010070611656609236,0.0,0.022524473456150496 -fcn,FCN,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.00021156854817603877,0.0004730816556225618,0.0,0.0010578427408801938,0.004199537050817615,0.009390450319657174,0.0,0.020997685254088072,0.004199537050817615,0.009390450319657174,0.0,0.020997685254088072 -fcn,FCN,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,5.754560074337778e-06,1.2867587506825515e-05,0.0,2.877280037168889e-05,0.011679805987441694,0.019872372458657642,0.0,0.045880503237594294,0.011679805987441694,0.019872372458657642,0.0,0.045880503237594294 -fengwu,FengWu,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -fengwu,FengWu,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0005029843170376968,0.0008109166521114917,0.0,0.0018628094907809783,0.008795951947678215,0.005532321338017505,0.0,0.01484136735033148,0.008795951947678215,0.005532321338017505,0.0,0.01484136735033148 -fengwu,FengWu,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.000495228089292582,0.0007349190216431337,0.0,0.0016387212062008855,0.00402300475984525,0.005510851442075993,0.0,0.010273937098576491,0.00402300475984525,0.005510851442075993,0.0,0.010273937098576491 -fengwu,FengWu,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0006908222234409067,0.0011910586589384115,0.0,0.0027505832409660327,0.005222389249812243,0.0062394095558402415,0.0,0.015150501067724198,0.005222389249812243,0.0062394095558402415,0.0,0.015150501067724198 -fuxi,FuXi,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -fuxi,FuXi,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.002973545331200933,0.0023946274991058026,0.0010927807990139538,0.007024214143542151,0.013545122545609134,0.02097023683418404,0.0,0.050156261654859424,0.013545122545609134,0.02097023683418404,0.0,0.050156261654859424 -fuxi,FuXi,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.001383793743586542,0.0019248128430711165,0.0,0.003938013087198336,0.0016559834970027332,0.0037028916689159307,0.0,0.008279917485013666,0.0016559834970027332,0.0037028916689159307,0.0,0.008279917485013666 -fuxi,FuXi,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.00283318355751887,0.0036808289681375247,0.0,0.008746323525994693,0.00283318355751887,0.0036808289681375247,0.0,0.008746323525994693 -pangu6,Pangu-Weather,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 -pangu6,Pangu-Weather,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.003154674487241463,0.002546125713211599,0.0,0.005711594157251587,0.007592888149777122,0.00897418737588444,0.0,0.019790633919317124,0.007592888149777122,0.00897418737588444,0.0,0.019790633919317124 -pangu6,Pangu-Weather,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0017345627303725214,0.0019305189318827886,0.0,0.004535321555179647,0.003047840737004992,0.005053805614558161,0.0,0.011660780793438352,0.003047840737004992,0.005053805614558161,0.0,0.011660780793438352 -pangu6,Pangu-Weather,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0007280423771922354,0.001178746460551365,0.0,0.0027096086413018403,0.0018679847512695024,0.0032548337047755126,0.0,0.007639684811918013,0.0018679847512695024,0.0032548337047755126,0.0,0.007639684811918013 -alphaearth,AlphaEarth,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.1722171037486726,0.08849214830495522,0.050655329891645895,0.272468682523694,0.1722171037486726,0.08849214830495522,0.050655329891645895,0.272468682523694 -alphaearth,AlphaEarth,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.06384618090125256,0.04965276403138872,0.0,0.1365277562230962,0.06384618090125256,0.04965276403138872,0.0,0.1365277562230962 -alphaearth,AlphaEarth,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.06573776411084173,0.06897015340160571,0.0,0.1615566566666954,0.06573776411084173,0.06897015340160571,0.0,0.1615566566666954 -alphaearth,AlphaEarth,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.038803552655092256,0.0594825313313219,0.0,0.13482302181308625,0.038803552655092256,0.0594825313313219,0.0,0.13482302181308625 +model_tag,label,scope,scope_label,n,seeds,exact_regret_mean,exact_regret_std,exact_regret_min,exact_regret_max,tolerated_regret_mean,tolerated_regret_std,tolerated_regret_min,tolerated_regret_max,union_regret_mean,union_regret_std,union_regret_min,union_regret_max +reference,FireWx-FM ref.,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.07383089600948442,0.07453636071372995,0.0,0.17497107865629713,0.07383089600948442,0.07453636071372995,0.0,0.17497107865629713 +reference,FireWx-FM ref.,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.003663718115532055,0.006812231244812292,0.0,0.015676762201120686,0.003663718115532055,0.006812231244812292,0.0,0.015676762201120686 +reference,FireWx-FM ref.,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.012275489592085752,0.012665162001740834,0.0,0.02670880922526031,0.012275489592085752,0.012665162001740834,0.0,0.02670880922526031 +reference,FireWx-FM ref.,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.029384646387840017,0.02751315335001922,0.0,0.05675140203555318,0.029384646387840017,0.02751315335001922,0.0,0.05675140203555318 +prithvi_wxc,Prithvi-WxC,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +prithvi_wxc,Prithvi-WxC,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +prithvi_wxc,Prithvi-WxC,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +prithvi_wxc,Prithvi-WxC,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +aurora,Aurora,global,\(\Omega=\)global,5,1 7 42 99 123,0.00010153879814819402,0.00021861477435572763,0.0,0.0004925501476206198,0.04945471159670635,0.10697394238964528,0.0,0.240793572992212,0.04945471159670635,0.10697394238964528,0.0,0.240793572992212 +aurora,Aurora,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.023667154505540456,0.05292136630837888,0.0,0.11833577252770228,0.1542829840966487,0.34498724021162547,0.0,0.7714149204832434,0.1542829840966487,0.34498724021162547,0.0,0.7714149204832434 +aurora,Aurora,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.014651279478173606,0.032761256870543834,0.0,0.07325639739086803,0.1399343063221699,0.31290262132065055,0.0,0.6996715316108496,0.1399343063221699,0.31290262132065055,0.0,0.6996715316108496 +aurora,Aurora,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.008202508825959588,0.01834136732088763,0.0,0.04101254412979794,0.1437064941446237,0.32133748971555404,0.0,0.7185324707231184,0.1437064941446237,0.32133748971555404,0.0,0.7185324707231184 +climax,ClimaX,global,\(\Omega=\)global,5,1 7 42 99 123,3.0287686240700486e-06,4.147312242167625e-06,0.0,7.571921560175121e-06,0.0012959969982639485,0.0017746169760203706,0.0,0.0032399924956598714,0.0012959969982639485,0.0017746169760203706,0.0,0.0032399924956598714 +climax,ClimaX,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +climax,ClimaX,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +climax,ClimaX,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +stormcast,StormCast,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +stormcast,StormCast,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +stormcast,StormCast,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +stormcast,StormCast,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +dlwp,DLWP,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +dlwp,DLWP,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0048037709215293075,0.006217185202152866,0.0,0.015203005078871956,0.016716228534155796,0.016079313546074458,0.0,0.03305057342744666,0.016716228534155796,0.016079313546074458,0.0,0.03305057342744666 +dlwp,DLWP,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0017281632798742507,0.002514722758075371,0.0,0.005523780499856246,0.02846514801700826,0.026938012702643194,0.0,0.053927677500854476,0.02846514801700826,0.026938012702643194,0.0,0.053927677500854476 +dlwp,DLWP,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0007702319787454587,0.0010995336594539604,0.0,0.0023651634514294945,0.04463354681768479,0.04356064433532197,0.0,0.11215006616165157,0.04463354681768479,0.04356064433532197,0.0,0.11215006616165157 +fcn,FCN,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +fcn,FCN,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0006342898232943345,0.0009899554165032742,0.0,0.002257520679520411,0.004509624980300697,0.010070611656609236,0.0,0.022524473456150496,0.004509624980300697,0.010070611656609236,0.0,0.022524473456150496 +fcn,FCN,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.00021156854817603877,0.0004730816556225618,0.0,0.0010578427408801938,0.004199537050817615,0.009390450319657174,0.0,0.020997685254088072,0.004199537050817615,0.009390450319657174,0.0,0.020997685254088072 +fcn,FCN,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,5.754560074337778e-06,1.2867587506825515e-05,0.0,2.877280037168889e-05,0.011679805987441694,0.019872372458657642,0.0,0.045880503237594294,0.011679805987441694,0.019872372458657642,0.0,0.045880503237594294 +fengwu,FengWu,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +fengwu,FengWu,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0005029843170376968,0.0008109166521114917,0.0,0.0018628094907809783,0.008795951947678215,0.005532321338017505,0.0,0.01484136735033148,0.008795951947678215,0.005532321338017505,0.0,0.01484136735033148 +fengwu,FengWu,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.000495228089292582,0.0007349190216431337,0.0,0.0016387212062008855,0.00402300475984525,0.005510851442075993,0.0,0.010273937098576491,0.00402300475984525,0.005510851442075993,0.0,0.010273937098576491 +fengwu,FengWu,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0006908222234409067,0.0011910586589384115,0.0,0.0027505832409660327,0.005222389249812243,0.0062394095558402415,0.0,0.015150501067724198,0.005222389249812243,0.0062394095558402415,0.0,0.015150501067724198 +fuxi,FuXi,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +fuxi,FuXi,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.002973545331200933,0.0023946274991058026,0.0010927807990139538,0.007024214143542151,0.013545122545609134,0.02097023683418404,0.0,0.050156261654859424,0.013545122545609134,0.02097023683418404,0.0,0.050156261654859424 +fuxi,FuXi,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.001383793743586542,0.0019248128430711165,0.0,0.003938013087198336,0.0016559834970027332,0.0037028916689159307,0.0,0.008279917485013666,0.0016559834970027332,0.0037028916689159307,0.0,0.008279917485013666 +fuxi,FuXi,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.00283318355751887,0.0036808289681375247,0.0,0.008746323525994693,0.00283318355751887,0.0036808289681375247,0.0,0.008746323525994693 +pangu6,Pangu-Weather,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 +pangu6,Pangu-Weather,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.003154674487241463,0.002546125713211599,0.0,0.005711594157251587,0.007592888149777122,0.00897418737588444,0.0,0.019790633919317124,0.007592888149777122,0.00897418737588444,0.0,0.019790633919317124 +pangu6,Pangu-Weather,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0017345627303725214,0.0019305189318827886,0.0,0.004535321555179647,0.003047840737004992,0.005053805614558161,0.0,0.011660780793438352,0.003047840737004992,0.005053805614558161,0.0,0.011660780793438352 +pangu6,Pangu-Weather,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0007280423771922354,0.001178746460551365,0.0,0.0027096086413018403,0.0018679847512695024,0.0032548337047755126,0.0,0.007639684811918013,0.0018679847512695024,0.0032548337047755126,0.0,0.007639684811918013 +alphaearth,AlphaEarth,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.1722171037486726,0.08849214830495522,0.050655329891645895,0.272468682523694,0.1722171037486726,0.08849214830495522,0.050655329891645895,0.272468682523694 +alphaearth,AlphaEarth,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.06384618090125256,0.04965276403138872,0.0,0.1365277562230962,0.06384618090125256,0.04965276403138872,0.0,0.1365277562230962 +alphaearth,AlphaEarth,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.06573776411084173,0.06897015340160571,0.0,0.1615566566666954,0.06573776411084173,0.06897015340160571,0.0,0.1615566566666954 +alphaearth,AlphaEarth,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.038803552655092256,0.0594825313313219,0.0,0.13482302181308625,0.038803552655092256,0.0594825313313219,0.0,0.13482302181308625 diff --git a/artifacts/results/selection_regret_scope_sweep_20260505.json b/artifacts/results/selection_regret_scope_sweep_20260505.json index 3d57532d393728a04d354cb2a0b5b451e0221690..33665462fbbf84a44d04e58164562e78f83fc063 100644 --- a/artifacts/results/selection_regret_scope_sweep_20260505.json +++ b/artifacts/results/selection_regret_scope_sweep_20260505.json @@ -1,64 +1,4 @@ { - "issues": [], - "notes": [ - "FireWx-FM ref. global exact_regret has true zero displayed std", - "FireWx-FM ref. top5 exact_regret has true zero displayed std", - "FireWx-FM ref. top10 exact_regret has true zero displayed std", - "FireWx-FM ref. top20 exact_regret has true zero displayed std", - "Prithvi-WxC global exact_regret has true zero displayed std", - "Prithvi-WxC global tolerated_regret has true zero displayed std", - "Prithvi-WxC global union_regret has true zero displayed std", - "Prithvi-WxC top5 exact_regret has true zero displayed std", - "Prithvi-WxC top5 tolerated_regret has true zero displayed std", - "Prithvi-WxC top5 union_regret has true zero displayed std", - "Prithvi-WxC top10 exact_regret has true zero displayed std", - "Prithvi-WxC top10 tolerated_regret has true zero displayed std", - "Prithvi-WxC top10 union_regret has true zero displayed std", - "Prithvi-WxC top20 exact_regret has true zero displayed std", - "Prithvi-WxC top20 tolerated_regret has true zero displayed std", - "Prithvi-WxC top20 union_regret has true zero displayed std", - "ClimaX top5 exact_regret has true zero displayed std", - "ClimaX top5 tolerated_regret has true zero displayed std", - "ClimaX top5 union_regret has true zero displayed std", - "ClimaX top10 exact_regret has true zero displayed std", - "ClimaX top10 tolerated_regret has true zero displayed std", - "ClimaX top10 union_regret has true zero displayed std", - "ClimaX top20 exact_regret has true zero displayed std", - "ClimaX top20 tolerated_regret has true zero displayed std", - "ClimaX top20 union_regret has true zero displayed std", - "StormCast global exact_regret has true zero displayed std", - "StormCast global tolerated_regret has true zero displayed std", - "StormCast global union_regret has true zero displayed std", - "StormCast top5 exact_regret has true zero displayed std", - "StormCast top5 tolerated_regret has true zero displayed std", - "StormCast top5 union_regret has true zero displayed std", - "StormCast top10 exact_regret has true zero displayed std", - "StormCast top10 tolerated_regret has true zero displayed std", - "StormCast top10 union_regret has true zero displayed std", - "StormCast top20 exact_regret has true zero displayed std", - "StormCast top20 tolerated_regret has true zero displayed std", - "StormCast top20 union_regret has true zero displayed std", - "DLWP global exact_regret has true zero displayed std", - "DLWP global tolerated_regret has true zero displayed std", - "DLWP global union_regret has true zero displayed std", - "FCN global exact_regret has true zero displayed std", - "FCN global tolerated_regret has true zero displayed std", - "FCN global union_regret has true zero displayed std", - "FengWu global exact_regret has true zero displayed std", - "FengWu global tolerated_regret has true zero displayed std", - "FengWu global union_regret has true zero displayed std", - "FuXi global exact_regret has true zero displayed std", - "FuXi global tolerated_regret has true zero displayed std", - "FuXi global union_regret has true zero displayed std", - "FuXi top20 exact_regret has true zero displayed std", - "Pangu-Weather global exact_regret has true zero displayed std", - "Pangu-Weather global tolerated_regret has true zero displayed std", - "Pangu-Weather global union_regret has true zero displayed std", - "AlphaEarth global exact_regret has true zero displayed std", - "AlphaEarth top5 exact_regret has true zero displayed std", - "AlphaEarth top10 exact_regret has true zero displayed std", - "AlphaEarth top20 exact_regret has true zero displayed std" - ], "rows": [ { "exact_decision_head": "shallow spatial adapter", @@ -72,7 +12,6 @@ "exact_val_ranking_score": "0.0056438127090301", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0006549130347299629", @@ -96,7 +35,8 @@ "union_test_gap": "0.22814380718883542", "union_val_decision_score": "0.6238973446702573", "union_val_gap": "0.1715500493543587", - "union_val_ranking_score": "0.45234729531589857" + "union_val_ranking_score": "0.45234729531589857", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -110,7 +50,6 @@ "exact_val_ranking_score": "0.24264049955396969", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.13521999470870394", @@ -134,7 +73,8 @@ "union_test_gap": "0.06535707609592145", "union_val_decision_score": "0.8921006948278831", "union_val_gap": "0.06576085909428331", - "union_val_ranking_score": "0.8263398357335998" + "union_val_ranking_score": "0.8263398357335998", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -148,7 +88,6 @@ "exact_val_ranking_score": "0.18440523619806493", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.08541592232377082", @@ -172,7 +111,8 @@ "union_test_gap": "0.06671878280938992", "union_val_decision_score": "0.8034698049616", "union_val_gap": "0.04257704982233934", - "union_val_ranking_score": "0.7608927551392607" + "union_val_ranking_score": "0.7608927551392607", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -186,7 +126,6 @@ "exact_val_ranking_score": "0.13462469733656174", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.04390614036305261", @@ -210,7 +149,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.758785818734687", "union_val_gap": "0.0", - "union_val_ranking_score": "0.758785818734687" + "union_val_ranking_score": "0.758785818734687", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -224,7 +164,6 @@ "exact_val_ranking_score": "0.006399069226294357", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0007047712457371991", @@ -248,7 +187,8 @@ "union_test_gap": "0.11770098848343075", "union_val_decision_score": "0.5441785824390972", "union_val_gap": "0.056405166058839995", - "union_val_ranking_score": "0.48777341638025723" + "union_val_ranking_score": "0.48777341638025723", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -262,7 +202,6 @@ "exact_val_ranking_score": "0.2180376610505451", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.13463750657454468", @@ -286,7 +225,8 @@ "union_test_gap": "0.04372586180889426", "union_val_decision_score": "0.8625937780701644", "union_val_gap": "0.05250190173735958", - "union_val_ranking_score": "0.8100918763328048" + "union_val_ranking_score": "0.8100918763328048", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -300,7 +240,6 @@ "exact_val_ranking_score": "0.1708413615928067", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0877757525096481", @@ -324,7 +263,8 @@ "union_test_gap": "-0.06879477717769467", "union_val_decision_score": "0.7559010025052879", "union_val_gap": "0.0006607651392849245", - "union_val_ranking_score": "0.755240237366003" + "union_val_ranking_score": "0.755240237366003", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -338,7 +278,6 @@ "exact_val_ranking_score": "0.12267343485617596", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.045809049876129763", @@ -362,7 +301,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.749825256630312", "union_val_gap": "0.0", - "union_val_ranking_score": "0.749825256630312" + "union_val_ranking_score": "0.749825256630312", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -376,7 +316,6 @@ "exact_val_ranking_score": "0.004359099620457705", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0005634701573991865", @@ -400,7 +339,8 @@ "union_test_gap": "0.272468682523694", "union_val_decision_score": "0.7447325774946227", "union_val_gap": "0.3021550064919567", - "union_val_ranking_score": "0.44257757100266604" + "union_val_ranking_score": "0.44257757100266604", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -414,7 +354,6 @@ "exact_val_ranking_score": "0.20273972602739726", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.11776218604745192", @@ -438,7 +377,8 @@ "union_test_gap": "0.1365277562230962", "union_val_decision_score": "0.8560076563904758", "union_val_gap": "0.058814205003907216", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -452,7 +392,6 @@ "exact_val_ranking_score": "0.1536748329621381", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.07431134992726657", @@ -476,7 +415,8 @@ "union_test_gap": "0.1615566566666954", "union_val_decision_score": "0.8130323993938946", "union_val_gap": "0.0827924722936264", - "union_val_ranking_score": "0.7302399271002682" + "union_val_ranking_score": "0.7302399271002682", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -490,7 +430,6 @@ "exact_val_ranking_score": "0.10515392597717052", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.038083070948941686", @@ -514,7 +453,8 @@ "union_test_gap": "0.13482302181308625", "union_val_decision_score": "0.806760850261024", "union_val_gap": "0.07815518654517495", - "union_val_ranking_score": "0.728605663715849" + "union_val_ranking_score": "0.728605663715849", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -528,7 +468,6 @@ "exact_val_ranking_score": "0.009547591069330199", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.001005722733868245", @@ -552,7 +491,8 @@ "union_test_gap": "0.050655329891645895", "union_val_decision_score": "0.677404587086621", "union_val_gap": "0.011527156971746999", - "union_val_ranking_score": "0.665877430114874" + "union_val_ranking_score": "0.665877430114874", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -566,7 +506,6 @@ "exact_val_ranking_score": "0.22265122265122267", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.1415020173172751", @@ -590,7 +529,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8765714285714286", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8765714285714286" + "union_val_ranking_score": "0.8765714285714286", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -604,7 +544,6 @@ "exact_val_ranking_score": "0.17872968980797638", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0993207020173454", @@ -628,7 +567,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8241729549074388", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8241729549074388" + "union_val_ranking_score": "0.8241729549074388", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -642,7 +582,6 @@ "exact_val_ranking_score": "0.14325685506435365", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.05955397140893137", @@ -666,7 +605,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8107581951083846", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8107581951083846" + "union_val_ranking_score": "0.8107581951083846", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -680,7 +620,6 @@ "exact_val_ranking_score": "0.005964994898359627", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0006577120081349608", @@ -704,7 +643,8 @@ "union_test_gap": "0.19211671065575697", "union_val_decision_score": "0.5663548190132099", "union_val_gap": "0.10549666519652506", - "union_val_ranking_score": "0.4608581538166849" + "union_val_ranking_score": "0.4608581538166849", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -718,7 +658,6 @@ "exact_val_ranking_score": "0.2253944402704733", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.13302539422913043", @@ -742,7 +681,8 @@ "union_test_gap": "0.07362021037835087", "union_val_decision_score": "0.8737527215993107", "union_val_gap": "0.06839730093782548", - "union_val_ranking_score": "0.8053554206614852" + "union_val_ranking_score": "0.8053554206614852", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -756,7 +696,6 @@ "exact_val_ranking_score": "0.16870573084352863", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.08656943374833374", @@ -780,7 +719,8 @@ "union_test_gap": "0.10041338107812336", "union_val_decision_score": "0.8092827685342129", "union_val_gap": "0.06382807430932869", - "union_val_ranking_score": "0.7454546942248842" + "union_val_ranking_score": "0.7454546942248842", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -794,7 +734,6 @@ "exact_val_ranking_score": "0.12251941328731666", "family": "AlphaEarth", "model_tag": "alphaearth", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0458102699699856", @@ -818,7 +757,8 @@ "union_test_gap": "0.05919474146237502", "union_val_decision_score": "0.7736506356603177", "union_val_gap": "0.027586681737492236", - "union_val_ranking_score": "0.7460639539228254" + "union_val_ranking_score": "0.7460639539228254", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -832,7 +772,6 @@ "exact_val_ranking_score": "0.00046544514931712863", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.00017433024575177084", @@ -856,7 +795,8 @@ "union_test_gap": "0.0032399924956598714", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.00299097714352789", - "union_val_ranking_score": "0.2149459035878704" + "union_val_ranking_score": "0.2149459035878704", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -870,7 +810,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.043017434088706286", @@ -894,7 +833,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -908,7 +848,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.025652132600196966", @@ -932,7 +871,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -946,7 +886,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.014185996780547955", @@ -970,7 +909,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -984,7 +924,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0001870694786133523", @@ -1008,7 +947,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1022,7 +962,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0476637592736729", @@ -1046,7 +985,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1060,7 +1000,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.028202726028592855", @@ -1084,7 +1023,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1098,7 +1038,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.015530707998309533", @@ -1122,7 +1061,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1136,7 +1076,6 @@ "exact_val_ranking_score": "0.0", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.000198182392518492", @@ -1160,7 +1099,8 @@ "union_test_gap": "0.240793572992212", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.2179368807313983", - "union_val_ranking_score": "0.0" + "union_val_ranking_score": "0.0", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1174,7 +1114,6 @@ "exact_val_ranking_score": "0.0", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.047074707804185205", @@ -1198,7 +1137,8 @@ "union_test_gap": "0.7714149204832434", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.7971934513865686", - "union_val_ranking_score": "0.0" + "union_val_ranking_score": "0.0", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1212,7 +1152,6 @@ "exact_val_ranking_score": "0.0", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.027753810085825968", @@ -1236,7 +1175,8 @@ "union_test_gap": "0.6996715316108496", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.7015182768065465", - "union_val_ranking_score": "0.0" + "union_val_ranking_score": "0.0", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1250,7 +1190,6 @@ "exact_val_ranking_score": "0.0", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.015401540261000917", @@ -1274,7 +1213,8 @@ "union_test_gap": "0.7185324707231184", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.6906754133476136", - "union_val_ranking_score": "0.0" + "union_val_ranking_score": "0.0", + "path": "" }, { "exact_decision_head": "linear probe", @@ -1288,7 +1228,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002120945228028488", @@ -1312,7 +1251,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1326,7 +1266,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.05983564956422643", @@ -1350,7 +1289,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1364,7 +1304,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03479712897675945", @@ -1388,7 +1327,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1402,7 +1342,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.01908165402331067", @@ -1426,7 +1365,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1440,7 +1380,6 @@ "exact_val_ranking_score": "0.00046544514931712863", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.00016818557842652088", @@ -1464,7 +1403,8 @@ "union_test_gap": "0.0032399924956598714", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0025778293487095094", - "union_val_ranking_score": "0.2153590513826888" + "union_val_ranking_score": "0.2153590513826888", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -1478,7 +1418,6 @@ "exact_val_ranking_score": "0.1250696689332293", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.04280276014558355", @@ -1502,7 +1441,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7985410856039477", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7985410856039477" + "union_val_ranking_score": "0.7985410856039477", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -1516,7 +1456,6 @@ "exact_val_ranking_score": "0.07801114444920705", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.025614133200453124", @@ -1540,7 +1479,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7022486149835246", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7022486149835246" + "union_val_ranking_score": "0.7022486149835246", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -1554,7 +1494,6 @@ "exact_val_ranking_score": "0.04437663936620095", "family": "Aurora", "model_tag": "aurora", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.01412932353396452", @@ -1578,7 +1517,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.692216769941617", "union_val_gap": "0.0", - "union_val_ranking_score": "0.692216769941617" + "union_val_ranking_score": "0.692216769941617", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1592,7 +1532,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00023234964096770515", @@ -1616,7 +1555,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1630,7 +1570,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.055890306674778144", @@ -1654,7 +1593,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1668,7 +1608,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03345772284683149", @@ -1692,7 +1631,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1706,7 +1646,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.01871363925580095", @@ -1730,7 +1669,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1744,7 +1684,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00023488826232608934", @@ -1768,7 +1707,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1782,7 +1722,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.06647102886669706", @@ -1806,7 +1745,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1820,7 +1760,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03955291143531272", @@ -1844,7 +1783,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1858,7 +1798,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.021395640000026753", @@ -1882,7 +1821,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -1896,7 +1836,6 @@ "exact_val_ranking_score": "0.00046544514931712863", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.00023439386477689915", @@ -1920,7 +1859,8 @@ "union_test_gap": "0.0032399924956598714", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.00299097714352789", - "union_val_ranking_score": "0.2149459035878704" + "union_val_ranking_score": "0.2149459035878704", + "path": "" }, { "exact_decision_head": "linear probe", @@ -1934,7 +1874,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06646486382553912", @@ -1958,7 +1897,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "linear probe", @@ -1972,7 +1912,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.039777194719281345", @@ -1996,7 +1935,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2010,7 +1950,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.021516770872035896", @@ -2034,7 +1973,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2048,7 +1988,6 @@ "exact_val_ranking_score": "0.00046544514931712863", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.00025423546642937565", @@ -2072,7 +2011,8 @@ "union_test_gap": "0.0032399924956598714", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.00299097714352789", - "union_val_ranking_score": "0.2149459035878704" + "union_val_ranking_score": "0.2149459035878704", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2086,7 +2026,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06582645251185222", @@ -2110,7 +2049,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2124,7 +2064,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03938368107690723", @@ -2148,7 +2087,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2162,7 +2102,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.021317405351800135", @@ -2186,7 +2125,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2200,7 +2140,6 @@ "exact_val_ranking_score": "0.00046735226565276225", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002540711776626832", @@ -2224,7 +2163,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.21573180756095836", "union_val_gap": "0.0", - "union_val_ranking_score": "0.21573180756095836" + "union_val_ranking_score": "0.21573180756095836", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2238,7 +2178,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06558789102558008", @@ -2262,7 +2201,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2276,7 +2216,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.039026227487447884", @@ -2300,7 +2239,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2314,7 +2254,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "ClimaX", "model_tag": "climax", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.021159149554047907", @@ -2338,7 +2277,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2352,7 +2292,6 @@ "exact_val_ranking_score": "0.0016902678135446794", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0006824870760177812", @@ -2376,7 +2315,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3983991584038598", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3983991584038598" + "union_val_ranking_score": "0.3983991584038598", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2390,7 +2330,6 @@ "exact_val_ranking_score": "0.1225081551286698", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.05689342648649434", @@ -2414,7 +2353,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8195894738438443", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8195894738438443" + "union_val_ranking_score": "0.8195894738438443", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2428,7 +2368,6 @@ "exact_val_ranking_score": "0.07851948205522379", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03678683052582872", @@ -2452,7 +2391,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7278456297612906", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7278456297612906" + "union_val_ranking_score": "0.7278456297612906", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2466,7 +2406,6 @@ "exact_val_ranking_score": "0.04461624715984538", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.019747545663020845", @@ -2490,7 +2429,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7120828721033882", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7120828721033882" + "union_val_ranking_score": "0.7120828721033882", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2504,7 +2444,6 @@ "exact_val_ranking_score": "0.0020376974019358125", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0007448529317390826", @@ -2528,7 +2467,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.4280727908627396", "union_val_gap": "0.0", - "union_val_ranking_score": "0.4280727908627396" + "union_val_ranking_score": "0.4280727908627396", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2542,7 +2482,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.06589923575225648", @@ -2566,7 +2505,8 @@ "union_test_gap": "0.01968276007752523", "union_val_decision_score": "0.8156849318846232", "union_val_gap": "0.018491480498054602", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2580,7 +2520,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03734242132263064", @@ -2604,7 +2543,8 @@ "union_test_gap": "0.03587298070772449", "union_val_decision_score": "0.7311417346046021", "union_val_gap": "0.02962345779805553", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2618,7 +2558,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.021118517500793188", @@ -2642,7 +2581,8 @@ "union_test_gap": "0.013613503156997275", "union_val_decision_score": "0.7128995538153613", "union_val_gap": "0.022224140467747677", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2656,7 +2596,6 @@ "exact_val_ranking_score": "0.0020120724346076456", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0006203713852571992", @@ -2680,7 +2619,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.474350305376108", "union_val_gap": "0.0", - "union_val_ranking_score": "0.474350305376108" + "union_val_ranking_score": "0.474350305376108", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2694,7 +2634,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.058186924071015014", @@ -2718,7 +2657,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8131649530621368", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8131649530621368" + "union_val_ranking_score": "0.8131649530621368", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -2732,7 +2672,6 @@ "exact_val_ranking_score": "0.07607950651130911", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03845833667153789", @@ -2756,7 +2695,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7265137957010752", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7265137957010752" + "union_val_ranking_score": "0.7265137957010752", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -2770,7 +2710,6 @@ "exact_val_ranking_score": "0.044021164021164016", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.020762153794205103", @@ -2794,7 +2733,8 @@ "union_test_gap": "0.11215006616165157", "union_val_decision_score": "0.7122584178150692", "union_val_gap": "0.009110393702543962", - "union_val_ranking_score": "0.7031480241125252" + "union_val_ranking_score": "0.7031480241125252", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2808,7 +2748,6 @@ "exact_val_ranking_score": "0.0010056989607777405", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0005264872646085452", @@ -2832,7 +2771,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3469162743670964", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3469162743670964" + "union_val_ranking_score": "0.3469162743670964", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2846,7 +2786,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.05890031091946385", @@ -2870,7 +2809,8 @@ "union_test_gap": "0.03305057342744666", "union_val_decision_score": "0.8071267256214067", "union_val_gap": "0.009933274234838141", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2884,7 +2824,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03308736705588518", @@ -2908,7 +2847,8 @@ "union_test_gap": "0.05252508187646232", "union_val_decision_score": "0.7227897891452092", "union_val_gap": "0.021271512338662646", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2922,7 +2862,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.018519739310339497", @@ -2946,7 +2885,8 @@ "union_test_gap": "0.044004424364723516", "union_val_decision_score": "0.7274867592832499", "union_val_gap": "0.036811345935636286", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -2960,7 +2900,6 @@ "exact_val_ranking_score": "0.0016250253910217347", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0007269780629045868", @@ -2984,7 +2923,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.41347824096521096", "union_val_gap": "0.0", - "union_val_ranking_score": "0.41347824096521096" + "union_val_ranking_score": "0.41347824096521096", + "path": "" }, { "exact_decision_head": "linear probe", @@ -2998,7 +2938,6 @@ "exact_val_ranking_score": "0.15619694397283532", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.0672622238447139", @@ -3022,7 +2961,8 @@ "union_test_gap": "0.030847809165807094", "union_val_decision_score": "0.8078282610569877", "union_val_gap": "0.010634809670419076", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "linear probe", @@ -3036,7 +2976,6 @@ "exact_val_ranking_score": "0.09763863093658795", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.037689588394968596", @@ -3060,7 +2999,8 @@ "union_test_gap": "0.053927677500854476", "union_val_decision_score": "0.7243805445090241", "union_val_gap": "0.02259882240347566", - "union_val_ranking_score": "0.7017817221055485" + "union_val_ranking_score": "0.7017817221055485", + "path": "" }, { "exact_decision_head": "linear probe", @@ -3074,7 +3014,6 @@ "exact_val_ranking_score": "0.05605272285813389", "family": "DLWP", "model_tag": "dlwp", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02136633936888583", @@ -3098,7 +3037,8 @@ "union_test_gap": "0.05339974040505158", "union_val_decision_score": "0.7238375070567806", "union_val_gap": "0.032337052985272896", - "union_val_ranking_score": "0.6915004540715077" + "union_val_ranking_score": "0.6915004540715077", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3112,7 +3052,6 @@ "exact_val_ranking_score": "0.004184100418410042", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0003658682156259175", @@ -3136,7 +3075,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3016823150717557", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3016823150717557" + "union_val_ranking_score": "0.3016823150717557", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3150,7 +3090,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.0689177522952707", @@ -3174,7 +3113,8 @@ "union_test_gap": "2.365144535299013e-05", "union_val_decision_score": "0.7976559342583861", "union_val_gap": "0.00046248287181749337", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3188,7 +3128,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03927901160149114", @@ -3212,7 +3151,8 @@ "union_test_gap": "-0.024036806500324803", "union_val_decision_score": "0.7162675437027928", "union_val_gap": "0.014749266896246227", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3226,7 +3166,6 @@ "exact_val_ranking_score": "0.04316055094101566", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.017947733209238474", @@ -3250,7 +3189,8 @@ "union_test_gap": "-0.0003149084688589543", "union_val_decision_score": "0.6906967717983766", "union_val_gap": "2.135845076300935e-05", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3264,7 +3204,6 @@ "exact_val_ranking_score": "0.0034982508745627187", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0004059591718395402", @@ -3288,7 +3227,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.30078743758851223", "union_val_gap": "0.0", - "union_val_ranking_score": "0.30078743758851223" + "union_val_ranking_score": "0.30078743758851223", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -3302,7 +3242,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.058898489734440555", @@ -3326,7 +3265,8 @@ "union_test_gap": "0.022524473456150496", "union_val_decision_score": "0.8166754936823787", "union_val_gap": "0.019482042295810076", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -3340,7 +3280,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03552513494863446", @@ -3364,7 +3303,8 @@ "union_test_gap": "0.020997685254088072", "union_val_decision_score": "0.7230799450009822", "union_val_gap": "0.021561668194435613", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3378,7 +3318,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.01989120112622579", @@ -3402,7 +3341,8 @@ "union_test_gap": "0.012518526699614174", "union_val_decision_score": "0.7058203125", "union_val_gap": "0.01514489915238637", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3416,7 +3356,6 @@ "exact_val_ranking_score": "0.0030852294639413806", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00036191656354256076", @@ -3440,7 +3379,8 @@ "union_test_gap": "-0.027846200639709717", "union_val_decision_score": "0.31159367647460456", "union_val_gap": "0.009129115930882226", - "union_val_ranking_score": "0.30246456054372234" + "union_val_ranking_score": "0.30246456054372234", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3454,7 +3394,6 @@ "exact_val_ranking_score": "0.11731255265374893", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.058729135567964254", @@ -3478,7 +3417,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7976559342583861", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7976559342583861" + "union_val_ranking_score": "0.7976559342583861", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3492,7 +3432,6 @@ "exact_val_ranking_score": "0.07542516569901865", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.031759062375776324", @@ -3516,7 +3455,8 @@ "union_test_gap": "-0.20878833679978748", "union_val_decision_score": "0.7133624143149648", "union_val_gap": "0.011793150245700712", - "union_val_ranking_score": "0.7015692640692641" + "union_val_ranking_score": "0.7015692640692641", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3530,7 +3470,6 @@ "exact_val_ranking_score": "0.043099210968826795", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.017928450700376908", @@ -3554,7 +3493,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6912897822445562", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6912897822445562" + "union_val_ranking_score": "0.6912897822445562", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3568,7 +3508,6 @@ "exact_val_ranking_score": "0.003349460364719018", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0003268363416054406", @@ -3592,7 +3531,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2948179787588662", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2948179787588662" + "union_val_ranking_score": "0.2948179787588662", + "path": "" }, { "exact_decision_head": "linear probe", @@ -3606,7 +3546,6 @@ "exact_val_ranking_score": "0.12086776859504132", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.07575936549127547", @@ -3630,7 +3569,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7976559342583861", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7976559342583861" + "union_val_ranking_score": "0.7976559342583861", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3644,7 +3584,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03431519870355593", @@ -3668,7 +3607,8 @@ "union_test_gap": "-0.00029696594024497536", "union_val_decision_score": "0.7015653241482795", "union_val_gap": "4.7047341732997516e-05", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3682,7 +3622,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.02050876208409485", @@ -3706,7 +3645,8 @@ "union_test_gap": "0.045880503237594294", "union_val_decision_score": "0.6962583126352054", "union_val_gap": "0.005582899287591814", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3720,7 +3660,6 @@ "exact_val_ranking_score": "0.0036289408029031526", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00037608715321531055", @@ -3744,7 +3683,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.29429605373755735", "union_val_gap": "0.0", - "union_val_ranking_score": "0.29429605373755735" + "union_val_ranking_score": "0.29429605373755735", + "path": "" }, { "exact_decision_head": "linear probe", @@ -3758,7 +3698,6 @@ "exact_val_ranking_score": "0.12163050624589085", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.09974311720982598", @@ -3782,7 +3721,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7976559342583861", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7976559342583861" + "union_val_ranking_score": "0.7976559342583861", + "path": "" }, { "exact_decision_head": "linear probe", @@ -3796,7 +3736,6 @@ "exact_val_ranking_score": "0.07647961072618607", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.058967493704484314", @@ -3820,7 +3759,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7021793497677742", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7021793497677742" + "union_val_ranking_score": "0.7021793497677742", + "path": "" }, { "exact_decision_head": "linear probe", @@ -3834,7 +3774,6 @@ "exact_val_ranking_score": "0.043420983295568316", "family": "FCN", "model_tag": "fcn", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.029098665712304895", @@ -3858,7 +3797,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6933831038489982", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6933831038489982" + "union_val_ranking_score": "0.6933831038489982", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3872,7 +3812,6 @@ "exact_val_ranking_score": "0.0031948881789137383", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.000398077435365184", @@ -3896,7 +3835,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3130246199713141", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3130246199713141" + "union_val_ranking_score": "0.3130246199713141", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3910,7 +3850,6 @@ "exact_val_ranking_score": "0.12572845629262244", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.06541062880458605", @@ -3934,7 +3873,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8136773785024538", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8136773785024538" + "union_val_ranking_score": "0.8136773785024538", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3948,7 +3888,6 @@ "exact_val_ranking_score": "0.08108271403540572", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.04061009784825756", @@ -3972,7 +3911,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7207024927160893", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7207024927160893" + "union_val_ranking_score": "0.7207024927160893", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -3986,7 +3926,6 @@ "exact_val_ranking_score": "0.0455414882958375", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.022736128443885992", @@ -4010,7 +3949,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7088464801269441", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7088464801269441" + "union_val_ranking_score": "0.7088464801269441", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4024,7 +3964,6 @@ "exact_val_ranking_score": "0.003656307129798903", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0004116035724473925", @@ -4048,7 +3987,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3129112248106385", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3129112248106385" + "union_val_ranking_score": "0.3129112248106385", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4062,7 +4002,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.06604572317905974", @@ -4086,7 +4025,8 @@ "union_test_gap": "0.01484136735033148", "union_val_decision_score": "0.8131524008350731", "union_val_gap": "0.01595894944850451", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4100,7 +4040,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.038869102384950775", @@ -4124,7 +4063,8 @@ "union_test_gap": "0.010273937098576491", "union_val_decision_score": "0.7148757170172084", "union_val_gap": "0.013357440210661875", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4138,7 +4078,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.020495144103834257", @@ -4162,7 +4101,8 @@ "union_test_gap": "0.006592781710044404", "union_val_decision_score": "0.7015002586652871", "union_val_gap": "0.01082484531767347", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4176,7 +4116,6 @@ "exact_val_ranking_score": "0.001528350909368791", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00036987379624400454", @@ -4200,7 +4139,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.31476872634310843", "union_val_gap": "0.0", - "union_val_ranking_score": "0.31476872634310843" + "union_val_ranking_score": "0.31476872634310843", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4214,7 +4154,6 @@ "exact_val_ranking_score": "0.1186024147879047", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.08182944446297657", @@ -4238,7 +4177,8 @@ "union_test_gap": "0.011682222509434403", "union_val_decision_score": "0.8181423987511924", "union_val_gap": "0.020669744358910225", - "union_val_ranking_score": "0.7974726543922822" + "union_val_ranking_score": "0.7974726543922822", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4252,7 +4192,6 @@ "exact_val_ranking_score": "0.08148543948704247", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03819635082226218", @@ -4276,7 +4215,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7256034753011779", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7256034753011779" + "union_val_ranking_score": "0.7256034753011779", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4290,7 +4230,6 @@ "exact_val_ranking_score": "0.04613821138211382", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.021165185967854567", @@ -4314,7 +4253,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7074814597750377", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7074814597750377" + "union_val_ranking_score": "0.7074814597750377", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4328,7 +4268,6 @@ "exact_val_ranking_score": "0.0027012425715829285", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00036963872610239895", @@ -4352,7 +4291,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.30868675618673724", "union_val_gap": "0.0", - "union_val_ranking_score": "0.30868675618673724" + "union_val_ranking_score": "0.30868675618673724", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4366,7 +4306,6 @@ "exact_val_ranking_score": "0.11824647065067402", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.055750221614838426", @@ -4390,7 +4329,8 @@ "union_test_gap": "0.008435912152656933", "union_val_decision_score": "0.8111111111111111", "union_val_gap": "0.013638456718828906", - "union_val_ranking_score": "0.7974726543922822" + "union_val_ranking_score": "0.7974726543922822", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4404,7 +4344,6 @@ "exact_val_ranking_score": "0.07768666378938284", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0356468915576778", @@ -4428,7 +4367,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7188508982531667", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7188508982531667" + "union_val_ranking_score": "0.7188508982531667", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4442,7 +4382,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.016801706970978273", @@ -4466,7 +4405,8 @@ "union_test_gap": "0.004368663471292611", "union_val_decision_score": "0.7033268834654516", "union_val_gap": "0.01265147011783796", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4480,7 +4420,6 @@ "exact_val_ranking_score": "0.0041356492969396195", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00042782651526874734", @@ -4504,7 +4443,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.30674980181224504", "union_val_gap": "0.0", - "union_val_ranking_score": "0.30674980181224504" + "union_val_ranking_score": "0.30674980181224504", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4518,7 +4458,6 @@ "exact_val_ranking_score": "0.12025521790851086", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06535531985010146", @@ -4542,7 +4481,8 @@ "union_test_gap": "0.009020257725968261", "union_val_decision_score": "0.804416403785489", "union_val_gap": "0.006943749393206744", - "union_val_ranking_score": "0.7974726543922822" + "union_val_ranking_score": "0.7974726543922822", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4556,7 +4496,6 @@ "exact_val_ranking_score": "0.07531901649548707", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.04049928566381453", @@ -4580,7 +4519,8 @@ "union_test_gap": "0.00984108670064976", "union_val_decision_score": "0.7096515175847117", "union_val_gap": "0.00813324077816513", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "linear probe", @@ -4594,7 +4534,6 @@ "exact_val_ranking_score": "0.04882017900732303", "family": "FengWu", "model_tag": "fengwu", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02199779031689959", @@ -4618,7 +4557,8 @@ "union_test_gap": "0.015150501067724198", "union_val_decision_score": "0.6987796297897932", "union_val_gap": "0.008104216442179624", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4632,7 +4572,6 @@ "exact_val_ranking_score": "0.0008811590083408097", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0003596048414560045", @@ -4656,7 +4595,8 @@ "union_test_gap": "-0.09403226703808687", "union_val_decision_score": "0.3118144595732139", "union_val_gap": "0.001671993688703799", - "union_val_ranking_score": "0.3101424658845101" + "union_val_ranking_score": "0.3101424658845101", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4670,7 +4610,6 @@ "exact_val_ranking_score": "0.11941859249189585", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.06032490713940969", @@ -4694,7 +4633,8 @@ "union_test_gap": "0.050156261654859424", "union_val_decision_score": "0.8658519277773397", "union_val_gap": "0.06831247872679147", - "union_val_ranking_score": "0.7975394490505482" + "union_val_ranking_score": "0.7975394490505482", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4708,7 +4648,6 @@ "exact_val_ranking_score": "0.08072936959401543", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03373982859419053", @@ -4732,7 +4671,8 @@ "union_test_gap": "-0.011076643607030201", "union_val_decision_score": "0.754546876833418", "union_val_gap": "0.04256271595760763", - "union_val_ranking_score": "0.7119841608758104" + "union_val_ranking_score": "0.7119841608758104", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4746,7 +4686,6 @@ "exact_val_ranking_score": "0.04576053854784421", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.01904942261850253", @@ -4770,7 +4709,8 @@ "union_test_gap": "0.001455907837012771", "union_val_decision_score": "0.7033365957825429", "union_val_gap": "0.003706992036648815", - "union_val_ranking_score": "0.6996296037458941" + "union_val_ranking_score": "0.6996296037458941", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4784,7 +4724,6 @@ "exact_val_ranking_score": "0.0010117798317730948", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00039999784430104664", @@ -4808,7 +4747,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.31039074285584656", "union_val_gap": "0.0", - "union_val_ranking_score": "0.31039074285584656" + "union_val_ranking_score": "0.31039074285584656", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4822,7 +4762,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.06578907632994217", @@ -4846,7 +4785,8 @@ "union_test_gap": "0.012039669560081956", "union_val_decision_score": "0.8107372400756144", "union_val_gap": "0.013543788689045821", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4860,7 +4800,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03878602989774424", @@ -4884,7 +4823,8 @@ "union_test_gap": "0.008279917485013666", "union_val_decision_score": "0.7130764176104518", "union_val_gap": "0.01155814080390527", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -4898,7 +4838,6 @@ "exact_val_ranking_score": "0.04600811907983762", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.021045021724179047", @@ -4922,7 +4861,8 @@ "union_test_gap": "0.0039636864245868875", "union_val_decision_score": "0.6975587357465372", "union_val_gap": "0.00688332239892353", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4936,7 +4876,6 @@ "exact_val_ranking_score": "0.0011176098051633573", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0003830459807690152", @@ -4960,7 +4899,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3180456774666437", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3180456774666437" + "union_val_ranking_score": "0.3180456774666437", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -4974,7 +4914,6 @@ "exact_val_ranking_score": "0.12152010605391074", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.07205944796033896", @@ -4998,7 +4937,8 @@ "union_test_gap": "-0.010472768553227318", "union_val_decision_score": "0.810906174819567", "union_val_gap": "0.009041372954765037", - "union_val_ranking_score": "0.8018648018648019" + "union_val_ranking_score": "0.8018648018648019", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5012,7 +4952,6 @@ "exact_val_ranking_score": "0.07581073896863369", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.0420642409494585", @@ -5036,7 +4975,8 @@ "union_test_gap": "-0.041394747241786156", "union_val_decision_score": "0.7266319786024423", "union_val_gap": "0.020155767669137514", - "union_val_ranking_score": "0.7064762109333048" + "union_val_ranking_score": "0.7064762109333048", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5050,7 +4990,6 @@ "exact_val_ranking_score": "0.04428715218026559", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.020363596550511454", @@ -5074,7 +5013,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.72648949860566", "union_val_gap": "0.0", - "union_val_ranking_score": "0.72648949860566" + "union_val_ranking_score": "0.72648949860566", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5088,7 +5028,6 @@ "exact_val_ranking_score": "0.0009152735719220856", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00037118462783325537", @@ -5112,7 +5051,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.30677382687191324", "union_val_gap": "0.0", - "union_val_ranking_score": "0.30677382687191324" + "union_val_ranking_score": "0.30677382687191324", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5126,7 +5066,6 @@ "exact_val_ranking_score": "0.11910835089275812", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06705127569812482", @@ -5150,7 +5089,8 @@ "union_test_gap": "0.0032455383122205195", "union_val_decision_score": "0.8054623862002874", "union_val_gap": "0.007989731808005218", - "union_val_ranking_score": "0.7974726543922822" + "union_val_ranking_score": "0.7974726543922822", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5164,7 +5104,6 @@ "exact_val_ranking_score": "0.07953855494839102", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03317855540638228", @@ -5188,7 +5127,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.710349352140397", "union_val_gap": "0.0", - "union_val_ranking_score": "0.710349352140397" + "union_val_ranking_score": "0.710349352140397", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5202,7 +5142,6 @@ "exact_val_ranking_score": "0.04500911929271383", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.018717347905238046", @@ -5226,7 +5165,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6980281690140845", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6980281690140845" + "union_val_ranking_score": "0.6980281690140845", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5240,7 +5180,6 @@ "exact_val_ranking_score": "0.000883817480228723", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0003867174887645146", @@ -5264,7 +5203,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.31099917685286793", "union_val_gap": "0.0", - "union_val_ranking_score": "0.31099917685286793" + "union_val_ranking_score": "0.31099917685286793", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5278,7 +5218,6 @@ "exact_val_ranking_score": "0.11949356492623209", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.0679936659760212", @@ -5302,7 +5241,8 @@ "union_test_gap": "0.0022841432008837703", "union_val_decision_score": "0.8023909485519105", "union_val_gap": "0.004531416779335218", - "union_val_ranking_score": "0.7978595317725753" + "union_val_ranking_score": "0.7978595317725753", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5316,7 +5256,6 @@ "exact_val_ranking_score": "0.0748566254150317", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03972371884586323", @@ -5340,7 +5279,8 @@ "union_test_gap": "-0.00019367663146285796", "union_val_decision_score": "0.7072442664725155", "union_val_gap": "0.0042447748609242275", - "union_val_ranking_score": "0.7029994916115913" + "union_val_ranking_score": "0.7029994916115913", + "path": "" }, { "exact_decision_head": "pixel MLP head", @@ -5354,7 +5294,6 @@ "exact_val_ranking_score": "0.042816610781532115", "family": "FuXi", "model_tag": "fuxi", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.026062499558490423", @@ -5378,7 +5317,8 @@ "union_test_gap": "0.008746323525994693", "union_val_decision_score": "0.6966977214053203", "union_val_gap": "0.006022308057706649", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5392,7 +5332,6 @@ "exact_val_ranking_score": "0.004253509145044662", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00042881385578730365", @@ -5416,7 +5355,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3120952399779768", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3120952399779768" + "union_val_ranking_score": "0.3120952399779768", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5430,7 +5370,6 @@ "exact_val_ranking_score": "0.1289902280130293", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.07239587203738174", @@ -5454,7 +5393,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.8022016303211871", "union_val_gap": "0.0", - "union_val_ranking_score": "0.8022016303211871" + "union_val_ranking_score": "0.8022016303211871", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5468,7 +5408,6 @@ "exact_val_ranking_score": "0.08153057396523697", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.04315307069319409", @@ -5492,7 +5431,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7088338471351122", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7088338471351122" + "union_val_ranking_score": "0.7088338471351122", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5506,7 +5446,6 @@ "exact_val_ranking_score": "0.04535556523362161", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.02328829578760066", @@ -5530,7 +5469,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.698733466652633", "union_val_gap": "0.0", - "union_val_ranking_score": "0.698733466652633" + "union_val_ranking_score": "0.698733466652633", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5544,7 +5484,6 @@ "exact_val_ranking_score": "0.0042863266180882984", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00043738577276574255", @@ -5568,7 +5507,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3113263265858816", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3113263265858816" + "union_val_ranking_score": "0.3113263265858816", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5582,7 +5522,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.07483645613667965", @@ -5606,7 +5545,8 @@ "union_test_gap": "0.014263344771421216", "union_val_decision_score": "0.8139394851723619", "union_val_gap": "0.016746033785793313", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5620,7 +5560,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.04354779125348941", @@ -5644,7 +5583,8 @@ "union_test_gap": "0.011660780793438352", "union_val_decision_score": "0.7182744065368083", "union_val_gap": "0.01675612973026175", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5658,7 +5598,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.02276041840297606", @@ -5682,7 +5621,8 @@ "union_test_gap": "0.007639684811918013", "union_val_decision_score": "0.7019436364687074", "union_val_gap": "0.011268223121093768", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5696,7 +5636,6 @@ "exact_val_ranking_score": "0.0024802356223841263", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00040651309469793043", @@ -5720,7 +5659,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.31489341288382094", "union_val_gap": "0.0", - "union_val_ranking_score": "0.31489341288382094" + "union_val_ranking_score": "0.31489341288382094", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5734,7 +5674,6 @@ "exact_val_ranking_score": "0.11818568994889267", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.08436045538757507", @@ -5758,7 +5697,8 @@ "union_test_gap": "0.019790633919317124", "union_val_decision_score": "0.8039800995024876", "union_val_gap": "0.005996860420931505", - "union_val_ranking_score": "0.7979832390815561" + "union_val_ranking_score": "0.7979832390815561", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5772,7 +5712,6 @@ "exact_val_ranking_score": "0.07748859672658975", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03994690077965589", @@ -5796,7 +5735,8 @@ "union_test_gap": "-0.1691520952567045", "union_val_decision_score": "0.7221556788281226", "union_val_gap": "0.013579036492356145", - "union_val_ranking_score": "0.7085766423357664" + "union_val_ranking_score": "0.7085766423357664", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5810,7 +5750,6 @@ "exact_val_ranking_score": "0.044111302847282145", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.021420904714594617", @@ -5834,7 +5773,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6965685921876124", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6965685921876124" + "union_val_ranking_score": "0.6965685921876124", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5848,7 +5788,6 @@ "exact_val_ranking_score": "0.003894080996884735", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00038395193539280824", @@ -5872,7 +5811,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.30691963220059004", "union_val_gap": "0.0", - "union_val_ranking_score": "0.30691963220059004" + "union_val_ranking_score": "0.30691963220059004", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5886,7 +5826,6 @@ "exact_val_ranking_score": "0.11731255265374893", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.05616248338975538", @@ -5910,7 +5849,8 @@ "union_test_gap": "0.003910462058147268", "union_val_decision_score": "0.8010033444816053", "union_val_gap": "0.0033474102232191605", - "union_val_ranking_score": "0.7976559342583861" + "union_val_ranking_score": "0.7976559342583861", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5924,7 +5864,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03227259772140749", @@ -5948,7 +5887,8 @@ "union_test_gap": "0.003554729307719051", "union_val_decision_score": "0.7063110230649498", "union_val_gap": "0.004792746258403202", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -5962,7 +5902,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.018664183428008234", @@ -5986,7 +5925,8 @@ "union_test_gap": "0.0009289061408825905", "union_val_decision_score": "0.6947253251451462", "union_val_gap": "0.004049911797532557", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6000,7 +5940,6 @@ "exact_val_ranking_score": "0.003532320734722713", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.00044306685015413984", @@ -6024,7 +5963,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.3031747624303057", "union_val_gap": "0.0", - "union_val_ranking_score": "0.3031747624303057" + "union_val_ranking_score": "0.3031747624303057", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6038,7 +5978,6 @@ "exact_val_ranking_score": "0.12016050319921917", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.0711724674009212", @@ -6062,7 +6001,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7976559342583861", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7976559342583861" + "union_val_ranking_score": "0.7976559342583861", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6076,7 +6016,6 @@ "exact_val_ranking_score": "0.07520124642949881", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.045241782115254044", @@ -6100,7 +6039,8 @@ "union_test_gap": "2.369358386755671e-05", "union_val_decision_score": "0.7016606351675854", "union_val_gap": "0.00014235836103881905", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "linear probe", @@ -6114,7 +6054,6 @@ "exact_val_ranking_score": "0.06142857142857142", "family": "Pangu-Weather", "model_tag": "pangu6", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.024740444457605402", @@ -6138,7 +6077,8 @@ "union_test_gap": "0.0007713328035469091", "union_val_decision_score": "0.6915201983814487", "union_val_gap": "0.0008234265830721066", - "union_val_ranking_score": "0.6906967717983766" + "union_val_ranking_score": "0.6906967717983766", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6152,7 +6092,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -6176,7 +6115,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6190,7 +6130,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -6214,7 +6153,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6228,7 +6168,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -6252,7 +6191,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6266,7 +6206,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -6290,7 +6229,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6304,7 +6244,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -6328,7 +6267,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6342,7 +6282,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -6366,7 +6305,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6380,7 +6320,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -6404,7 +6343,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6418,7 +6358,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -6442,7 +6381,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6456,7 +6396,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -6480,7 +6419,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6494,7 +6434,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -6518,7 +6457,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6532,7 +6472,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -6556,7 +6495,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6570,7 +6510,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -6594,7 +6533,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6608,7 +6548,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -6632,7 +6571,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6646,7 +6586,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -6670,7 +6609,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6684,7 +6624,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -6708,7 +6647,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6722,7 +6662,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -6746,7 +6685,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6760,7 +6700,6 @@ "exact_val_ranking_score": "0.00046544514931712863", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.000244434472614863", @@ -6784,7 +6723,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2149459035878704", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2149459035878704" + "union_val_ranking_score": "0.2149459035878704", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6798,7 +6738,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -6822,7 +6761,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6836,7 +6776,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -6860,7 +6799,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6874,7 +6814,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -6898,7 +6837,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6912,7 +6852,6 @@ "exact_val_ranking_score": "0.016877637130801686", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0027275598484246552", @@ -6936,7 +6875,8 @@ "union_test_gap": "0.09923730638441586", "union_val_decision_score": "0.7800653928063089", "union_val_gap": "0.02211882039338331", - "union_val_ranking_score": "0.7579465724129256" + "union_val_ranking_score": "0.7579465724129256", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6950,7 +6890,6 @@ "exact_val_ranking_score": "0.2308050255932992", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.1876883857001677", @@ -6974,7 +6913,8 @@ "union_test_gap": "0.015676762201120686", "union_val_decision_score": "0.8904903385864028", "union_val_gap": "0.029999615649296207", - "union_val_ranking_score": "0.8604907229371066" + "union_val_ranking_score": "0.8604907229371066", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -6988,7 +6928,6 @@ "exact_val_ranking_score": "0.20189905047476264", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.1514864942696933", @@ -7012,7 +6951,8 @@ "union_test_gap": "0.023688286303374273", "union_val_decision_score": "0.8461060598607641", "union_val_gap": "0.02668959040704999", - "union_val_ranking_score": "0.8194164694537142" + "union_val_ranking_score": "0.8194164694537142", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7026,7 +6966,6 @@ "exact_val_ranking_score": "0.16846229187071501", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.10815899441738173", @@ -7050,7 +6989,8 @@ "union_test_gap": "0.05675140203555318", "union_val_decision_score": "0.8546299882836719", "union_val_gap": "0.06584679531636395", - "union_val_ranking_score": "0.7887831929673079" + "union_val_ranking_score": "0.7887831929673079", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7064,7 +7004,6 @@ "exact_val_ranking_score": "0.022231722958529286", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0027479122488527847", @@ -7088,7 +7027,8 @@ "union_test_gap": "0.17497107865629713", "union_val_decision_score": "0.7498273537587483", "union_val_gap": "0.059135931802591135", - "union_val_ranking_score": "0.6906914219561572" + "union_val_ranking_score": "0.6906914219561572", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7102,7 +7042,6 @@ "exact_val_ranking_score": "0.32366412213740453", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.22871793267947133", @@ -7126,7 +7065,8 @@ "union_test_gap": "0.00264182837653959", "union_val_decision_score": "0.8667860857895108", "union_val_gap": "0.027700659869665634", - "union_val_ranking_score": "0.8390854259198451" + "union_val_ranking_score": "0.8390854259198451", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7140,7 +7080,6 @@ "exact_val_ranking_score": "0.2748815165876777", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.18388043920625555", @@ -7164,7 +7103,8 @@ "union_test_gap": "0.010980352431794183", "union_val_decision_score": "0.8240905796307538", "union_val_gap": "0.034993802443587296", - "union_val_ranking_score": "0.7890967771871665" + "union_val_ranking_score": "0.7890967771871665", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7178,7 +7118,6 @@ "exact_val_ranking_score": "0.20444444444444446", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.12031424704824639", @@ -7202,7 +7141,8 @@ "union_test_gap": "0.039655419640547196", "union_val_decision_score": "0.8349865778850624", "union_val_gap": "0.06746718455313094", - "union_val_ranking_score": "0.7675193933319314" + "union_val_ranking_score": "0.7675193933319314", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7216,7 +7156,6 @@ "exact_val_ranking_score": "0.0233226297110022", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0036593166775145225", @@ -7240,7 +7179,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7530848622062031", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7530848622062031" + "union_val_ranking_score": "0.7530848622062031", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7254,7 +7194,6 @@ "exact_val_ranking_score": "0.28806133625410735", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.2392007486709378", @@ -7278,7 +7217,8 @@ "union_test_gap": "-0.0009059642606021878", "union_val_decision_score": "0.8718139484243548", "union_val_gap": "0.02207407287605323", - "union_val_ranking_score": "0.8497398755483015" + "union_val_ranking_score": "0.8497398755483015", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7292,7 +7232,6 @@ "exact_val_ranking_score": "0.23774647887323946", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.19246520724380978", @@ -7316,7 +7255,8 @@ "union_test_gap": "-0.04054078762905722", "union_val_decision_score": "0.796582029663564", "union_val_gap": "0.006808162632569004", - "union_val_ranking_score": "0.789773867030995" + "union_val_ranking_score": "0.789773867030995", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7330,7 +7270,6 @@ "exact_val_ranking_score": "0.17796610169491525", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.12467047355946634", @@ -7354,7 +7293,8 @@ "union_test_gap": "-0.02891476357741396", "union_val_decision_score": "0.7802052162248743", "union_val_gap": "0.02094585104470381", - "union_val_ranking_score": "0.7592593651801705" + "union_val_ranking_score": "0.7592593651801705", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7368,7 +7308,6 @@ "exact_val_ranking_score": "0.02669537136706136", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0030782058188584395", @@ -7392,7 +7331,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6888590116997952", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6888590116997952" + "union_val_ranking_score": "0.6888590116997952", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7406,7 +7346,6 @@ "exact_val_ranking_score": "0.3481654957064793", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.26178619067169895", @@ -7430,7 +7369,8 @@ "union_test_gap": "-0.03511216656359095", "union_val_decision_score": "0.8463626765874013", "union_val_gap": "0.020898995996049274", - "union_val_ranking_score": "0.825463680591352" + "union_val_ranking_score": "0.825463680591352", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7444,7 +7384,6 @@ "exact_val_ranking_score": "0.29323758228605623", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.21329613245127957", @@ -7468,7 +7407,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7878499830095744", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7878499830095744" + "union_val_ranking_score": "0.7878499830095744", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7482,7 +7422,6 @@ "exact_val_ranking_score": "0.22416812609457093", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.14100646513486956", @@ -7506,7 +7445,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7682832316376906", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7682832316376906" + "union_val_ranking_score": "0.7682832316376906", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7520,7 +7460,6 @@ "exact_val_ranking_score": "0.017964421114027414", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0031953098868323193", @@ -7544,7 +7483,8 @@ "union_test_gap": "0.09494609500670914", "union_val_decision_score": "0.7870299254989238", "union_val_gap": "0.010195461444427734", - "union_val_ranking_score": "0.7768344640544961" + "union_val_ranking_score": "0.7768344640544961", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7558,7 +7498,6 @@ "exact_val_ranking_score": "0.2726772952171523", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.22946688569711154", @@ -7582,7 +7521,8 @@ "union_test_gap": "-0.006476561500051492", "union_val_decision_score": "0.8688807502119548", "union_val_gap": "0.0021751625260553187", - "union_val_ranking_score": "0.8667055876858994" + "union_val_ranking_score": "0.8667055876858994", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7596,7 +7536,6 @@ "exact_val_ranking_score": "0.22014622258326566", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.1797901995298074", @@ -7620,7 +7559,8 @@ "union_test_gap": "0.02670880922526031", "union_val_decision_score": "0.8661975382820603", "union_val_gap": "0.05189228302437687", - "union_val_ranking_score": "0.8143052552576834" + "union_val_ranking_score": "0.8143052552576834", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7634,7 +7574,6 @@ "exact_val_ranking_score": "0.1633352370074243", "family": "Reference", "model_tag": "reference", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.11947246238005743", @@ -7658,7 +7597,8 @@ "union_test_gap": "0.05051641026309972", "union_val_decision_score": "0.8730039040005804", "union_val_gap": "0.08884125994391634", - "union_val_ranking_score": "0.784162644056664" + "union_val_ranking_score": "0.784162644056664", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7672,7 +7612,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -7696,7 +7635,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7710,7 +7650,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -7734,7 +7673,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7748,7 +7688,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -7772,7 +7711,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7786,7 +7726,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -7810,7 +7749,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7824,7 +7764,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -7848,7 +7787,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7862,7 +7802,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -7886,7 +7825,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7900,7 +7840,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -7924,7 +7863,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7938,7 +7878,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -7962,7 +7901,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -7976,7 +7916,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -8000,7 +7939,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8014,7 +7954,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -8038,7 +7977,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8052,7 +7992,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -8076,7 +8015,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8090,7 +8030,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -8114,7 +8053,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8128,7 +8068,6 @@ "exact_val_ranking_score": "0.0004727121748000204", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -8152,7 +8091,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.2179368807313983", "union_val_gap": "0.0", - "union_val_ranking_score": "0.2179368807313983" + "union_val_ranking_score": "0.2179368807313983", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8166,7 +8106,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -8190,7 +8129,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8204,7 +8144,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -8228,7 +8167,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8242,7 +8182,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "linear probe", "ranking_selected_head_arch": "linear", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -8266,7 +8205,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8280,7 +8220,6 @@ "exact_val_ranking_score": "0.00046545205608870404", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "shallow spatial adapter", "ranking_selected_head_arch": "shallow", "ranking_selected_test_pr_auc": "0.0002463357401629007", @@ -8304,7 +8243,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.21494875104722905", "union_val_gap": "0.0", - "union_val_ranking_score": "0.21494875104722905" + "union_val_ranking_score": "0.21494875104722905", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8318,7 +8258,6 @@ "exact_val_ranking_score": "0.11931877546755824", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.06288888888888888", @@ -8342,7 +8281,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7971934513865686", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7971934513865686" + "union_val_ranking_score": "0.7971934513865686", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8356,7 +8296,6 @@ "exact_val_ranking_score": "0.07460890493381468", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.03802083333333333", @@ -8380,7 +8319,8 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.7015182768065465", "union_val_gap": "0.0", - "union_val_ranking_score": "0.7015182768065465" + "union_val_ranking_score": "0.7015182768065465", + "path": "" }, { "exact_decision_head": "shallow spatial adapter", @@ -8394,7 +8334,6 @@ "exact_val_ranking_score": "0.04253189892419314", "family": "StormCast", "model_tag": "stormcast", - "path": "REDACTED_LOCAL_RESULT_PATH", "ranking_selected_head": "pixel MLP head", "ranking_selected_head_arch": "pixel_mlp", "ranking_selected_test_pr_auc": "0.02093558282208589", @@ -8418,30 +8357,17 @@ "union_test_gap": "0.0", "union_val_decision_score": "0.6906754133476136", "union_val_gap": "0.0", - "union_val_ranking_score": "0.6906754133476136" + "union_val_ranking_score": "0.6906754133476136", + "path": "" } ], "summary": [ { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FireWx-FM ref.", "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FireWx-FM ref.", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -8449,41 +8375,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.17497107865629713, + "n": 5, "mean": 0.07383089600948442, + "std": 0.07453636071372995, "min": 0.0, - "n": 5, - "std": 0.07453636071372995 + "max": 0.17497107865629713 }, "union_regret": { - "max": 0.17497107865629713, + "n": 5, "mean": 0.07383089600948442, + "std": 0.07453636071372995, "min": 0.0, - "n": 5, - "std": 0.07453636071372995 + "max": 0.17497107865629713 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FireWx-FM ref.", "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FireWx-FM ref.", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -8491,41 +8417,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.015676762201120686, + "n": 5, "mean": 0.003663718115532055, + "std": 0.006812231244812292, "min": 0.0, - "n": 5, - "std": 0.006812231244812292 + "max": 0.015676762201120686 }, "union_regret": { - "max": 0.015676762201120686, + "n": 5, "mean": 0.003663718115532055, + "std": 0.006812231244812292, "min": 0.0, - "n": 5, - "std": 0.006812231244812292 + "max": 0.015676762201120686 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FireWx-FM ref.", "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FireWx-FM ref.", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -8533,41 +8459,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.02670880922526031, + "n": 5, "mean": 0.012275489592085752, + "std": 0.012665162001740834, "min": 0.0, - "n": 5, - "std": 0.012665162001740834 + "max": 0.02670880922526031 }, "union_regret": { - "max": 0.02670880922526031, + "n": 5, "mean": 0.012275489592085752, + "std": 0.012665162001740834, "min": 0.0, - "n": 5, - "std": 0.012665162001740834 + "max": 0.02670880922526031 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FireWx-FM ref.", "model_tag": "reference", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FireWx-FM ref.", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -8575,41 +8501,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.05675140203555318, + "n": 5, "mean": 0.029384646387840017, + "std": 0.02751315335001922, "min": 0.0, - "n": 5, - "std": 0.02751315335001922 + "max": 0.05675140203555318 }, "union_regret": { - "max": 0.05675140203555318, + "n": 5, "mean": 0.029384646387840017, + "std": 0.02751315335001922, "min": 0.0, - "n": 5, - "std": 0.02751315335001922 + "max": 0.05675140203555318 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Prithvi-WxC", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -8617,41 +8543,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Prithvi-WxC", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -8659,41 +8585,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Prithvi-WxC", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -8701,41 +8627,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Prithvi-WxC", "model_tag": "prithvi_wxc", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Prithvi-WxC", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -8743,41 +8669,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0004925501476206198, - "mean": 0.00010153879814819402, - "min": 0.0, - "n": 5, - "std": 0.00021861477435572763 - }, - "label": "Aurora", "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Aurora", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -8785,41 +8711,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.00010153879814819402, + "std": 0.00021861477435572763, + "min": 0.0, + "max": 0.0004925501476206198 + }, "tolerated_regret": { - "max": 0.240793572992212, + "n": 5, "mean": 0.04945471159670635, + "std": 0.10697394238964528, "min": 0.0, - "n": 5, - "std": 0.10697394238964528 + "max": 0.240793572992212 }, "union_regret": { - "max": 0.240793572992212, + "n": 5, "mean": 0.04945471159670635, + "std": 0.10697394238964528, "min": 0.0, - "n": 5, - "std": 0.10697394238964528 + "max": 0.240793572992212 } }, { - "exact_regret": { - "max": 0.11833577252770228, - "mean": 0.023667154505540456, - "min": 0.0, - "n": 5, - "std": 0.05292136630837888 - }, - "label": "Aurora", "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Aurora", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -8827,41 +8753,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.023667154505540456, + "std": 0.05292136630837888, + "min": 0.0, + "max": 0.11833577252770228 + }, "tolerated_regret": { - "max": 0.7714149204832434, + "n": 5, "mean": 0.1542829840966487, + "std": 0.34498724021162547, "min": 0.0, - "n": 5, - "std": 0.34498724021162547 + "max": 0.7714149204832434 }, "union_regret": { - "max": 0.7714149204832434, + "n": 5, "mean": 0.1542829840966487, + "std": 0.34498724021162547, "min": 0.0, - "n": 5, - "std": 0.34498724021162547 + "max": 0.7714149204832434 } }, { - "exact_regret": { - "max": 0.07325639739086803, - "mean": 0.014651279478173606, - "min": 0.0, - "n": 5, - "std": 0.032761256870543834 - }, - "label": "Aurora", "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Aurora", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -8869,41 +8795,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.014651279478173606, + "std": 0.032761256870543834, + "min": 0.0, + "max": 0.07325639739086803 + }, "tolerated_regret": { - "max": 0.6996715316108496, + "n": 5, "mean": 0.1399343063221699, + "std": 0.31290262132065055, "min": 0.0, - "n": 5, - "std": 0.31290262132065055 + "max": 0.6996715316108496 }, "union_regret": { - "max": 0.6996715316108496, + "n": 5, "mean": 0.1399343063221699, + "std": 0.31290262132065055, "min": 0.0, - "n": 5, - "std": 0.31290262132065055 + "max": 0.6996715316108496 } }, { - "exact_regret": { - "max": 0.04101254412979794, - "mean": 0.008202508825959588, - "min": 0.0, - "n": 5, - "std": 0.01834136732088763 - }, - "label": "Aurora", "model_tag": "aurora", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Aurora", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -8911,41 +8837,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.008202508825959588, + "std": 0.01834136732088763, + "min": 0.0, + "max": 0.04101254412979794 + }, "tolerated_regret": { - "max": 0.7185324707231184, + "n": 5, "mean": 0.1437064941446237, + "std": 0.32133748971555404, "min": 0.0, - "n": 5, - "std": 0.32133748971555404 + "max": 0.7185324707231184 }, "union_regret": { - "max": 0.7185324707231184, + "n": 5, "mean": 0.1437064941446237, + "std": 0.32133748971555404, "min": 0.0, - "n": 5, - "std": 0.32133748971555404 + "max": 0.7185324707231184 } }, { - "exact_regret": { - "max": 7.571921560175121e-06, - "mean": 3.0287686240700486e-06, - "min": 0.0, - "n": 5, - "std": 4.147312242167625e-06 - }, - "label": "ClimaX", "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "ClimaX", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -8953,41 +8879,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 3.0287686240700486e-06, + "std": 4.147312242167625e-06, + "min": 0.0, + "max": 7.571921560175121e-06 + }, "tolerated_regret": { - "max": 0.0032399924956598714, + "n": 5, "mean": 0.0012959969982639485, + "std": 0.0017746169760203706, "min": 0.0, - "n": 5, - "std": 0.0017746169760203706 + "max": 0.0032399924956598714 }, "union_regret": { - "max": 0.0032399924956598714, + "n": 5, "mean": 0.0012959969982639485, + "std": 0.0017746169760203706, "min": 0.0, - "n": 5, - "std": 0.0017746169760203706 + "max": 0.0032399924956598714 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "ClimaX", "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "ClimaX", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -8995,41 +8921,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "ClimaX", "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "ClimaX", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -9037,41 +8963,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "ClimaX", "model_tag": "climax", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "ClimaX", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -9079,41 +9005,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "StormCast", "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "StormCast", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -9121,41 +9047,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "StormCast", "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "StormCast", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -9163,41 +9089,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "StormCast", "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "StormCast", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -9205,41 +9131,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "StormCast", "model_tag": "stormcast", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "StormCast", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -9247,41 +9173,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "DLWP", "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "DLWP", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -9289,41 +9215,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.015203005078871956, - "mean": 0.0048037709215293075, - "min": 0.0, - "n": 5, - "std": 0.006217185202152866 - }, - "label": "DLWP", "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "DLWP", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -9331,41 +9257,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0048037709215293075, + "std": 0.006217185202152866, + "min": 0.0, + "max": 0.015203005078871956 + }, "tolerated_regret": { - "max": 0.03305057342744666, + "n": 5, "mean": 0.016716228534155796, + "std": 0.016079313546074458, "min": 0.0, - "n": 5, - "std": 0.016079313546074458 + "max": 0.03305057342744666 }, "union_regret": { - "max": 0.03305057342744666, + "n": 5, "mean": 0.016716228534155796, + "std": 0.016079313546074458, "min": 0.0, - "n": 5, - "std": 0.016079313546074458 + "max": 0.03305057342744666 } }, { - "exact_regret": { - "max": 0.005523780499856246, - "mean": 0.0017281632798742507, - "min": 0.0, - "n": 5, - "std": 0.002514722758075371 - }, - "label": "DLWP", "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "DLWP", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -9373,41 +9299,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0017281632798742507, + "std": 0.002514722758075371, + "min": 0.0, + "max": 0.005523780499856246 + }, "tolerated_regret": { - "max": 0.053927677500854476, + "n": 5, "mean": 0.02846514801700826, + "std": 0.026938012702643194, "min": 0.0, - "n": 5, - "std": 0.026938012702643194 + "max": 0.053927677500854476 }, "union_regret": { - "max": 0.053927677500854476, + "n": 5, "mean": 0.02846514801700826, + "std": 0.026938012702643194, "min": 0.0, - "n": 5, - "std": 0.026938012702643194 + "max": 0.053927677500854476 } }, { - "exact_regret": { - "max": 0.0023651634514294945, - "mean": 0.0007702319787454587, - "min": 0.0, - "n": 5, - "std": 0.0010995336594539604 - }, - "label": "DLWP", "model_tag": "dlwp", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "DLWP", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -9415,41 +9341,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0007702319787454587, + "std": 0.0010995336594539604, + "min": 0.0, + "max": 0.0023651634514294945 + }, "tolerated_regret": { - "max": 0.11215006616165157, + "n": 5, "mean": 0.04463354681768479, + "std": 0.04356064433532197, "min": 0.0, - "n": 5, - "std": 0.04356064433532197 + "max": 0.11215006616165157 }, "union_regret": { - "max": 0.11215006616165157, + "n": 5, "mean": 0.04463354681768479, + "std": 0.04356064433532197, "min": 0.0, - "n": 5, - "std": 0.04356064433532197 + "max": 0.11215006616165157 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FCN", "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FCN", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -9457,41 +9383,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.002257520679520411, - "mean": 0.0006342898232943345, - "min": 0.0, - "n": 5, - "std": 0.0009899554165032742 - }, - "label": "FCN", "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FCN", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -9499,41 +9425,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0006342898232943345, + "std": 0.0009899554165032742, + "min": 0.0, + "max": 0.002257520679520411 + }, "tolerated_regret": { - "max": 0.022524473456150496, + "n": 5, "mean": 0.004509624980300697, + "std": 0.010070611656609236, "min": 0.0, - "n": 5, - "std": 0.010070611656609236 + "max": 0.022524473456150496 }, "union_regret": { - "max": 0.022524473456150496, + "n": 5, "mean": 0.004509624980300697, + "std": 0.010070611656609236, "min": 0.0, - "n": 5, - "std": 0.010070611656609236 + "max": 0.022524473456150496 } }, { - "exact_regret": { - "max": 0.0010578427408801938, - "mean": 0.00021156854817603877, - "min": 0.0, - "n": 5, - "std": 0.0004730816556225618 - }, - "label": "FCN", "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FCN", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -9541,41 +9467,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.00021156854817603877, + "std": 0.0004730816556225618, + "min": 0.0, + "max": 0.0010578427408801938 + }, "tolerated_regret": { - "max": 0.020997685254088072, + "n": 5, "mean": 0.004199537050817615, + "std": 0.009390450319657174, "min": 0.0, - "n": 5, - "std": 0.009390450319657174 + "max": 0.020997685254088072 }, "union_regret": { - "max": 0.020997685254088072, + "n": 5, "mean": 0.004199537050817615, + "std": 0.009390450319657174, "min": 0.0, - "n": 5, - "std": 0.009390450319657174 + "max": 0.020997685254088072 } }, { - "exact_regret": { - "max": 2.877280037168889e-05, - "mean": 5.754560074337778e-06, - "min": 0.0, - "n": 5, - "std": 1.2867587506825515e-05 - }, - "label": "FCN", "model_tag": "fcn", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FCN", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -9583,41 +9509,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 5.754560074337778e-06, + "std": 1.2867587506825515e-05, + "min": 0.0, + "max": 2.877280037168889e-05 + }, "tolerated_regret": { - "max": 0.045880503237594294, + "n": 5, "mean": 0.011679805987441694, + "std": 0.019872372458657642, "min": 0.0, - "n": 5, - "std": 0.019872372458657642 + "max": 0.045880503237594294 }, "union_regret": { - "max": 0.045880503237594294, + "n": 5, "mean": 0.011679805987441694, + "std": 0.019872372458657642, "min": 0.0, - "n": 5, - "std": 0.019872372458657642 + "max": 0.045880503237594294 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FengWu", "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FengWu", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -9625,41 +9551,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.0018628094907809783, - "mean": 0.0005029843170376968, - "min": 0.0, - "n": 5, - "std": 0.0008109166521114917 - }, - "label": "FengWu", "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FengWu", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -9667,41 +9593,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0005029843170376968, + "std": 0.0008109166521114917, + "min": 0.0, + "max": 0.0018628094907809783 + }, "tolerated_regret": { - "max": 0.01484136735033148, + "n": 5, "mean": 0.008795951947678215, + "std": 0.005532321338017505, "min": 0.0, - "n": 5, - "std": 0.005532321338017505 + "max": 0.01484136735033148 }, "union_regret": { - "max": 0.01484136735033148, + "n": 5, "mean": 0.008795951947678215, + "std": 0.005532321338017505, "min": 0.0, - "n": 5, - "std": 0.005532321338017505 + "max": 0.01484136735033148 } }, { - "exact_regret": { - "max": 0.0016387212062008855, - "mean": 0.000495228089292582, - "min": 0.0, - "n": 5, - "std": 0.0007349190216431337 - }, - "label": "FengWu", "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FengWu", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -9709,41 +9635,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.000495228089292582, + "std": 0.0007349190216431337, + "min": 0.0, + "max": 0.0016387212062008855 + }, "tolerated_regret": { - "max": 0.010273937098576491, + "n": 5, "mean": 0.00402300475984525, + "std": 0.005510851442075993, "min": 0.0, - "n": 5, - "std": 0.005510851442075993 + "max": 0.010273937098576491 }, "union_regret": { - "max": 0.010273937098576491, + "n": 5, "mean": 0.00402300475984525, + "std": 0.005510851442075993, "min": 0.0, - "n": 5, - "std": 0.005510851442075993 + "max": 0.010273937098576491 } }, { - "exact_regret": { - "max": 0.0027505832409660327, - "mean": 0.0006908222234409067, - "min": 0.0, - "n": 5, - "std": 0.0011910586589384115 - }, - "label": "FengWu", "model_tag": "fengwu", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FengWu", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -9751,41 +9677,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0006908222234409067, + "std": 0.0011910586589384115, + "min": 0.0, + "max": 0.0027505832409660327 + }, "tolerated_regret": { - "max": 0.015150501067724198, + "n": 5, "mean": 0.005222389249812243, + "std": 0.0062394095558402415, "min": 0.0, - "n": 5, - "std": 0.0062394095558402415 + "max": 0.015150501067724198 }, "union_regret": { - "max": 0.015150501067724198, + "n": 5, "mean": 0.005222389249812243, + "std": 0.0062394095558402415, "min": 0.0, - "n": 5, - "std": 0.0062394095558402415 + "max": 0.015150501067724198 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FuXi", "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FuXi", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -9793,41 +9719,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.007024214143542151, - "mean": 0.002973545331200933, - "min": 0.0010927807990139538, - "n": 5, - "std": 0.0023946274991058026 - }, - "label": "FuXi", "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FuXi", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -9835,41 +9761,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.002973545331200933, + "std": 0.0023946274991058026, + "min": 0.0010927807990139538, + "max": 0.007024214143542151 + }, "tolerated_regret": { - "max": 0.050156261654859424, + "n": 5, "mean": 0.013545122545609134, + "std": 0.02097023683418404, "min": 0.0, - "n": 5, - "std": 0.02097023683418404 + "max": 0.050156261654859424 }, "union_regret": { - "max": 0.050156261654859424, + "n": 5, "mean": 0.013545122545609134, + "std": 0.02097023683418404, "min": 0.0, - "n": 5, - "std": 0.02097023683418404 + "max": 0.050156261654859424 } }, { - "exact_regret": { - "max": 0.003938013087198336, - "mean": 0.001383793743586542, - "min": 0.0, - "n": 5, - "std": 0.0019248128430711165 - }, - "label": "FuXi", "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FuXi", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -9877,41 +9803,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.001383793743586542, + "std": 0.0019248128430711165, + "min": 0.0, + "max": 0.003938013087198336 + }, "tolerated_regret": { - "max": 0.008279917485013666, + "n": 5, "mean": 0.0016559834970027332, + "std": 0.0037028916689159307, "min": 0.0, - "n": 5, - "std": 0.0037028916689159307 + "max": 0.008279917485013666 }, "union_regret": { - "max": 0.008279917485013666, + "n": 5, "mean": 0.0016559834970027332, + "std": 0.0037028916689159307, "min": 0.0, - "n": 5, - "std": 0.0037028916689159307 + "max": 0.008279917485013666 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "FuXi", "model_tag": "fuxi", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "FuXi", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -9919,41 +9845,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.008746323525994693, + "n": 5, "mean": 0.00283318355751887, + "std": 0.0036808289681375247, "min": 0.0, - "n": 5, - "std": 0.0036808289681375247 + "max": 0.008746323525994693 }, "union_regret": { - "max": 0.008746323525994693, + "n": 5, "mean": 0.00283318355751887, + "std": 0.0036808289681375247, "min": 0.0, - "n": 5, - "std": 0.0036808289681375247 + "max": 0.008746323525994693 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "Pangu-Weather", "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Pangu-Weather", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -9961,41 +9887,41 @@ 99, 123 ], - "tolerated_regret": { - "max": 0.0, + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, + "max": 0.0 + }, + "tolerated_regret": { "n": 5, - "std": 0.0 + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 }, "union_regret": { - "max": 0.0, + "n": 5, "mean": 0.0, + "std": 0.0, "min": 0.0, - "n": 5, - "std": 0.0 + "max": 0.0 } }, { - "exact_regret": { - "max": 0.005711594157251587, - "mean": 0.003154674487241463, - "min": 0.0, - "n": 5, - "std": 0.002546125713211599 - }, - "label": "Pangu-Weather", "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Pangu-Weather", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -10003,41 +9929,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.003154674487241463, + "std": 0.002546125713211599, + "min": 0.0, + "max": 0.005711594157251587 + }, "tolerated_regret": { - "max": 0.019790633919317124, + "n": 5, "mean": 0.007592888149777122, + "std": 0.00897418737588444, "min": 0.0, - "n": 5, - "std": 0.00897418737588444 + "max": 0.019790633919317124 }, "union_regret": { - "max": 0.019790633919317124, + "n": 5, "mean": 0.007592888149777122, + "std": 0.00897418737588444, "min": 0.0, - "n": 5, - "std": 0.00897418737588444 + "max": 0.019790633919317124 } }, { - "exact_regret": { - "max": 0.004535321555179647, - "mean": 0.0017345627303725214, - "min": 0.0, - "n": 5, - "std": 0.0019305189318827886 - }, - "label": "Pangu-Weather", "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Pangu-Weather", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -10045,41 +9971,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0017345627303725214, + "std": 0.0019305189318827886, + "min": 0.0, + "max": 0.004535321555179647 + }, "tolerated_regret": { - "max": 0.011660780793438352, + "n": 5, "mean": 0.003047840737004992, + "std": 0.005053805614558161, "min": 0.0, - "n": 5, - "std": 0.005053805614558161 + "max": 0.011660780793438352 }, "union_regret": { - "max": 0.011660780793438352, + "n": 5, "mean": 0.003047840737004992, + "std": 0.005053805614558161, "min": 0.0, - "n": 5, - "std": 0.005053805614558161 + "max": 0.011660780793438352 } }, { - "exact_regret": { - "max": 0.0027096086413018403, - "mean": 0.0007280423771922354, - "min": 0.0, - "n": 5, - "std": 0.001178746460551365 - }, - "label": "Pangu-Weather", "model_tag": "pangu6", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "Pangu-Weather", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -10087,41 +10013,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0007280423771922354, + "std": 0.001178746460551365, + "min": 0.0, + "max": 0.0027096086413018403 + }, "tolerated_regret": { - "max": 0.007639684811918013, + "n": 5, "mean": 0.0018679847512695024, + "std": 0.0032548337047755126, "min": 0.0, - "n": 5, - "std": 0.0032548337047755126 + "max": 0.007639684811918013 }, "union_regret": { - "max": 0.007639684811918013, + "n": 5, "mean": 0.0018679847512695024, + "std": 0.0032548337047755126, "min": 0.0, - "n": 5, - "std": 0.0032548337047755126 + "max": 0.007639684811918013 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "AlphaEarth", "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "AlphaEarth", "scope": "global", "scope_label": "\\(\\Omega=\\)global", + "n": 5, "seeds": [ 1, 7, @@ -10129,41 +10055,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.272468682523694, + "n": 5, "mean": 0.1722171037486726, + "std": 0.08849214830495522, "min": 0.050655329891645895, - "n": 5, - "std": 0.08849214830495522 + "max": 0.272468682523694 }, "union_regret": { - "max": 0.272468682523694, + "n": 5, "mean": 0.1722171037486726, + "std": 0.08849214830495522, "min": 0.050655329891645895, - "n": 5, - "std": 0.08849214830495522 + "max": 0.272468682523694 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "AlphaEarth", "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "AlphaEarth", "scope": "top5", "scope_label": "\\(\\Omega=\\)top 5\\%", + "n": 5, "seeds": [ 1, 7, @@ -10171,41 +10097,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.1365277562230962, + "n": 5, "mean": 0.06384618090125256, + "std": 0.04965276403138872, "min": 0.0, - "n": 5, - "std": 0.04965276403138872 + "max": 0.1365277562230962 }, "union_regret": { - "max": 0.1365277562230962, + "n": 5, "mean": 0.06384618090125256, + "std": 0.04965276403138872, "min": 0.0, - "n": 5, - "std": 0.04965276403138872 + "max": 0.1365277562230962 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "AlphaEarth", "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "AlphaEarth", "scope": "top10", "scope_label": "\\(\\Omega=\\)top 10\\%", + "n": 5, "seeds": [ 1, 7, @@ -10213,41 +10139,41 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.1615566566666954, + "n": 5, "mean": 0.06573776411084173, + "std": 0.06897015340160571, "min": 0.0, - "n": 5, - "std": 0.06897015340160571 + "max": 0.1615566566666954 }, "union_regret": { - "max": 0.1615566566666954, + "n": 5, "mean": 0.06573776411084173, + "std": 0.06897015340160571, "min": 0.0, - "n": 5, - "std": 0.06897015340160571 + "max": 0.1615566566666954 } }, { - "exact_regret": { - "max": 0.0, - "mean": 0.0, - "min": 0.0, - "n": 5, - "std": 0.0 - }, - "label": "AlphaEarth", "model_tag": "alphaearth", - "n": 5, - "paths": [ - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH", - "REDACTED_LOCAL_RESULT_PATH" - ], + "label": "AlphaEarth", "scope": "top20", "scope_label": "\\(\\Omega=\\)top 20\\%", + "n": 5, "seeds": [ 1, 7, @@ -10255,20 +10181,94 @@ 99, 123 ], + "paths": [ + "", + "", + "", + "", + "" + ], + "exact_regret": { + "n": 5, + "mean": 0.0, + "std": 0.0, + "min": 0.0, + "max": 0.0 + }, "tolerated_regret": { - "max": 0.13482302181308625, + "n": 5, "mean": 0.038803552655092256, + "std": 0.0594825313313219, "min": 0.0, - "n": 5, - "std": 0.0594825313313219 + "max": 0.13482302181308625 }, "union_regret": { - "max": 0.13482302181308625, + "n": 5, "mean": 0.038803552655092256, + "std": 0.0594825313313219, "min": 0.0, - "n": 5, - "std": 0.0594825313313219 + "max": 0.13482302181308625 } } + ], + "issues": [], + "notes": [ + "FireWx-FM ref. global exact_regret has true zero displayed std", + "FireWx-FM ref. top5 exact_regret has true zero displayed std", + "FireWx-FM ref. top10 exact_regret has true zero displayed std", + "FireWx-FM ref. top20 exact_regret has true zero displayed std", + "Prithvi-WxC global exact_regret has true zero displayed std", + "Prithvi-WxC global tolerated_regret has true zero displayed std", + "Prithvi-WxC global union_regret has true zero displayed std", + "Prithvi-WxC top5 exact_regret has true zero displayed std", + "Prithvi-WxC top5 tolerated_regret has true zero displayed std", + "Prithvi-WxC top5 union_regret has true zero displayed std", + "Prithvi-WxC top10 exact_regret has true zero displayed std", + "Prithvi-WxC top10 tolerated_regret has true zero displayed std", + "Prithvi-WxC top10 union_regret has true zero displayed std", + "Prithvi-WxC top20 exact_regret has true zero displayed std", + "Prithvi-WxC top20 tolerated_regret has true zero displayed std", + "Prithvi-WxC top20 union_regret has true zero displayed std", + "ClimaX top5 exact_regret has true zero displayed std", + "ClimaX top5 tolerated_regret has true zero displayed std", + "ClimaX top5 union_regret has true zero displayed std", + "ClimaX top10 exact_regret has true zero displayed std", + "ClimaX top10 tolerated_regret has true zero displayed std", + "ClimaX top10 union_regret has true zero displayed std", + "ClimaX top20 exact_regret has true zero displayed std", + "ClimaX top20 tolerated_regret has true zero displayed std", + "ClimaX top20 union_regret has true zero displayed std", + "StormCast global exact_regret has true zero displayed std", + "StormCast global tolerated_regret has true zero displayed std", + "StormCast global union_regret has true zero displayed std", + "StormCast top5 exact_regret has true zero displayed std", + "StormCast top5 tolerated_regret has true zero displayed std", + "StormCast top5 union_regret has true zero displayed std", + "StormCast top10 exact_regret has true zero displayed std", + "StormCast top10 tolerated_regret has true zero displayed std", + "StormCast top10 union_regret has true zero displayed std", + "StormCast top20 exact_regret has true zero displayed std", + "StormCast top20 tolerated_regret has true zero displayed std", + "StormCast top20 union_regret has true zero displayed std", + "DLWP global exact_regret has true zero displayed std", + "DLWP global tolerated_regret has true zero displayed std", + "DLWP global union_regret has true zero displayed std", + "FCN global exact_regret has true zero displayed std", + "FCN global tolerated_regret has true zero displayed std", + "FCN global union_regret has true zero displayed std", + "FengWu global exact_regret has true zero displayed std", + "FengWu global tolerated_regret has true zero displayed std", + "FengWu global union_regret has true zero displayed std", + "FuXi global exact_regret has true zero displayed std", + "FuXi global tolerated_regret has true zero displayed std", + "FuXi global union_regret has true zero displayed std", + "FuXi top20 exact_regret has true zero displayed std", + "Pangu-Weather global exact_regret has true zero displayed std", + "Pangu-Weather global tolerated_regret has true zero displayed std", + "Pangu-Weather global union_regret has true zero displayed std", + "AlphaEarth global exact_regret has true zero displayed std", + "AlphaEarth top5 exact_regret has true zero displayed std", + "AlphaEarth top10 exact_regret has true zero displayed std", + "AlphaEarth top20 exact_regret has true zero displayed std" ] } diff --git a/assets/primary_rank_change_preview.png b/assets/primary_rank_change_preview.png new file mode 100644 index 0000000000000000000000000000000000000000..d64f146f539d0ce06434aec3e8ec746bcd833cc0 --- /dev/null +++ b/assets/primary_rank_change_preview.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0f724c709d39f9655e6e0c74b2ef51e6d7fadec6fb9d272fa605660d9416ff5 +size 984374 diff --git a/assets/selection_regret_preview.png b/assets/selection_regret_preview.png new file mode 100644 index 0000000000000000000000000000000000000000..5b6bfb17059d06f7c5f881cefe257455e83cdb86 --- /dev/null +++ b/assets/selection_regret_preview.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44bd6782e56031510ebb2401f42983d625294ccc3573659277876be7c158a97a +size 3798853 diff --git a/assets/task_rank_map_preview.png b/assets/task_rank_map_preview.png new file mode 100644 index 0000000000000000000000000000000000000000..992537903f9a876839cc2c0b617460179603304b --- /dev/null +++ b/assets/task_rank_map_preview.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a209caf6d8fa1d2c3be45086e397c534dbefe79176f87a8fa9b12c82b770e475 +size 1237090 diff --git a/models/wildfire_fm/README.md b/models/wildfire_fm/README.md new file mode 100644 index 0000000000000000000000000000000000000000..4cd8944159e35f7b8d780518426120b0fd7530cd --- /dev/null +++ b/models/wildfire_fm/README.md @@ -0,0 +1,14 @@ +# WildFIRE-FM Checkpoints + +This directory contains metadata for the released WildFIRE-FM seeded checkpoints. +The checkpoint files are stored on the Hugging Face Hub under: + +```text +models/wildfire_fm/checkpoints/seed_1/best_firms_prauc.pt +models/wildfire_fm/checkpoints/seed_7/best_firms_prauc.pt +models/wildfire_fm/checkpoints/seed_42/best_firms_prauc.pt +models/wildfire_fm/checkpoints/seed_99/best_firms_prauc.pt +models/wildfire_fm/checkpoints/seed_123/best_firms_prauc.pt +``` + +Use `modeling_unet.py` to instantiate the compact U-Net architecture before loading a checkpoint state dict. diff --git a/models/wildfire_fm/checkpoint_manifest.json b/models/wildfire_fm/checkpoint_manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..e494bc36f67e2b9b340b475b495a2b5fbccfcbd0 --- /dev/null +++ b/models/wildfire_fm/checkpoint_manifest.json @@ -0,0 +1,36 @@ +{ + "model": "WildFIRE-FM", + "weights": "uploaded to the Hugging Face Hub under the filenames listed below", + "checkpoints": [ + { + "seed": 123, + "filename": "models/wildfire_fm/checkpoints/seed_123/best_firms_prauc.pt", + "sha256": "4629a54134efb0e6fe5dabc177630923e4f4b5520ceff5076a36c03ae45cfbab", + "bytes": 31092197 + }, + { + "seed": 1, + "filename": "models/wildfire_fm/checkpoints/seed_1/best_firms_prauc.pt", + "sha256": "4a8f7e500bbdf222b33e2bc056d7e6202b526745c75dca26abc478bee9b2d84f", + "bytes": 31092197 + }, + { + "seed": 42, + "filename": "models/wildfire_fm/checkpoints/seed_42/best_firms_prauc.pt", + "sha256": "2ba80ddfdcf173a425eafc1d40dfd06088ac6f3e588e9adb866660df84694346", + "bytes": 31092197 + }, + { + "seed": 7, + "filename": "models/wildfire_fm/checkpoints/seed_7/best_firms_prauc.pt", + "sha256": "6a07d34302f63937e8e74c43c651cf3859b74bc95a5b68e37ffb2f37739a0b30", + "bytes": 31092197 + }, + { + "seed": 99, + "filename": "models/wildfire_fm/checkpoints/seed_99/best_firms_prauc.pt", + "sha256": "d5ce9d77c84167a7e24e3953c1f939fa7f621ceb905ac4d33e27eae523c0f5f8", + "bytes": 31092197 + } + ] +} diff --git a/models/wildfire_fm/configs/reference_fireprone_seed_123_20260504.json b/models/wildfire_fm/configs/reference_fireprone_seed_123_20260504.json new file mode 100644 index 0000000000000000000000000000000000000000..8dfb9ebc5e12509e0c7ff089b815ae5126cd0aaf --- /dev/null +++ b/models/wildfire_fm/configs/reference_fireprone_seed_123_20260504.json @@ -0,0 +1,62 @@ +{ + "index_root": "", + "run_root": "", + "tile_size": 32, + "max_positive_tiles_per_sample": 64, + "min_negative_tiles_per_sample": 4, + "negative_to_positive_ratio": 3.0, + "positive_rate_source": "tiles", + "pos_weight_source": "tiles", + "pos_weight_cap": 50.0, + "batch_size": 32, + "eval_batch_size": 2, + "epochs": 10, + "learning_rate": 0.0002, + "weight_decay": 0.0001, + "base_channels": 32, + "dropout": 0.1, + "norm_type": "group", + "norm_groups": 8, + "init_head_bias_from_positive_rate": true, + "augment_flip": true, + "threshold": 0.5, + "metric_thresholds": [ + 0.1, + 0.2, + 0.3, + 0.5, + 0.7, + 0.9 + ], + "topk_area_fractions": [ + 0.01, + 0.05, + 0.1 + ], + "fss_radii": [ + 1, + 2, + 4, + 8 + ], + "reliability_bins": 10, + "amp": true, + "seed": 123, + "boundary_radii": [ + 1, + 2, + 4 + ], + "coarsen_factors": [ + 2, + 4, + 8 + ], + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "train_target_mode": "dilate_max", + "train_target_radius": 2, + "loss_type": "bce", + "init_checkpoint": "" +} diff --git a/models/wildfire_fm/configs/reference_fireprone_seed_1_20260504.json b/models/wildfire_fm/configs/reference_fireprone_seed_1_20260504.json new file mode 100644 index 0000000000000000000000000000000000000000..6278357a9221e2611c999d1691e8a5af78f202a8 --- /dev/null +++ b/models/wildfire_fm/configs/reference_fireprone_seed_1_20260504.json @@ -0,0 +1,62 @@ +{ + "index_root": "", + "run_root": "", + "tile_size": 32, + "max_positive_tiles_per_sample": 64, + "min_negative_tiles_per_sample": 4, + "negative_to_positive_ratio": 3.0, + "positive_rate_source": "tiles", + "pos_weight_source": "tiles", + "pos_weight_cap": 50.0, + "batch_size": 32, + "eval_batch_size": 2, + "epochs": 10, + "learning_rate": 0.0002, + "weight_decay": 0.0001, + "base_channels": 32, + "dropout": 0.1, + "norm_type": "group", + "norm_groups": 8, + "init_head_bias_from_positive_rate": true, + "augment_flip": true, + "threshold": 0.5, + "metric_thresholds": [ + 0.1, + 0.2, + 0.3, + 0.5, + 0.7, + 0.9 + ], + "topk_area_fractions": [ + 0.01, + 0.05, + 0.1 + ], + "fss_radii": [ + 1, + 2, + 4, + 8 + ], + "reliability_bins": 10, + "amp": true, + "seed": 1, + "boundary_radii": [ + 1, + 2, + 4 + ], + "coarsen_factors": [ + 2, + 4, + 8 + ], + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "train_target_mode": "dilate_max", + "train_target_radius": 2, + "loss_type": "bce", + "init_checkpoint": "" +} diff --git a/models/wildfire_fm/configs/reference_fireprone_seed_42_20260504.json b/models/wildfire_fm/configs/reference_fireprone_seed_42_20260504.json new file mode 100644 index 0000000000000000000000000000000000000000..80e76025a665d23a78615e3958f40205af529ef1 --- /dev/null +++ b/models/wildfire_fm/configs/reference_fireprone_seed_42_20260504.json @@ -0,0 +1,62 @@ +{ + "index_root": "", + "run_root": "", + "tile_size": 32, + "max_positive_tiles_per_sample": 64, + "min_negative_tiles_per_sample": 4, + "negative_to_positive_ratio": 3.0, + "positive_rate_source": "tiles", + "pos_weight_source": "tiles", + "pos_weight_cap": 50.0, + "batch_size": 32, + "eval_batch_size": 2, + "epochs": 10, + "learning_rate": 0.0002, + "weight_decay": 0.0001, + "base_channels": 32, + "dropout": 0.1, + "norm_type": "group", + "norm_groups": 8, + "init_head_bias_from_positive_rate": true, + "augment_flip": true, + "threshold": 0.5, + "metric_thresholds": [ + 0.1, + 0.2, + 0.3, + 0.5, + 0.7, + 0.9 + ], + "topk_area_fractions": [ + 0.01, + 0.05, + 0.1 + ], + "fss_radii": [ + 1, + 2, + 4, + 8 + ], + "reliability_bins": 10, + "amp": true, + "seed": 42, + "boundary_radii": [ + 1, + 2, + 4 + ], + "coarsen_factors": [ + 2, + 4, + 8 + ], + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "train_target_mode": "dilate_max", + "train_target_radius": 2, + "loss_type": "bce", + "init_checkpoint": "" +} diff --git a/models/wildfire_fm/configs/reference_fireprone_seed_7_20260504.json b/models/wildfire_fm/configs/reference_fireprone_seed_7_20260504.json new file mode 100644 index 0000000000000000000000000000000000000000..b6db09ac28b0c59ca87ee0af6375685e9f60963a --- /dev/null +++ b/models/wildfire_fm/configs/reference_fireprone_seed_7_20260504.json @@ -0,0 +1,62 @@ +{ + "index_root": "", + "run_root": "", + "tile_size": 32, + "max_positive_tiles_per_sample": 64, + "min_negative_tiles_per_sample": 4, + "negative_to_positive_ratio": 3.0, + "positive_rate_source": "tiles", + "pos_weight_source": "tiles", + "pos_weight_cap": 50.0, + "batch_size": 32, + "eval_batch_size": 2, + "epochs": 10, + "learning_rate": 0.0002, + "weight_decay": 0.0001, + "base_channels": 32, + "dropout": 0.1, + "norm_type": "group", + "norm_groups": 8, + "init_head_bias_from_positive_rate": true, + "augment_flip": true, + "threshold": 0.5, + "metric_thresholds": [ + 0.1, + 0.2, + 0.3, + 0.5, + 0.7, + 0.9 + ], + "topk_area_fractions": [ + 0.01, + 0.05, + 0.1 + ], + "fss_radii": [ + 1, + 2, + 4, + 8 + ], + "reliability_bins": 10, + "amp": true, + "seed": 7, + "boundary_radii": [ + 1, + 2, + 4 + ], + "coarsen_factors": [ + 2, + 4, + 8 + ], + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "train_target_mode": "dilate_max", + "train_target_radius": 2, + "loss_type": "bce", + "init_checkpoint": "" +} diff --git a/models/wildfire_fm/configs/reference_fireprone_seed_99_20260504.json b/models/wildfire_fm/configs/reference_fireprone_seed_99_20260504.json new file mode 100644 index 0000000000000000000000000000000000000000..3f03e50805481bb8360776287420472b70a6cff1 --- /dev/null +++ b/models/wildfire_fm/configs/reference_fireprone_seed_99_20260504.json @@ -0,0 +1,62 @@ +{ + "index_root": "", + "run_root": "", + "tile_size": 32, + "max_positive_tiles_per_sample": 64, + "min_negative_tiles_per_sample": 4, + "negative_to_positive_ratio": 3.0, + "positive_rate_source": "tiles", + "pos_weight_source": "tiles", + "pos_weight_cap": 50.0, + "batch_size": 32, + "eval_batch_size": 2, + "epochs": 10, + "learning_rate": 0.0002, + "weight_decay": 0.0001, + "base_channels": 32, + "dropout": 0.1, + "norm_type": "group", + "norm_groups": 8, + "init_head_bias_from_positive_rate": true, + "augment_flip": true, + "threshold": 0.5, + "metric_thresholds": [ + 0.1, + 0.2, + 0.3, + 0.5, + 0.7, + 0.9 + ], + "topk_area_fractions": [ + 0.01, + 0.05, + 0.1 + ], + "fss_radii": [ + 1, + 2, + 4, + 8 + ], + "reliability_bins": 10, + "amp": true, + "seed": 99, + "boundary_radii": [ + 1, + 2, + 4 + ], + "coarsen_factors": [ + 2, + 4, + 8 + ], + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "train_target_mode": "dilate_max", + "train_target_radius": 2, + "loss_type": "bce", + "init_checkpoint": "" +} diff --git a/models/wildfire_fm/eval_only_manifest.json b/models/wildfire_fm/eval_only_manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..33afdd1012d955f22cdfb19578e016bfbece6e96 --- /dev/null +++ b/models/wildfire_fm/eval_only_manifest.json @@ -0,0 +1,17 @@ +{ + "launched_at": "2026-05-04T00:25:21", + "purpose": "Eval-only repair for Reference Table 2 seeded checkpoints.", + "policy": "Reuse completed Reference checkpoints; only fire-prone readout summaries are generated.", + "job_id": "31780571", + "seeds": [ + 1, + 7, + 42, + 99, + 123 + ], + "run_base": "", + "train_root": "", + "output_root": "", + "evaluator": "" +} diff --git a/models/wildfire_fm/manifest.json b/models/wildfire_fm/manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..c9a800dd7aeea10082c92eb09893f0f03357a6c2 --- /dev/null +++ b/models/wildfire_fm/manifest.json @@ -0,0 +1,19 @@ +{ + "launched_at": "2026-05-04T00:17:48", + "purpose": "Reference-only repair for Table 2: train and evaluate true seeded reference checkpoints.", + "policy": "Only Reference is rerun; existing normal backbone rows are not touched.", + "job_id": "31779899", + "seeds": [ + 1, + 7, + 42, + 99, + 123 + ], + "run_base": "", + "train_root": "", + "output_root": "", + "base_config": "", + "trainer": "", + "evaluator": "" +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/history.json b/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/history.json new file mode 100644 index 0000000000000000000000000000000000000000..c1c5d03fbd7dd5bdc1439995f226cd92eb65fab7 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/history.json @@ -0,0 +1,1682 @@ +[ + { + "epoch": 1, + "train_loss": 0.5734782497771297, + "train_main_loss": 0.5351956177899154, + "train_aux_spatial_loss": 0.07656526550555741, + "val_loss": 0.37402819047371544, + "val_pr_auc": 0.003734718829113345, + "val_auroc": 0.8395962406652433, + "val_brier": 0.11044556647539139, + "val_brier_skill_score": -473.5761171163099, + "val_ece": 0.22885958394596995, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0009322534799702475, + "val_positive_recall": 0.5967056323060573, + "val_far": 0.9990677465200297, + "val_csi": 0.0009316664551128411, + "val_f1": 0.0018615985213304707, + "val_f2": 0.004632318485714474, + "val_frequency_bias": 640.068012752391, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1820, + "fp": 4753782, + "fn": 62, + "precision": 0.00038270654272582106, + "recall": 0.9670563230605739, + "far": 0.9996172934572741, + "csi": 0.0003827015533477554, + "f1": 0.000765110297796062, + "f2": 0.0019105084261819435, + "frequency_bias": 2526.887353878852, + "predicted_positive_rate": 0.5882006184291898 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1674, + "fp": 3286292, + "fn": 208, + "precision": 0.0005091293523108207, + "recall": 0.8894792773645058, + "far": 0.9994908706476892, + "csi": 0.0005090971463188992, + "f1": 0.0010176761965902375, + "f2": 0.0025398316610499064, + "frequency_bias": 1747.0595111583423, + "predicted_positive_rate": 0.4066748299319728 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1532, + "fp": 2265579, + "fn": 350, + "precision": 0.0006757498860885065, + "recall": 0.8140276301806588, + "far": 0.9993242501139115, + "csi": 0.0006756455789096262, + "f1": 0.0013503787803664444, + "f2": 0.00336756733705876, + "frequency_bias": 1204.6285866099893, + "predicted_positive_rate": 0.28040952380952383 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1123, + "fp": 1203485, + "fn": 759, + "precision": 0.0009322534799702475, + "recall": 0.5967056323060573, + "far": 0.9990677465200297, + "csi": 0.0009316664551128411, + "f1": 0.0018615985213304707, + "f2": 0.004632318485714474, + "frequency_bias": 640.068012752391, + "predicted_positive_rate": 0.14899294990723563 + }, + "0.7000": { + "threshold": 0.7, + "tp": 749, + "fp": 580251, + "fn": 1133, + "precision": 0.001289156626506024, + "recall": 0.39798087141339, + "far": 0.998710843373494, + "csi": 0.0012866475530505916, + "f1": 0.0025699884367676473, + "f2": 0.006363333605198053, + "frequency_bias": 308.7141339001063, + "predicted_positive_rate": 0.07186147186147186 + }, + "0.9000": { + "threshold": 0.9, + "tp": 451, + "fp": 151874, + "fn": 1431, + "precision": 0.002960774659445265, + "recall": 0.23963868225292242, + "far": 0.9970392253405548, + "csi": 0.002933218866255626, + "f1": 0.005849280512557796, + "f2": 0.014106710540308908, + "frequency_bias": 80.93783209351753, + "predicted_positive_rate": 0.0188404452690167 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0040197898633778095, + "recall": 0.17268863320350647, + "lift": 17.268863741956263 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0016227582236751914, + "recall": 0.34856536984443665, + "lift": 6.971307357653297 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0011020408710464835, + "recall": 0.4734325110912323, + "lift": 4.7343254963524055 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007940178513994312, + "2": 0.0008116663451843698, + "4": 0.0008402116520870306, + "8": 0.0008768156583047748 + }, + "0.2000": { + "1": 0.0010809907649410588, + "2": 0.0011167526746238554, + "4": 0.001173494894416982, + "8": 0.0012204541855455053 + }, + "0.3000": { + "1": 0.0014536022725181175, + "2": 0.0015044478327594923, + "4": 0.001584095073473124, + "8": 0.001635036482863117 + }, + "0.5000": { + "1": 0.0020614198969559627, + "2": 0.0021684921567577398, + "4": 0.002317489707137632, + "8": 0.002385432991275249 + }, + "0.7000": { + "1": 0.0030144855102398616, + "2": 0.003272350672034019, + "4": 0.003663210167958564, + "8": 0.003844188110136648 + }, + "0.9000": { + "1": 0.007369068053339922, + "2": 0.008060652520539557, + "4": 0.009407819114658267, + "8": 0.01158882476665346 + } + }, + "minutes": 0.3749832232793172 + }, + { + "epoch": 2, + "train_loss": 0.5280527199025703, + "train_main_loss": 0.49724071630821165, + "train_aux_spatial_loss": 0.061624006741304034, + "val_loss": 0.3316249119738738, + "val_pr_auc": 0.001767264439689972, + "val_auroc": 0.8436037224883574, + "val_brier": 0.08831143379211426, + "val_brier_skill_score": -378.46744883936856, + "val_ece": 0.20197589957727682, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0012098819434426726, + "val_positive_recall": 0.5526036131774708, + "val_far": 0.9987901180565574, + "val_csi": 0.0012086979765930988, + "val_f1": 0.0024144775790219046, + "val_f2": 0.005996890842747683, + "val_frequency_bias": 456.74176408076517, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1785, + "fp": 4455552, + "fn": 97, + "precision": 0.000400463325972436, + "recall": 0.9484590860786397, + "far": 0.9995995366740276, + "csi": 0.0004004546113301958, + "f1": 0.0008005886232544309, + "f2": 0.0019989406174654775, + "frequency_bias": 2368.40435706695, + "predicted_positive_rate": 0.5513094619666048 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1586, + "fp": 2764970, + "fn": 296, + "precision": 0.0005732759430859163, + "recall": 0.842720510095643, + "far": 0.9994267240569141, + "csi": 0.0005732146135752834, + "f1": 0.0011457724536363106, + "f2": 0.0028586012536029916, + "frequency_bias": 1470.008501594049, + "predicted_positive_rate": 0.34218379715522573 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1385, + "fp": 1741264, + "fn": 497, + "precision": 0.000794767047179323, + "recall": 0.7359192348565357, + "far": 0.9992052329528207, + "csi": 0.0007945404458375833, + "f1": 0.0015878193050166494, + "f2": 0.003956742660885157, + "frequency_bias": 925.9558979808714, + "predicted_positive_rate": 0.21554100185528757 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1040, + "fp": 858548, + "fn": 842, + "precision": 0.0012098819434426726, + "recall": 0.5526036131774708, + "far": 0.9987901180565574, + "csi": 0.0012086979765930988, + "f1": 0.0024144775790219046, + "f2": 0.005996890842747683, + "frequency_bias": 456.74176408076517, + "predicted_positive_rate": 0.10631886209029066 + }, + "0.7000": { + "threshold": 0.7, + "tp": 786, + "fp": 438309, + "fn": 1096, + "precision": 0.0017900454343593073, + "recall": 0.41764080765143463, + "far": 0.9982099545656407, + "csi": 0.0017855885286159872, + "f1": 0.0035648117702283795, + "f2": 0.008799367699379567, + "frequency_bias": 233.31296493092455, + "predicted_positive_rate": 0.05430983302411874 + }, + "0.9000": { + "threshold": 0.9, + "tp": 490, + "fp": 167018, + "fn": 1392, + "precision": 0.0029252334216873224, + "recall": 0.2603613177470776, + "far": 0.9970747665783127, + "csi": 0.002901124925991711, + "f1": 0.005785465493830804, + "f2": 0.01399712059233529, + "frequency_bias": 89.00531349628055, + "predicted_positive_rate": 0.020718367346938774 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0032776747830212116, + "recall": 0.14080765843391418, + "lift": 14.080765697258915 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018528138753026724, + "recall": 0.39798086881637573, + "lift": 7.959617651486319 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012665429385378957, + "recall": 0.5441020131111145, + "lift": 5.4410200961522595 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0008428515962385363, + "2": 0.0008708457132048375, + "4": 0.0009155129443840782, + "8": 0.0009615746721725049 + }, + "0.2000": { + "1": 0.0012411931023150746, + "2": 0.001295326160666077, + "4": 0.0013845295073414698, + "8": 0.0014592629037508686 + }, + "0.3000": { + "1": 0.0017602573186886739, + "2": 0.0018528354423298365, + "4": 0.0020113329115427625, + "8": 0.0021305195762054696 + }, + "0.5000": { + "1": 0.0027948559926561645, + "2": 0.0030474081604696668, + "4": 0.0034677294952394355, + "8": 0.0037623477486223855 + }, + "0.7000": { + "1": 0.004259588970479955, + "2": 0.004814858068874828, + "4": 0.005858539237715177, + "8": 0.0067267307965462875 + }, + "0.9000": { + "1": 0.007704160826285555, + "2": 0.009190232441330881, + "4": 0.01342071380106813, + "8": 0.02008413371433937 + } + }, + "minutes": 0.34824419021606445 + }, + { + "epoch": 3, + "train_loss": 0.5097358830169797, + "train_main_loss": 0.48056654829095874, + "train_aux_spatial_loss": 0.05833866919872711, + "val_loss": 0.23263437176744142, + "val_pr_auc": 0.005799829091682691, + "val_auroc": 0.858433675989069, + "val_brier": 0.05747459828853607, + "val_brier_skill_score": -245.96393489611552, + "val_ece": 0.1401309918099933, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0016207128199668883, + "val_positive_recall": 0.4691817215727949, + "val_far": 0.9983792871800331, + "val_csi": 0.0016177464773249838, + "val_f1": 0.003230267201264304, + "val_f2": 0.007993120304154975, + "val_frequency_bias": 289.4909670563231, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1708, + "fp": 3257028, + "fn": 174, + "precision": 0.0005241296011705152, + "recall": 0.9075451647183846, + "far": 0.9994758703988295, + "csi": 0.0005241016167982546, + "f1": 0.001047654156359316, + "f2": 0.002614608004741809, + "frequency_bias": 1731.528161530287, + "predicted_positive_rate": 0.4030594928880643 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1407, + "fp": 1647128, + "fn": 475, + "precision": 0.0008534850640113798, + "recall": 0.7476089266737513, + "far": 0.9991465149359886, + "csi": 0.0008532392162570269, + "f1": 0.0017050236394802042, + "f2": 0.004248026796082034, + "frequency_bias": 875.9484590860786, + "predicted_positive_rate": 0.2039004329004329 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1191, + "fp": 1076231, + "fn": 691, + "precision": 0.0011054164477799786, + "recall": 0.6328374070138151, + "far": 0.99889458355222, + "csi": 0.0011047079480536828, + "f1": 0.0022069778301572127, + "f2": 0.005488732199640537, + "frequency_bias": 572.4877789585547, + "predicted_positive_rate": 0.13326184291898577 + }, + "0.5000": { + "threshold": 0.5, + "tp": 883, + "fp": 543939, + "fn": 999, + "precision": 0.0016207128199668883, + "recall": 0.4691817215727949, + "far": 0.9983792871800331, + "csi": 0.0016177464773249838, + "f1": 0.003230267201264304, + "f2": 0.007993120304154975, + "frequency_bias": 289.4909670563231, + "predicted_positive_rate": 0.06738676561533705 + }, + "0.7000": { + "threshold": 0.7, + "tp": 672, + "fp": 275187, + "fn": 1210, + "precision": 0.0024360271008014964, + "recall": 0.357066950053135, + "far": 0.9975639728991985, + "csi": 0.002425388621606892, + "f1": 0.004839040688987222, + "f2": 0.011856577754095989, + "frequency_bias": 146.57757704569607, + "predicted_positive_rate": 0.03411985157699443 + }, + "0.9000": { + "threshold": 0.9, + "tp": 509, + "fp": 128114, + "fn": 1373, + "precision": 0.003957301571258639, + "recall": 0.27045696068012753, + "far": 0.9960426984287414, + "csi": 0.003915505092464384, + "f1": 0.007800467415041569, + "f2": 0.018692481142261165, + "frequency_bias": 68.34378320935176, + "predicted_positive_rate": 0.015908843537414966 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004959802143275738, + "recall": 0.21307119727134705, + "lift": 21.307120598418535 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0019146567210555077, + "recall": 0.4112645983695984, + "lift": 8.225292155133914 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012863327283412218, + "recall": 0.5526036024093628, + "lift": 5.526036277398218 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0011067289165310745, + "2": 0.0011399809349705858, + "4": 0.0011913783552172097, + "8": 0.0012370094558790123 + }, + "0.2000": { + "1": 0.0018505622473496963, + "2": 0.0019252785696999286, + "4": 0.002041576664051692, + "8": 0.002124135134588423 + }, + "0.3000": { + "1": 0.00242666478591258, + "2": 0.0025485806202928396, + "4": 0.0027490318766942856, + "8": 0.0029084132219412417 + }, + "0.5000": { + "1": 0.003769582021368678, + "2": 0.0041531729728947475, + "4": 0.004731106915223382, + "8": 0.005255388249188031 + }, + "0.7000": { + "1": 0.006126287350098902, + "2": 0.0069839388676512515, + "4": 0.008523715958140055, + "8": 0.01053433503604162 + }, + "0.9000": { + "1": 0.009785190427335055, + "2": 0.011164824863941325, + "4": 0.014373278762976782, + "8": 0.019307251722885388 + } + }, + "minutes": 0.35075352191925047 + }, + { + "epoch": 4, + "train_loss": 0.49218875483820523, + "train_main_loss": 0.46398333852538587, + "train_aux_spatial_loss": 0.05641083258067807, + "val_loss": 0.4577095299959183, + "val_pr_auc": 0.0025830442889536323, + "val_auroc": 0.8551343969226862, + "val_brier": 0.1315883994102478, + "val_brier_skill_score": -564.4252465043933, + "val_ece": 0.2563538164779148, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0009896558085788286, + "val_positive_recall": 0.7353878852284803, + "val_far": 0.9990103441914212, + "val_csi": 0.0009893035131711752, + "val_f1": 0.0019766515180512276, + "val_f2": 0.004921784872481676, + "val_frequency_bias": 743.0743889479278, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1811, + "fp": 4984120, + "fn": 71, + "precision": 0.00036322203415971863, + "recall": 0.9622741764080766, + "far": 0.9996367779658403, + "csi": 0.00036321686192664987, + "f1": 0.0007261699666767779, + "f2": 0.0018133722535821364, + "frequency_bias": 2649.2725823591923, + "predicted_positive_rate": 0.6166890538033395 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1689, + "fp": 3529525, + "fn": 193, + "precision": 0.0004783057611348392, + "recall": 0.8974495217853348, + "far": 0.9995216942388652, + "csi": 0.0004782796205591709, + "f1": 0.0009561019570371142, + "f2": 0.002386441283371322, + "frequency_bias": 1876.3092454835282, + "predicted_positive_rate": 0.4367611626468769 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1564, + "fp": 2582550, + "fn": 318, + "precision": 0.0006052364562863713, + "recall": 0.8310308182784272, + "far": 0.9993947635437136, + "csi": 0.0006051619853027667, + "f1": 0.001209591971526638, + "f2": 0.003017392062638281, + "frequency_bias": 1373.068012752391, + "predicted_positive_rate": 0.3196183055040198 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1384, + "fp": 1397082, + "fn": 498, + "precision": 0.0009896558085788286, + "recall": 0.7353878852284803, + "far": 0.9990103441914212, + "csi": 0.0009893035131711752, + "f1": 0.0019766515180512276, + "f2": 0.004921784872481676, + "frequency_bias": 743.0743889479278, + "predicted_positive_rate": 0.1729704390847248 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1057, + "fp": 755716, + "fn": 825, + "precision": 0.0013967200204024193, + "recall": 0.5616365568544102, + "far": 0.9986032799795976, + "csi": 0.001395199036956275, + "f1": 0.0027865103373733782, + "f2": 0.006914814974728543, + "frequency_bias": 402.11105207226353, + "predicted_positive_rate": 0.09360210265924551 + }, + "0.9000": { + "threshold": 0.9, + "tp": 696, + "fp": 261226, + "fn": 1186, + "precision": 0.0026572796481395223, + "recall": 0.3698193411264612, + "far": 0.9973427203518604, + "csi": 0.0026453015491737235, + "f1": 0.0052766447817318915, + "f2": 0.012915197624791243, + "frequency_bias": 139.1721572794899, + "predicted_positive_rate": 0.03239604205318491 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004947433713823557, + "recall": 0.21253985166549683, + "lift": 21.253986297827947 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0019517624750733376, + "recall": 0.41923484206199646, + "lift": 8.384697057368781 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0013469387777149677, + "recall": 0.5786397457122803, + "lift": 5.7863975510330645 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007600908508323956, + "2": 0.0007824004837565512, + "4": 0.0008174067935393659, + "8": 0.0008576593024997026 + }, + "0.2000": { + "1": 0.0010137899344645485, + "2": 0.001052954821482821, + "4": 0.001117004853520176, + "8": 0.0011786437603573674 + }, + "0.3000": { + "1": 0.0013153040680959904, + "2": 0.0013806229283441773, + "4": 0.0014748630018216469, + "8": 0.0015478778504813828 + }, + "0.5000": { + "1": 0.002183108279669388, + "2": 0.0022919188759134013, + "4": 0.0024628992277530015, + "8": 0.0025537066150409515 + }, + "0.7000": { + "1": 0.003187314180517764, + "2": 0.003451823023046807, + "4": 0.003854865378136174, + "8": 0.004092752948271272 + }, + "0.9000": { + "1": 0.006778339261959476, + "2": 0.008030481875325624, + "4": 0.010576859714348186, + "8": 0.013750931457852666 + } + }, + "minutes": 0.3167848944664001 + }, + { + "epoch": 5, + "train_loss": 0.4798422370943648, + "train_main_loss": 0.4526658858268445, + "train_aux_spatial_loss": 0.05435270264757918, + "val_loss": 0.22279856478174528, + "val_pr_auc": 0.002003247222421217, + "val_auroc": 0.8534472608933964, + "val_brier": 0.0562748983502388, + "val_brier_skill_score": -240.80891639613208, + "val_ece": 0.13882232630158547, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0015668106718637735, + "val_positive_recall": 0.45536663124335813, + "val_far": 0.9984331893281362, + "val_csi": 0.0015638800283213747, + "val_f1": 0.003122876252840014, + "val_f2": 0.007727696533266967, + "val_frequency_bias": 290.6328374070138, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1660, + "fp": 3252099, + "fn": 222, + "precision": 0.0005101791497157596, + "recall": 0.8820403825717322, + "far": 0.9994898208502843, + "csi": 0.0005101443431906947, + "f1": 0.0010197684572715482, + "f2": 0.00254500753843498, + "frequency_bias": 1728.883634431456, + "predicted_positive_rate": 0.4024439084724799 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1427, + "fp": 1717127, + "fn": 455, + "precision": 0.0008303492354619058, + "recall": 0.7582359192348566, + "far": 0.9991696507645381, + "csi": 0.0008301294524926862, + "f1": 0.0016588818183297725, + "f2": 0.004133639073925804, + "frequency_bias": 913.15302869288, + "predicted_positive_rate": 0.21256079158936303 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1228, + "fp": 1043050, + "fn": 654, + "precision": 0.0011759320793888217, + "recall": 0.6524973432518597, + "far": 0.9988240679206112, + "csi": 0.0011751960893149028, + "f1": 0.002347633249216181, + "f2": 0.005837578412749118, + "frequency_bias": 554.8767268862912, + "predicted_positive_rate": 0.12916239950525665 + }, + "0.5000": { + "threshold": 0.5, + "tp": 857, + "fp": 546114, + "fn": 1025, + "precision": 0.0015668106718637735, + "recall": 0.45536663124335813, + "far": 0.9984331893281362, + "csi": 0.0015638800283213747, + "f1": 0.003122876252840014, + "f2": 0.007727696533266967, + "frequency_bias": 290.6328374070138, + "predicted_positive_rate": 0.0676525664811379 + }, + "0.7000": { + "threshold": 0.7, + "tp": 663, + "fp": 274830, + "fn": 1219, + "precision": 0.00240659472291492, + "recall": 0.3522848034006376, + "far": 0.9975934052770851, + "csi": 0.0023959929457341928, + "f1": 0.004780531771068049, + "f2": 0.011712911762731388, + "frequency_bias": 146.38310308182784, + "predicted_positive_rate": 0.03407458256029684 + }, + "0.9000": { + "threshold": 0.9, + "tp": 351, + "fp": 95622, + "fn": 1531, + "precision": 0.0036572786096089526, + "recall": 0.18650371944739638, + "far": 0.996342721390391, + "csi": 0.003599852313751231, + "f1": 0.0071738797199938685, + "f2": 0.016956357909585416, + "frequency_bias": 50.995217853347505, + "predicted_positive_rate": 0.011870500927643786 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0037229438312351704, + "recall": 0.15993623435497284, + "lift": 15.993624524077324 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001907235593535006, + "recall": 0.40967056155204773, + "lift": 8.193411274710096 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.00128262210637331, + "recall": 0.5510095357894897, + "lift": 5.51009558712842 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010862218732200501, + "2": 0.0011306068777684475, + "4": 0.0011982156126056909, + "8": 0.0012650951550728928 + }, + "0.2000": { + "1": 0.001835039598222954, + "2": 0.0019328571658292493, + "4": 0.002076142578762763, + "8": 0.0021710459616259836 + }, + "0.3000": { + "1": 0.002590627468139206, + "2": 0.0027374592974450662, + "4": 0.0029640942360077416, + "8": 0.0031140683173659367 + }, + "0.5000": { + "1": 0.0036841942890660784, + "2": 0.003975892870806685, + "4": 0.004437908744343022, + "8": 0.004817028530572198 + }, + "0.7000": { + "1": 0.005823401546553808, + "2": 0.0063452492211902856, + "4": 0.007432855129457017, + "8": 0.008860312688528271 + }, + "0.9000": { + "1": 0.00951053387465195, + "2": 0.011239285109680752, + "4": 0.016969728658484562, + "8": 0.028380582988892455 + } + }, + "minutes": 0.3421301166216532 + }, + { + "epoch": 6, + "train_loss": 0.47200144252198706, + "train_main_loss": 0.4454874005446253, + "train_aux_spatial_loss": 0.053028083716555464, + "val_loss": 0.20492182870705922, + "val_pr_auc": 0.006023304112441664, + "val_auroc": 0.8747266650018508, + "val_brier": 0.049230922013521194, + "val_brier_skill_score": -210.54149104245388, + "val_ece": 0.12249408376988412, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0018782184614922284, + "val_positive_recall": 0.46493092454835283, + "val_far": 0.9981217815385077, + "val_csi": 0.001874167334227222, + "val_f1": 0.0037413228034693822, + "val_f2": 0.009241753715184991, + "val_frequency_bias": 247.53825717322, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1672, + "fp": 2715974, + "fn": 210, + "precision": 0.0006152383349413426, + "recall": 0.8884165781083954, + "far": 0.9993847616650586, + "csi": 0.0006151907974521093, + "f1": 0.0012296251408332622, + "f2": 0.0030676940261429174, + "frequency_bias": 1444.020191285866, + "predicted_positive_rate": 0.3361343228200371 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1451, + "fp": 1391802, + "fn": 431, + "precision": 0.0010414476049934937, + "recall": 0.7709883103081828, + "far": 0.9989585523950065, + "csi": 0.0010411255349132228, + "f1": 0.00208008543976031, + "f2": 0.005179253573542188, + "frequency_bias": 740.3044633368756, + "predicted_positive_rate": 0.1723256648113791 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1254, + "fp": 892851, + "fn": 628, + "precision": 0.0014025198382740282, + "recall": 0.6663124335812965, + "far": 0.998597480161726, + "csi": 0.0014015354301227293, + "f1": 0.0027991477554919885, + "f2": 0.006954048931217025, + "frequency_bias": 475.0823591923486, + "predicted_positive_rate": 0.11058812615955473 + }, + "0.5000": { + "threshold": 0.5, + "tp": 875, + "fp": 464992, + "fn": 1007, + "precision": 0.0018782184614922284, + "recall": 0.46493092454835283, + "far": 0.9981217815385077, + "csi": 0.001874167334227222, + "f1": 0.0037413228034693822, + "f2": 0.009241753715184991, + "frequency_bias": 247.53825717322, + "predicted_positive_rate": 0.057621150278293135 + }, + "0.7000": { + "threshold": 0.7, + "tp": 665, + "fp": 253520, + "fn": 1217, + "precision": 0.0026162047327733737, + "recall": 0.35334750265674814, + "far": 0.9973837952672266, + "csi": 0.002603738420215973, + "f1": 0.005193953145075312, + "f2": 0.012704756737342051, + "frequency_bias": 135.06110520722635, + "predicted_positive_rate": 0.03143908472479901 + }, + "0.9000": { + "threshold": 0.9, + "tp": 437, + "fp": 108236, + "fn": 1445, + "precision": 0.004021238026004619, + "recall": 0.23219978746014877, + "far": 0.9959787619739954, + "csi": 0.00396847018652718, + "f1": 0.007905567364660124, + "f2": 0.018803624753659605, + "frequency_bias": 57.74335812964931, + "predicted_positive_rate": 0.013441311069882498 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0047000618651509285, + "recall": 0.20191286504268646, + "lift": 20.191286282774463 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0020086579024791718, + "recall": 0.43145591020584106, + "lift": 8.62911764073383 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014755720039829612, + "recall": 0.6339001059532166, + "lift": 6.339000978726575 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0013248742601514873, + "2": 0.00138582233525264, + "4": 0.0014793932080273686, + "8": 0.0015689502796606414 + }, + "0.2000": { + "1": 0.0023088385065342942, + "2": 0.002449153490201028, + "4": 0.002681505121324146, + "8": 0.0028831512044252827 + }, + "0.3000": { + "1": 0.0031503711680238577, + "2": 0.0033971130498923197, + "4": 0.003843801518220169, + "8": 0.00423863352271503 + }, + "0.5000": { + "1": 0.004490733019564042, + "2": 0.00513267474781276, + "4": 0.006267791721161653, + "8": 0.007324417037690489 + }, + "0.7000": { + "1": 0.006796398619910549, + "2": 0.008020939278983064, + "4": 0.010531110485300088, + "8": 0.013871225077630944 + }, + "0.9000": { + "1": 0.011415450466378263, + "2": 0.013839835459906191, + "4": 0.019889957919249968, + "8": 0.0312791122896422 + } + }, + "minutes": 0.3271060546239217 + }, + { + "epoch": 7, + "train_loss": 0.459911651689757, + "train_main_loss": 0.43420963327302686, + "train_aux_spatial_loss": 0.05140403612300002, + "val_loss": 0.30195711702108385, + "val_pr_auc": 0.007462668847578005, + "val_auroc": 0.8774544235682473, + "val_brier": 0.07583270967006683, + "val_brier_skill_score": -324.84732963135656, + "val_ece": 0.15615654503375537, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0015267438380724627, + "val_positive_recall": 0.6126461211477152, + "val_far": 0.9984732561619275, + "val_csi": 0.001525271486418734, + "val_f1": 0.0030458971527597993, + "val_f2": 0.00755837583417461, + "val_frequency_bias": 401.2763018065887, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1722, + "fp": 3083597, + "fn": 160, + "precision": 0.0005581270526645705, + "recall": 0.9149840595111584, + "far": 0.9994418729473354, + "csi": 0.000558098110536484, + "f1": 0.0011155736215426207, + "f2": 0.0027838428477063365, + "frequency_bias": 1639.3831030818278, + "predicted_positive_rate": 0.38161026592455166 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1535, + "fp": 1830430, + "fn": 347, + "precision": 0.0008378981039484925, + "recall": 0.8156216790648246, + "far": 0.9991621018960515, + "csi": 0.0008377394242901864, + "f1": 0.001674076408773469, + "f2": 0.004172345314714435, + "frequency_bias": 973.413921360255, + "predicted_positive_rate": 0.22658812615955473 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1416, + "fp": 1303639, + "fn": 466, + "precision": 0.0010850117428001119, + "recall": 0.7523910733262487, + "far": 0.9989149882571999, + "csi": 0.0010846244526131713, + "f1": 0.002166898633981592, + "f2": 0.005393944611502663, + "frequency_bias": 693.4404888416578, + "predicted_positive_rate": 0.16141682127396412 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1153, + "fp": 754049, + "fn": 729, + "precision": 0.0015267438380724627, + "recall": 0.6126461211477152, + "far": 0.9984732561619275, + "csi": 0.001525271486418734, + "f1": 0.0030458971527597993, + "f2": 0.00755837583417461, + "frequency_bias": 401.2763018065887, + "predicted_positive_rate": 0.0934077922077922 + }, + "0.7000": { + "threshold": 0.7, + "tp": 959, + "fp": 448284, + "fn": 923, + "precision": 0.002134702154513259, + "recall": 0.5095642933049946, + "far": 0.9978652978454867, + "csi": 0.0021303252577937917, + "f1": 0.004251593239124411, + "f2": 0.010497601642836345, + "frequency_bias": 238.70510095642933, + "predicted_positive_rate": 0.05556499690785405 + }, + "0.9000": { + "threshold": 0.9, + "tp": 660, + "fp": 227755, + "fn": 1222, + "precision": 0.002889477486154587, + "recall": 0.35069075451647186, + "far": 0.9971105225138455, + "csi": 0.002874101299006693, + "f1": 0.005731729028167975, + "f2": 0.013986428925630344, + "frequency_bias": 121.3682252922423, + "predicted_positive_rate": 0.028251700680272107 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004168212879449129, + "recall": 0.1790648251771927, + "lift": 17.906483350895734 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0022708720061928034, + "recall": 0.4877789616584778, + "lift": 9.755579416634951 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014594928361475468, + "recall": 0.6269925832748413, + "lift": 6.269925487750414 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0011922256706897727, + "2": 0.001241923098517117, + "4": 0.0013199747764957026, + "8": 0.0014018572289743458 + }, + "0.2000": { + "1": 0.0018412087795650622, + "2": 0.0019516962520630399, + "4": 0.0021122719154029745, + "8": 0.002258269200665519 + }, + "0.3000": { + "1": 0.0023970400348517495, + "2": 0.0025539329427168855, + "4": 0.0027957403234514233, + "8": 0.0029940557272635226 + }, + "0.5000": { + "1": 0.003458294529205541, + "2": 0.003808782182815551, + "4": 0.00435669994126564, + "8": 0.004792095791161022 + }, + "0.7000": { + "1": 0.005004234974898769, + "2": 0.005603576851424319, + "4": 0.006716963044160917, + "8": 0.007943951762853563 + }, + "0.9000": { + "1": 0.007190456588270111, + "2": 0.00818745081073502, + "4": 0.010293219734911818, + "8": 0.0139886851310419 + } + }, + "minutes": 0.32838242053985595 + }, + { + "epoch": 8, + "train_loss": 0.4499193408563182, + "train_main_loss": 0.4247753525127238, + "train_aux_spatial_loss": 0.05028797703851233, + "val_loss": 0.2679826237261295, + "val_pr_auc": 0.004651516864174585, + "val_auroc": 0.8589783508732233, + "val_brier": 0.07260610163211823, + "val_brier_skill_score": -310.98284269020706, + "val_ece": 0.16960450200447197, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.001397625988590239, + "val_positive_recall": 0.5324123273113709, + "val_far": 0.9986023740114097, + "val_csi": 0.0013959125673924854, + "val_f1": 0.0027879334234820786, + "val_f2": 0.006915514771042627, + "val_frequency_bias": 380.94048884165784, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1766, + "fp": 3901854, + "fn": 116, + "precision": 0.00045240059227076405, + "recall": 0.9383634431455898, + "far": 0.9995475994077292, + "csi": 0.0004523871491309863, + "f1": 0.0009043651750786454, + "f2": 0.0022576491608090514, + "frequency_bias": 2074.1870350690756, + "predicted_positive_rate": 0.4828225108225108 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1578, + "fp": 2284591, + "fn": 304, + "precision": 0.0006902376858403731, + "recall": 0.8384697130712009, + "far": 0.9993097623141596, + "csi": 0.0006901459146904424, + "f1": 0.001379339883595252, + "f2": 0.003439861498707109, + "frequency_bias": 1214.7550478214666, + "predicted_positive_rate": 0.2827667285095857 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1380, + "fp": 1443686, + "fn": 502, + "precision": 0.000954973682862928, + "recall": 0.7332624867162593, + "far": 0.9990450263171371, + "csi": 0.0009546420507371497, + "f1": 0.001907463156934458, + "f2": 0.004750122883613728, + "frequency_bias": 767.8352816153028, + "predicted_positive_rate": 0.17873419913419913 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1002, + "fp": 715928, + "fn": 880, + "precision": 0.001397625988590239, + "recall": 0.5324123273113709, + "far": 0.9986023740114097, + "csi": 0.0013959125673924854, + "f1": 0.0027879334234820786, + "f2": 0.006915514771042627, + "frequency_bias": 380.94048884165784, + "predicted_positive_rate": 0.08867408781694495 + }, + "0.7000": { + "threshold": 0.7, + "tp": 688, + "fp": 355882, + "fn": 1194, + "precision": 0.001929494909835376, + "recall": 0.3655685441020191, + "far": 0.9980705050901646, + "csi": 0.00192305542201004, + "f1": 0.0038387287558724744, + "f2": 0.00944800575669188, + "frequency_bias": 189.4633368756642, + "predicted_positive_rate": 0.04410265924551639 + }, + "0.9000": { + "threshold": 0.9, + "tp": 466, + "fp": 111220, + "fn": 1416, + "precision": 0.004172411940619236, + "recall": 0.24760892667375134, + "far": 0.9958275880593808, + "csi": 0.004120174709554208, + "f1": 0.008206537052690898, + "f2": 0.019544684349153626, + "frequency_bias": 59.34431455897981, + "predicted_positive_rate": 0.013813976499690786 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004873222205787897, + "recall": 0.20935174822807312, + "lift": 20.935176493358213 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001823129248805344, + "recall": 0.3916046619415283, + "lift": 7.83209362967527 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012888064375147223, + "recall": 0.5536662936210632, + "lift": 5.53666323753949 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0009531461813420128, + "2": 0.0009859296090429792, + "4": 0.0010304564828290674, + "8": 0.0010758667476578054 + }, + "0.2000": { + "1": 0.001509795785677337, + "2": 0.00158154075286987, + "4": 0.0016902515481980185, + "8": 0.0017673814967695822 + }, + "0.3000": { + "1": 0.0021042057052488294, + "2": 0.0022340194523658985, + "4": 0.0024437855961459976, + "8": 0.0025600611757777836 + }, + "0.5000": { + "1": 0.0031793578612200335, + "2": 0.0034648747296572857, + "4": 0.0039216381286044655, + "8": 0.004107196995981055 + }, + "0.7000": { + "1": 0.004753539211719615, + "2": 0.005295789241588578, + "4": 0.006140676468973316, + "8": 0.006471503504579701 + }, + "0.9000": { + "1": 0.011358743818156714, + "2": 0.013515668144941873, + "4": 0.017924796538675958, + "8": 0.02419228618876912 + } + }, + "minutes": 0.31785523096720375 + }, + { + "epoch": 9, + "train_loss": 0.43544893010666147, + "train_main_loss": 0.41090481131353246, + "train_aux_spatial_loss": 0.049088238095766304, + "val_loss": 0.35741244157155355, + "val_pr_auc": 0.002585840733061817, + "val_auroc": 0.8817129713606411, + "val_brier": 0.09716344624757767, + "val_brier_skill_score": -416.5038665412512, + "val_ece": 0.18796858609779482, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0012806769018722003, + "val_positive_recall": 0.7109458023379384, + "val_far": 0.9987193230981278, + "val_csi": 0.0012800104084553393, + "val_f1": 0.0025567481526634706, + "val_f2": 0.006357575112516725, + "val_frequency_bias": 555.1328374070138, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1758, + "fp": 3553508, + "fn": 124, + "precision": 0.0004944777690333157, + "recall": 0.9341126461211477, + "far": 0.9995055222309667, + "csi": 0.000494460523318117, + "f1": 0.0009884323058810035, + "f2": 0.0024671648150300014, + "frequency_bias": 1889.0892667375133, + "predicted_positive_rate": 0.4397360544217687 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1622, + "fp": 2315636, + "fn": 260, + "precision": 0.0006999652175113863, + "recall": 0.8618490967056323, + "far": 0.9993000347824886, + "csi": 0.0006998866891217241, + "f1": 0.0013987943806755953, + "f2": 0.003488493134421835, + "frequency_bias": 1231.2741764080765, + "predicted_positive_rate": 0.28661199752628325 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1531, + "fp": 1735553, + "fn": 351, + "precision": 0.0008813620987816363, + "recall": 0.8134962805526036, + "far": 0.9991186379012184, + "csi": 0.0008811840442951822, + "f1": 0.0017608164851986754, + "f2": 0.004387795108597212, + "frequency_bias": 922.9989373007439, + "predicted_positive_rate": 0.2148526901669759 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1338, + "fp": 1043422, + "fn": 544, + "precision": 0.0012806769018722003, + "recall": 0.7109458023379384, + "far": 0.9987193230981278, + "csi": 0.0012800104084553393, + "f1": 0.0025567481526634706, + "f2": 0.006357575112516725, + "frequency_bias": 555.1328374070138, + "predicted_positive_rate": 0.12922201607915892 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1104, + "fp": 583644, + "fn": 778, + "precision": 0.0018879927763754644, + "recall": 0.5866099893730075, + "far": 0.9981120072236246, + "csi": 0.0018854841629577508, + "f1": 0.0037638716056117147, + "f2": 0.009319979198887005, + "frequency_bias": 310.7056323060574, + "predicted_positive_rate": 0.07232504638218924 + }, + "0.9000": { + "threshold": 0.9, + "tp": 774, + "fp": 249875, + "fn": 1108, + "precision": 0.0030879835945884482, + "recall": 0.4112646121147715, + "far": 0.9969120164054116, + "csi": 0.003074393164837522, + "f1": 0.0061299404825546164, + "f2": 0.014989716357382724, + "frequency_bias": 133.18225292242295, + "predicted_positive_rate": 0.031001731601731603 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0047000618651509285, + "recall": 0.20191286504268646, + "lift": 20.191286282774463 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002372294431552291, + "recall": 0.5095642805099487, + "lift": 10.191286282774461 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015374149661511183, + "recall": 0.6604675650596619, + "lift": 6.604675982489947 + } + }, + "val_fss": { + "0.1000": { + "1": 0.001055759333565831, + "2": 0.0010976495838397327, + "4": 0.001157430376061197, + "8": 0.0012108268924492238 + }, + "0.2000": { + "1": 0.0015193550700235425, + "2": 0.0015891089547401815, + "4": 0.00168767572063655, + "8": 0.0017540287421734524 + }, + "0.3000": { + "1": 0.0019130861375810104, + "2": 0.00200484623513697, + "4": 0.0021368045034607652, + "8": 0.002225743866744012 + }, + "0.5000": { + "1": 0.0028767211501119494, + "2": 0.0030536082429117872, + "4": 0.0033209392323471087, + "8": 0.0035238602609450886 + }, + "0.7000": { + "1": 0.004462833938854716, + "2": 0.0048919187751156334, + "4": 0.005581633696340349, + "8": 0.006139927162454928 + }, + "0.9000": { + "1": 0.007988241741801017, + "2": 0.009144048690186368, + "4": 0.011405277940876535, + "8": 0.014317102888635747 + } + }, + "minutes": 0.34604887167612713 + }, + { + "epoch": 10, + "train_loss": 0.4286219271382876, + "train_main_loss": 0.404440414390365, + "train_aux_spatial_loss": 0.048363025166727865, + "val_loss": 0.42824133137861886, + "val_pr_auc": 0.003648303842240367, + "val_auroc": 0.8661930422066306, + "val_brier": 0.11499285697937012, + "val_brier_skill_score": -493.1154751878621, + "val_ece": 0.2244935538477554, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0011714322983281467, + "val_positive_recall": 0.7348565356004251, + "val_far": 0.9988285677016718, + "val_csi": 0.001170937384906507, + "val_f1": 0.0023391357882701557, + "val_f2": 0.005820050600353159, + "val_frequency_bias": 627.3145589798087, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1753, + "fp": 4504075, + "fn": 129, + "precision": 0.0003890516903885368, + "recall": 0.9314558979808714, + "far": 0.9996109483096115, + "csi": 0.0003890405523177429, + "f1": 0.0007777785172515536, + "f2": 0.0019420138805802158, + "frequency_bias": 2394.1700318809776, + "predicted_positive_rate": 0.5573071119356834 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1645, + "fp": 2894097, + "fn": 237, + "precision": 0.0005680754708119715, + "recall": 0.8740701381509033, + "far": 0.999431924529188, + "csi": 0.0005680289808731348, + "f1": 0.0011354130142489157, + "f2": 0.002833012430810775, + "frequency_bias": 1538.6514346439958, + "predicted_positive_rate": 0.3581622758194187 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1523, + "fp": 2045614, + "fn": 359, + "precision": 0.0007439658410746325, + "recall": 0.8092454835281615, + "far": 0.9992560341589254, + "csi": 0.0007438353969922286, + "f1": 0.00148656503429202, + "f2": 0.0037062002808243683, + "frequency_bias": 1087.7454835281615, + "predicted_positive_rate": 0.25320185528756955 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1383, + "fp": 1179223, + "fn": 499, + "precision": 0.0011714322983281467, + "recall": 0.7348565356004251, + "far": 0.9988285677016718, + "csi": 0.001170937384906507, + "f1": 0.0023391357882701557, + "f2": 0.005820050600353159, + "frequency_bias": 627.3145589798087, + "predicted_positive_rate": 0.14602424242424242 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1242, + "fp": 689422, + "fn": 640, + "precision": 0.0017982694913879977, + "recall": 0.6599362380446334, + "far": 0.998201730508612, + "csi": 0.0017966046775369448, + "f1": 0.003586765355658686, + "f2": 0.008894401539977542, + "frequency_bias": 366.98405951115836, + "predicted_positive_rate": 0.08542535559678417 + }, + "0.9000": { + "threshold": 0.9, + "tp": 930, + "fp": 302454, + "fn": 952, + "precision": 0.0030654220393956173, + "recall": 0.49415515409139216, + "far": 0.9969345779606044, + "csi": 0.00305583302665475, + "f1": 0.006093046719909849, + "f2": 0.014956000411692054, + "frequency_bias": 161.2029755579171, + "predicted_positive_rate": 0.03752430426716141 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005058750975877047, + "recall": 0.2173219919204712, + "lift": 21.73220100453255 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002555349376052618, + "recall": 0.5488841533660889, + "lift": 10.977683333691749 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001611626474186778, + "recall": 0.6923485398292542, + "lift": 6.923485786959681 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0008190413195137136, + "2": 0.0008461932775331027, + "4": 0.0008874666072198334, + "8": 0.0009354570390477379 + }, + "0.2000": { + "1": 0.0012240361098765362, + "2": 0.0012842780888633332, + "4": 0.0013672079974895723, + "8": 0.0014385782612603704 + }, + "0.3000": { + "1": 0.0016296457718533164, + "2": 0.00172595038714074, + "4": 0.0018654638060817685, + "8": 0.0019670592517716523 + }, + "0.5000": { + "1": 0.002614059790545964, + "2": 0.002783774308284359, + "4": 0.0030265840125089927, + "8": 0.0031953274262443987 + }, + "0.7000": { + "1": 0.004117212678526938, + "2": 0.00442237575814286, + "4": 0.00490498934248651, + "8": 0.005264629986009961 + }, + "0.9000": { + "1": 0.007561816117923059, + "2": 0.008478388329967612, + "4": 0.010294830494736118, + "8": 0.01224892406096545 + } + }, + "minutes": 0.31807684898376465 + } +] diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/run_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/run_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..531fa93f54789e5e0a619f468f4c794be8015e09 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/run_summary.json @@ -0,0 +1,172 @@ +{ + "run_name": "reference_fireprone_seed_123_20260504", + "device": "cuda", + "index_root": "", + "input_channels": 16, + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "train_positive_rate": 0.0018302686439820436, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight": 50.0, + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "aux_positive_rate": 0.004560296846011132, + "init_checkpoint": "", + "best_val_pr_auc": 0.007462668847578005, + "best_val_auroc": 0.8774544235682473, + "best_test_pr_auc": 0.001212019793975473, + "best_test_auroc": 0.8262438111464772, + "best_test_brier": 0.08898080885410309, + "best_test_ece": 0.17619957022100255, + "best_test_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0011750154662877321, + "recall": 0.04844466969370842, + "lift": 4.844467048285472 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001756338868290186, + "recall": 0.36206015944480896, + "lift": 7.241203215763695 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012789116008207202, + "recall": 0.5272819995880127, + "lift": 5.272820048420403 + } + }, + "best_test_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1706, + "fp": 3320161, + "fn": 255, + "precision": 0.0005135666178085998, + "recall": 0.8699643039265681, + "far": 0.9994864333821915, + "csi": 0.0005135271973756533, + "f1": 0.0010265272450921048, + "f2": 0.002561783890553865, + "frequency_bias": 1693.9658337582866, + "predicted_positive_rate": 0.4108679035250464 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1496, + "fp": 2134207, + "fn": 465, + "precision": 0.0007004719289152097, + "recall": 0.7628760836308006, + "far": 0.9992995280710848, + "csi": 0.0007003194505301081, + "f1": 0.0013996586928535072, + "f2": 0.0034895432663711134, + "frequency_bias": 1089.0887302396736, + "predicted_positive_rate": 0.2641562152133581 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1345, + "fp": 1593641, + "fn": 616, + "precision": 0.0008432675898095657, + "recall": 0.6858745537990821, + "far": 0.9991567324101904, + "csi": 0.0008429420369239948, + "f1": 0.0016844641681909294, + "f2": 0.004195703848817405, + "frequency_bias": 813.353391126976, + "predicted_positive_rate": 0.19727717996289426 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1088, + "fp": 912330, + "fn": 873, + "precision": 0.0011911304572495833, + "recall": 0.5548189699133096, + "far": 0.9988088695427504, + "csi": 0.001189993120352273, + "f1": 0.002377157439705302, + "f2": 0.005904943436286311, + "frequency_bias": 465.7919428862825, + "predicted_positive_rate": 0.11297687074829932 + }, + "0.7000": { + "threshold": 0.7, + "tp": 836, + "fp": 535422, + "fn": 1125, + "precision": 0.0015589511018949834, + "recall": 0.4263131055583886, + "far": 0.998441048898105, + "csi": 0.0015556874705749903, + "f1": 0.003106542132477672, + "f2": 0.007682383082583779, + "frequency_bias": 273.46149923508415, + "predicted_positive_rate": 0.06632752009894867 + }, + "0.9000": { + "threshold": 0.9, + "tp": 405, + "fp": 246171, + "fn": 1556, + "precision": 0.00164249562001168, + "recall": 0.2065272819989801, + "far": 0.9983575043799883, + "csi": 0.0016321957667693003, + "f1": 0.003259072089869919, + "f2": 0.007959279930823049, + "frequency_bias": 125.73992860785313, + "predicted_positive_rate": 0.03049795918367347 + } + }, + "best_test_fss": { + "0.1000": { + "1": 0.0010957041077256369, + "2": 0.0011377779586465575, + "4": 0.0011947677181146643, + "8": 0.0012470182839290356 + }, + "0.2000": { + "1": 0.0015170543049650709, + "2": 0.0015877971019127024, + "4": 0.0016814901428712226, + "8": 0.0017498239432793827 + }, + "0.3000": { + "1": 0.0018421386324615208, + "2": 0.0019416047187071905, + "4": 0.00207931527885874, + "8": 0.0021687368541809526 + }, + "0.5000": { + "1": 0.0026440385644092013, + "2": 0.002817713478884265, + "4": 0.0030958883338918497, + "8": 0.003316757768246803 + }, + "0.7000": { + "1": 0.0035326072867271074, + "2": 0.0038065323590337208, + "4": 0.004319262090524356, + "8": 0.004943475244678974 + }, + "0.9000": { + "1": 0.004119845404175648, + "2": 0.004748959668431807, + "4": 0.006481934958671998, + "8": 0.009729806335369129 + } + }, + "epochs": 10 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/tile_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/tile_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..6bc5fdb7a28709bfd7523073af81f9402c86dc03 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_123_20260504/tile_summary.json @@ -0,0 +1,16 @@ +{ + "index_root": "", + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "input_channels": 16, + "train_positive_rate": 0.0018302686439820436, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight_source": "tiles", + "positive_rate_source": "tiles", + "pos_weight": 50.0, + "loss_type": "bce", + "train_target_mode": "dilate_max", + "train_target_radius": 2 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/history.json b/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/history.json new file mode 100644 index 0000000000000000000000000000000000000000..487fbf8b6931bc6c4f359964467e4de8afb6b7cd --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/history.json @@ -0,0 +1,1682 @@ +[ + { + "epoch": 1, + "train_loss": 0.573664065749939, + "train_main_loss": 0.5326367103663546, + "train_aux_spatial_loss": 0.08205470970182109, + "val_loss": 0.5349265058835347, + "val_pr_auc": 0.0018654076712032604, + "val_auroc": 0.8500520368869484, + "val_brier": 0.16269078850746155, + "val_brier_skill_score": -698.0698238454416, + "val_ece": 0.2954815917108688, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0008273187647548224, + "val_positive_recall": 0.7545164718384697, + "val_far": 0.9991726812352452, + "val_csi": 0.0008270961353641844, + "val_f1": 0.001652825225372031, + "val_f2": 0.00411853013719926, + "val_frequency_bias": 912.0021253985122, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1857, + "fp": 5423702, + "fn": 25, + "precision": 0.00034226887957535803, + "recall": 0.9867162592986185, + "far": 0.9996577311204247, + "csi": 0.0003422673024691904, + "f1": 0.0006843003912893757, + "f2": 0.0017089731859622345, + "frequency_bias": 2882.8687566418703, + "predicted_positive_rate": 0.6710648113790971 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1800, + "fp": 4467036, + "fn": 82, + "precision": 0.000402789451212799, + "recall": 0.9564293304994687, + "far": 0.9995972105487873, + "csi": 0.00040278206044505626, + "f1": 0.0008052397847504584, + "f2": 0.002010560356575113, + "frequency_bias": 2374.5143464399575, + "predicted_positive_rate": 0.5527317254174398 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1720, + "fp": 3430554, + "fn": 162, + "precision": 0.000501125492894798, + "recall": 0.9139213602550478, + "far": 0.9994988745071052, + "csi": 0.0005011018413744641, + "f1": 0.001001701728168435, + "f2": 0.002500143903631663, + "frequency_bias": 1823.7375132837408, + "predicted_positive_rate": 0.42452368583797156 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1420, + "fp": 1714968, + "fn": 462, + "precision": 0.0008273187647548224, + "recall": 0.7545164718384697, + "far": 0.9991726812352452, + "csi": 0.0008270961353641844, + "f1": 0.001652825225372031, + "f2": 0.00411853013719926, + "frequency_bias": 912.0021253985122, + "predicted_positive_rate": 0.21229288806431665 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1050, + "fp": 918424, + "fn": 832, + "precision": 0.0011419572494708932, + "recall": 0.5579171094580234, + "far": 0.9988580427505291, + "csi": 0.0011409248662944717, + "f1": 0.002279249280408442, + "f2": 0.005663418201902477, + "frequency_bias": 488.56216790648244, + "predicted_positive_rate": 0.11372591218305504 + }, + "0.9000": { + "threshold": 0.9, + "tp": 610, + "fp": 339523, + "fn": 1272, + "precision": 0.0017934161048766219, + "recall": 0.3241232731137088, + "far": 0.9982065838951234, + "csi": 0.0017867342306058787, + "f1": 0.0035670950104527586, + "f2": 0.008772913844233319, + "frequency_bias": 180.72954303931988, + "predicted_positive_rate": 0.042069635126777985 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.002622139872983098, + "recall": 0.11264611780643463, + "lift": 11.264612757853444 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0016697588143870234, + "recall": 0.3586609959602356, + "lift": 7.173220100453255 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012207792606204748, + "recall": 0.5244420766830444, + "lift": 5.244421083480826 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0006955746753622982, + "2": 0.0007043072962146235, + "4": 0.0007184011183636319, + "8": 0.0007378939517806993 + }, + "0.2000": { + "1": 0.0008314270074428753, + "2": 0.0008458942856476304, + "4": 0.0008682692460694197, + "8": 0.0008963995358247656 + }, + "0.3000": { + "1": 0.0010519556378738582, + "2": 0.0010779076249621822, + "4": 0.001118951638737209, + "8": 0.0011639559236974462 + }, + "0.5000": { + "1": 0.001817472102352835, + "2": 0.0019079406476985028, + "4": 0.0020492128618442296, + "8": 0.002166019917211992 + }, + "0.7000": { + "1": 0.0025795801528006868, + "2": 0.0027239390264135466, + "4": 0.0029668123834800664, + "8": 0.003214106060204891 + }, + "0.9000": { + "1": 0.004266565643152309, + "2": 0.004592005457379367, + "4": 0.005288208913823755, + "8": 0.006419555152632372 + } + }, + "minutes": 0.39989219506581625 + }, + { + "epoch": 2, + "train_loss": 0.5310182435958991, + "train_main_loss": 0.5007337570216748, + "train_aux_spatial_loss": 0.06056897338257285, + "val_loss": 0.38492824087540306, + "val_pr_auc": 0.007534030208235136, + "val_auroc": 0.8668780863986516, + "val_brier": 0.11073322594165802, + "val_brier_skill_score": -474.81216775110823, + "val_ece": 0.24345293157139367, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0011833787160754057, + "val_positive_recall": 0.6849096705632306, + "val_far": 0.9988166212839246, + "val_csi": 0.0011827348242459722, + "val_f1": 0.002362675230218781, + "val_f2": 0.005876281704112576, + "val_frequency_bias": 578.7747077577046, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1846, + "fp": 5321276, + "fn": 36, + "precision": 0.00034678897083328166, + "recall": 0.9808714133900106, + "far": 0.9996532110291667, + "csi": 0.0003467866255331891, + "f1": 0.000693332812519953, + "f2": 0.0017314961590049992, + "frequency_bias": 2828.4388947927737, + "predicted_positive_rate": 0.6583948051948052 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1770, + "fp": 3730075, + "fn": 112, + "precision": 0.00047429622612943465, + "recall": 0.9404888416578109, + "far": 0.9995257037738705, + "csi": 0.0004742819919950846, + "f1": 0.0009481143104463716, + "f2": 0.002366706931884035, + "frequency_bias": 1982.914452709883, + "predicted_positive_rate": 0.46157637600494744 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1617, + "fp": 2614515, + "fn": 265, + "precision": 0.0006180880781244983, + "recall": 0.859192348565356, + "far": 0.9993819119218755, + "csi": 0.0006180254754916781, + "f1": 0.0012352875118314876, + "f2": 0.003081573069681285, + "frequency_bias": 1390.0807651434643, + "predicted_positive_rate": 0.3235784786641929 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1289, + "fp": 1087965, + "fn": 593, + "precision": 0.0011833787160754057, + "recall": 0.6849096705632306, + "far": 0.9988166212839246, + "csi": 0.0011827348242459722, + "f1": 0.002362675230218781, + "f2": 0.005876281704112576, + "frequency_bias": 578.7747077577046, + "predicted_positive_rate": 0.1347252937538652 + }, + "0.7000": { + "threshold": 0.7, + "tp": 895, + "fp": 459555, + "fn": 987, + "precision": 0.0019437506786838961, + "recall": 0.47555791710945805, + "far": 0.9980562493213161, + "csi": 0.0019395930538730097, + "f1": 0.003871676630646375, + "f2": 0.009562415327216238, + "frequency_bias": 244.65993623804462, + "predicted_positive_rate": 0.05695114409400124 + }, + "0.9000": { + "threshold": 0.9, + "tp": 579, + "fp": 168755, + "fn": 1303, + "precision": 0.0034192778768587524, + "recall": 0.30765143464399575, + "far": 0.9965807221231412, + "csi": 0.003393167953023084, + "f1": 0.006763386599383235, + "f2": 0.01636869423618414, + "frequency_bias": 89.97555791710946, + "predicted_positive_rate": 0.02094421768707483 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004910327959805727, + "recall": 0.21094579994678497, + "lift": 21.094581395593078 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0020927644800394773, + "recall": 0.4495217800140381, + "lift": 8.99043628600021 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001413729158230126, + "recall": 0.6073326468467712, + "lift": 6.07332647507898 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007153574635551108, + "2": 0.0007299636456388203, + "4": 0.0007542019435764491, + "8": 0.0007881568769358971 + }, + "0.2000": { + "1": 0.0009921033280078095, + "2": 0.0010166009346802563, + "4": 0.0010565960391205609, + "8": 0.001105867798709581 + }, + "0.3000": { + "1": 0.0013214325599043564, + "2": 0.0013689794030732338, + "4": 0.0014429016059425148, + "8": 0.0015061326630954053 + }, + "0.5000": { + "1": 0.0026297043943714904, + "2": 0.0027883475350057862, + "4": 0.0030552534698978784, + "8": 0.003262873105559061 + }, + "0.7000": { + "1": 0.004579501009785769, + "2": 0.005086050780598539, + "4": 0.00593845439040841, + "8": 0.00666004273072307 + }, + "0.9000": { + "1": 0.008891211711206992, + "2": 0.010287198394993147, + "4": 0.013515825530875492, + "8": 0.01799509369630581 + } + }, + "minutes": 0.3222003976504008 + }, + { + "epoch": 3, + "train_loss": 0.5111621306671731, + "train_main_loss": 0.48230293577254363, + "train_aux_spatial_loss": 0.05771839129187639, + "val_loss": 0.3889197667439779, + "val_pr_auc": 0.002294898825423087, + "val_auroc": 0.8612537704398479, + "val_brier": 0.1059054285287857, + "val_brier_skill_score": -454.0674930344856, + "val_ece": 0.2274857362708612, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0012264433693911362, + "val_positive_recall": 0.7088204038257173, + "val_far": 0.9987735566306088, + "val_csi": 0.001225825778362613, + "val_f1": 0.002448649938508416, + "val_f2": 0.006090067255525344, + "val_frequency_bias": 577.9479277364505, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1765, + "fp": 4849350, + "fn": 117, + "precision": 0.00036383388148910093, + "recall": 0.9378320935175345, + "far": 0.9996361661185109, + "csi": 0.0003638251066945469, + "f1": 0.0007273855722556596, + "f2": 0.0018163507794254484, + "frequency_bias": 2577.6381509032944, + "predicted_positive_rate": 0.6000142238713667 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1663, + "fp": 3010913, + "fn": 219, + "precision": 0.0005520192685595318, + "recall": 0.8836344314558979, + "far": 0.9994479807314405, + "csi": 0.0005519791422914603, + "f1": 0.0011033492588053972, + "f2": 0.0027532164455263794, + "frequency_bias": 1600.7311370882042, + "predicted_positive_rate": 0.372612987012987 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1557, + "fp": 2091911, + "fn": 325, + "precision": 0.0007437419630966415, + "recall": 0.8273113708820404, + "far": 0.9992562580369033, + "csi": 0.0007436265189538794, + "f1": 0.0014861478989190349, + "f2": 0.0037053854457600108, + "frequency_bias": 1112.3634431455898, + "predicted_positive_rate": 0.25893234384662955 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1334, + "fp": 1086364, + "fn": 548, + "precision": 0.0012264433693911362, + "recall": 0.7088204038257173, + "far": 0.9987735566306088, + "csi": 0.001225825778362613, + "f1": 0.002448649938508416, + "f2": 0.006090067255525344, + "frequency_bias": 577.9479277364505, + "predicted_positive_rate": 0.13453283858998144 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1035, + "fp": 542540, + "fn": 847, + "precision": 0.001904061077128271, + "recall": 0.5499468650371945, + "far": 0.9980959389228717, + "csi": 0.001901098779990522, + "f1": 0.0037949829225768484, + "f2": 0.009390259171153124, + "frequency_bias": 288.8283740701381, + "predicted_positive_rate": 0.06723252937538651 + }, + "0.9000": { + "threshold": 0.9, + "tp": 668, + "fp": 215450, + "fn": 1214, + "precision": 0.003090904043161606, + "recall": 0.35494155154091395, + "far": 0.9969090959568384, + "csi": 0.0030736384885796846, + "f1": 0.006128440366972477, + "f2": 0.014934315838423222, + "frequency_bias": 114.83421891604677, + "predicted_positive_rate": 0.02673073593073593 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.003809523768723011, + "recall": 0.16365568339824677, + "lift": 16.365568629137645 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0022337662521749735, + "recall": 0.4798087179660797, + "lift": 9.596174514400083 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001512677758000791, + "recall": 0.6498405933380127, + "lift": 6.4984058809614424 + } + }, + "val_fss": { + "0.1000": { + "1": 0.000766430190058176, + "2": 0.0007919140518234391, + "4": 0.0008313854990248393, + "8": 0.0008733206367607105 + }, + "0.2000": { + "1": 0.001183482713540407, + "2": 0.0012321062159653673, + "4": 0.0013054377666424921, + "8": 0.001372829646716478 + }, + "0.3000": { + "1": 0.0016257922850572148, + "2": 0.0016941779647912103, + "4": 0.0018051660509925682, + "8": 0.0019058108793008621 + }, + "0.5000": { + "1": 0.002719293315745386, + "2": 0.002881546868972018, + "4": 0.003177453926384799, + "8": 0.003420751645848541 + }, + "0.7000": { + "1": 0.004398705160378991, + "2": 0.0048159192047847466, + "4": 0.0055998536030907875, + "8": 0.006373926334045654 + }, + "0.9000": { + "1": 0.007842918508978447, + "2": 0.009046435090570681, + "4": 0.01234545434024481, + "8": 0.01698910587516045 + } + }, + "minutes": 0.3439892331759135 + }, + { + "epoch": 4, + "train_loss": 0.49738037993818707, + "train_main_loss": 0.46940162589285345, + "train_aux_spatial_loss": 0.05595750918425256, + "val_loss": 0.17950345166027545, + "val_pr_auc": 0.001609884375517846, + "val_auroc": 0.8526320117801028, + "val_brier": 0.042433541268110275, + "val_brier_skill_score": -181.33366800650572, + "val_ece": 0.10635442963314233, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.00177325478436419, + "val_positive_recall": 0.3990435706695005, + "val_far": 0.9982267452156358, + "val_csi": 0.0017685319065763013, + "val_f1": 0.003530819446305451, + "val_f2": 0.008711427862185442, + "val_frequency_bias": 225.03453772582358, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1553, + "fp": 2208817, + "fn": 329, + "precision": 0.0007025973027140253, + "recall": 0.8251859723698194, + "far": 0.999297402697286, + "csi": 0.0007024927409837341, + "f1": 0.001403999182733251, + "f2": 0.0035010627179428455, + "frequency_bias": 1174.4792773645058, + "predicted_positive_rate": 0.27339146567718 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1192, + "fp": 1156206, + "fn": 690, + "precision": 0.0010298963709977035, + "recall": 0.6333687566418703, + "far": 0.9989701036290023, + "csi": 0.0010292827488066537, + "f1": 0.002056448830308467, + "f2": 0.005116204806142193, + "frequency_bias": 614.9829968119022, + "predicted_positive_rate": 0.14315374149659865 + }, + "0.3000": { + "threshold": 0.3, + "tp": 969, + "fp": 763555, + "fn": 913, + "precision": 0.0012674553055234369, + "recall": 0.5148777895855473, + "far": 0.9987325446944766, + "csi": 0.0012659435067810937, + "f1": 0.0025286858401421704, + "f2": 0.006275484034754136, + "frequency_bias": 406.2295430393199, + "predicted_positive_rate": 0.09456079158936302 + }, + "0.5000": { + "threshold": 0.5, + "tp": 751, + "fp": 422764, + "fn": 1131, + "precision": 0.00177325478436419, + "recall": 0.3990435706695005, + "far": 0.9982267452156358, + "csi": 0.0017685319065763013, + "f1": 0.003530819446305451, + "f2": 0.008711427862185442, + "frequency_bias": 225.03453772582358, + "predicted_positive_rate": 0.052382807668521957 + }, + "0.7000": { + "threshold": 0.7, + "tp": 524, + "fp": 220213, + "fn": 1358, + "precision": 0.002373865731617264, + "recall": 0.2784272051009564, + "far": 0.9976261342683828, + "csi": 0.0023593507282919472, + "f1": 0.004707594589859806, + "f2": 0.011477887542987317, + "frequency_bias": 117.288522848034, + "predicted_positive_rate": 0.02730204081632653 + }, + "0.9000": { + "threshold": 0.9, + "tp": 192, + "fp": 86741, + "fn": 1690, + "precision": 0.0022085974256036258, + "recall": 0.10201912858660998, + "far": 0.9977914025743964, + "csi": 0.0021664804847500085, + "f1": 0.004323593987502111, + "f2": 0.010162924381490773, + "frequency_bias": 46.19181721572795, + "predicted_positive_rate": 0.010752380952380952 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0022387136705219746, + "recall": 0.09617428481578827, + "lift": 9.61742843468263 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018157081212848425, + "recall": 0.39001062512397766, + "lift": 7.800212749251452 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001217068638652563, + "recall": 0.5228480100631714, + "lift": 5.228480393211028 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0015197337144259304, + "2": 0.0015784324521881388, + "4": 0.001671154714090517, + "8": 0.0017417425576053969 + }, + "0.2000": { + "1": 0.0023132093983118107, + "2": 0.0024571591639005463, + "4": 0.0026935164349046747, + "8": 0.0028878255110219575 + }, + "0.3000": { + "1": 0.0029228902666941226, + "2": 0.0031646210950836418, + "4": 0.0035768996077181203, + "8": 0.003942389423331805 + }, + "0.5000": { + "1": 0.004163957521057138, + "2": 0.004514281813615906, + "4": 0.005317944336750369, + "8": 0.006180973638990284 + }, + "0.7000": { + "1": 0.005914502692297763, + "2": 0.006898543924811484, + "4": 0.009541669667679553, + "8": 0.012957827370078667 + }, + "0.9000": { + "1": 0.006528501792766983, + "2": 0.00881690475047614, + "4": 0.015511980715039009, + "8": 0.0273878632698773 + } + }, + "minutes": 0.31681207021077473 + }, + { + "epoch": 5, + "train_loss": 0.4843243009870309, + "train_main_loss": 0.45709068199154484, + "train_aux_spatial_loss": 0.054467237078325896, + "val_loss": 0.24718355586131413, + "val_pr_auc": 0.0019814322834402955, + "val_auroc": 0.8503887921685119, + "val_brier": 0.061934370547533035, + "val_brier_skill_score": -265.1272337902309, + "val_ece": 0.14998965867960198, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0014891822055106815, + "val_positive_recall": 0.44739638682252925, + "val_far": 0.9985108177944894, + "val_csi": 0.0014864480775918835, + "val_f1": 0.00296848365835644, + "val_f2": 0.007348077194954437, + "val_frequency_bias": 300.43092454835283, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1703, + "fp": 3546784, + "fn": 179, + "precision": 0.0004799228516266228, + "recall": 0.9048884165781084, + "far": 0.9995200771483734, + "csi": 0.00047989864360297645, + "f1": 0.0009593369027275756, + "f2": 0.0023945343312668814, + "frequency_bias": 1885.4872476089267, + "predicted_positive_rate": 0.43889758812615953 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1455, + "fp": 1959072, + "fn": 427, + "precision": 0.0007421473920022525, + "recall": 0.7731137088204039, + "far": 0.9992578526079977, + "csi": 0.0007419857885498589, + "f1": 0.0014828713076631833, + "f2": 0.003696543033604244, + "frequency_bias": 1041.7252922422954, + "predicted_positive_rate": 0.24248942486085343 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1192, + "fp": 1178364, + "fn": 690, + "precision": 0.001010549732272143, + "recall": 0.6333687566418703, + "far": 0.9989894502677279, + "csi": 0.0010099589407631968, + "f1": 0.0020178799056742715, + "f2": 0.005020706201077599, + "frequency_bias": 626.7566418703507, + "predicted_positive_rate": 0.1458943722943723 + }, + "0.5000": { + "threshold": 0.5, + "tp": 842, + "fp": 564569, + "fn": 1040, + "precision": 0.0014891822055106815, + "recall": 0.44739638682252925, + "far": 0.9985108177944894, + "csi": 0.0014864480775918835, + "f1": 0.00296848365835644, + "f2": 0.007348077194954437, + "frequency_bias": 300.43092454835283, + "predicted_positive_rate": 0.06993333333333333 + }, + "0.7000": { + "threshold": 0.7, + "tp": 668, + "fp": 291990, + "fn": 1214, + "precision": 0.0022825277286115534, + "recall": 0.35494155154091395, + "far": 0.9977174722713884, + "csi": 0.0022730984918604017, + "f1": 0.00453588646703334, + "f2": 0.011126434943668257, + "frequency_bias": 155.50371944739638, + "predicted_positive_rate": 0.03619764996907854 + }, + "0.9000": { + "threshold": 0.9, + "tp": 508, + "fp": 128530, + "fn": 1374, + "precision": 0.003936824811295897, + "recall": 0.26992561105207225, + "far": 0.9960631751887041, + "csi": 0.00389534705395209, + "f1": 0.007760464405743966, + "f2": 0.01859906565323726, + "frequency_bias": 68.56429330499469, + "predicted_positive_rate": 0.01596017316017316 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004155843984335661, + "recall": 0.17853347957134247, + "lift": 17.85334704984204 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018206555396318436, + "recall": 0.3910733163356781, + "lift": 7.821466669533997 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012257266789674759, + "recall": 0.5265674591064453, + "lift": 5.265675003763371 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010178924030178615, + "2": 0.0010515169589780937, + "4": 0.0011000285717616132, + "8": 0.001146573354002478 + }, + "0.2000": { + "1": 0.0016254090694596268, + "2": 0.001703218739850576, + "4": 0.0018200106661242499, + "8": 0.0019167352524950276 + }, + "0.3000": { + "1": 0.0022847465603914596, + "2": 0.0024507638687161215, + "4": 0.0027246032507362017, + "8": 0.002940764611886193 + }, + "0.5000": { + "1": 0.0035209305166398197, + "2": 0.003927423032519695, + "4": 0.004626604558975789, + "8": 0.005203306081351444 + }, + "0.7000": { + "1": 0.005774099909606312, + "2": 0.006526096761710121, + "4": 0.008174907733284553, + "8": 0.010327235764531983 + }, + "0.9000": { + "1": 0.009730396821955645, + "2": 0.011013779690799308, + "4": 0.015337678551787581, + "8": 0.022811547211147087 + } + }, + "minutes": 0.32158944606781004 + }, + { + "epoch": 6, + "train_loss": 0.4694202790356114, + "train_main_loss": 0.44288265654665, + "train_aux_spatial_loss": 0.05307524496860166, + "val_loss": 0.31109391575058304, + "val_pr_auc": 0.002276307511486534, + "val_auroc": 0.8565989334771861, + "val_brier": 0.0804823711514473, + "val_brier_skill_score": -344.82656793089006, + "val_ece": 0.1884108935656475, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0012985133535061177, + "val_positive_recall": 0.524442082890542, + "val_far": 0.9987014866464938, + "val_csi": 0.0012969861825636174, + "val_f1": 0.002590612376670315, + "val_f2": 0.0064288952461348465, + "val_frequency_bias": 403.87885228480343, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1792, + "fp": 4481944, + "fn": 90, + "precision": 0.0003996667065143889, + "recall": 0.9521785334750266, + "far": 0.9996003332934856, + "csi": 0.0003996586843468056, + "f1": 0.0007989980421872749, + "f2": 0.001994984040127679, + "frequency_bias": 2382.431455897981, + "predicted_positive_rate": 0.5545746444032158 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1565, + "fp": 2389441, + "fn": 317, + "precision": 0.0006545362077719588, + "recall": 0.8315621679064825, + "far": 0.999345463792228, + "csi": 0.0006544494407489076, + "f1": 0.001308042833596892, + "f2": 0.0032624094551088295, + "frequency_bias": 1270.460148777896, + "predicted_positive_rate": 0.29573358070500927 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1398, + "fp": 1490260, + "fn": 484, + "precision": 0.0009372121491655594, + "recall": 0.742826780021254, + "far": 0.9990627878508345, + "csi": 0.0009369081494924746, + "f1": 0.001872062348514268, + "f2": 0.004662530199721716, + "frequency_bias": 792.5919234856535, + "predicted_positive_rate": 0.1844969696969697 + }, + "0.5000": { + "threshold": 0.5, + "tp": 987, + "fp": 759113, + "fn": 895, + "precision": 0.0012985133535061177, + "recall": 0.524442082890542, + "far": 0.9987014866464938, + "csi": 0.0012969861825636174, + "f1": 0.002590612376670315, + "f2": 0.0064288952461348465, + "frequency_bias": 403.87885228480343, + "predicted_positive_rate": 0.09401360544217686 + }, + "0.7000": { + "threshold": 0.7, + "tp": 817, + "fp": 424418, + "fn": 1065, + "precision": 0.0019212905805025457, + "recall": 0.4341126461211477, + "far": 0.9980787094194975, + "csi": 0.0019164907342247243, + "f1": 0.0038256496463498295, + "f2": 0.009439346709399835, + "frequency_bias": 225.94845908607863, + "predicted_positive_rate": 0.05259554730983303 + }, + "0.9000": { + "threshold": 0.9, + "tp": 579, + "fp": 166503, + "fn": 1303, + "precision": 0.0034653643121341617, + "recall": 0.30765143464399575, + "far": 0.9965346356878658, + "csi": 0.003438548564302046, + "f1": 0.0068535309296654905, + "f2": 0.01657980642574881, + "frequency_bias": 88.77895855472902, + "predicted_positive_rate": 0.020665677179962896 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004650587681680918, + "recall": 0.1997874528169632, + "lift": 19.97874707994901 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001996289473026991, + "recall": 0.4287991523742676, + "lift": 8.575983340143242 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001272727269679308, + "recall": 0.546758770942688, + "lift": 5.467587746563329 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0008456018359040973, + "2": 0.0008747226916036022, + "4": 0.0009169214186822705, + "8": 0.0009552239909184079 + }, + "0.2000": { + "1": 0.0014371762062292959, + "2": 0.0015125023076059119, + "4": 0.0016182172539731532, + "8": 0.001698721433581074 + }, + "0.3000": { + "1": 0.002066243274234769, + "2": 0.0021820389578350863, + "4": 0.0023651199981745696, + "8": 0.00250168028465092 + }, + "0.5000": { + "1": 0.0030274957578868555, + "2": 0.0033090501311099363, + "4": 0.003784045326573948, + "8": 0.004098759927500861 + }, + "0.7000": { + "1": 0.0045338085209784, + "2": 0.004961997974645471, + "4": 0.005855624169939677, + "8": 0.006579670951755601 + }, + "0.9000": { + "1": 0.008876876067272832, + "2": 0.010220203844888975, + "4": 0.013385368093454919, + "8": 0.017526068461561795 + } + }, + "minutes": 0.35659926732381186 + }, + { + "epoch": 7, + "train_loss": 0.4594879526371206, + "train_main_loss": 0.43360181109779, + "train_aux_spatial_loss": 0.0517722830069709, + "val_loss": 0.24388390369713306, + "val_pr_auc": 0.00280721100408273, + "val_auroc": 0.8806628065269809, + "val_brier": 0.061469998210668564, + "val_brier_skill_score": -263.1318615217164, + "val_ece": 0.14419293259630367, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0018425305628724307, + "val_positive_recall": 0.5924548352816154, + "val_far": 0.9981574694371276, + "val_csi": 0.001840198180266804, + "val_f1": 0.0036736361419901554, + "val_f2": 0.009099455828058642, + "val_frequency_bias": 321.54410201912856, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1701, + "fp": 3069167, + "fn": 181, + "precision": 0.000553915049425765, + "recall": 0.9038257173219979, + "far": 0.9994460849505742, + "csi": 0.0005538824030486, + "f1": 0.0011071515743226751, + "f2": 0.0027628024464688757, + "frequency_bias": 1631.7045696068012, + "predicted_positive_rate": 0.3798228818800247 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1553, + "fp": 1735305, + "fn": 329, + "precision": 0.0008941433323852612, + "recall": 0.8251859723698194, + "far": 0.9991058566676148, + "csi": 0.0008939739935884853, + "f1": 0.0017863510358075386, + "f2": 0.004451423022198068, + "frequency_bias": 922.8788522848034, + "predicted_positive_rate": 0.21482473716759432 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1386, + "fp": 1140133, + "fn": 496, + "precision": 0.0012141716432227585, + "recall": 0.7364505844845909, + "far": 0.9987858283567772, + "csi": 0.001213644304146618, + "f1": 0.002424346314197731, + "f2": 0.006031084890348262, + "frequency_bias": 606.5456960680127, + "predicted_positive_rate": 0.14118973407544835 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1115, + "fp": 604031, + "fn": 767, + "precision": 0.0018425305628724307, + "recall": 0.5924548352816154, + "far": 0.9981574694371276, + "csi": 0.001840198180266804, + "f1": 0.0036736361419901554, + "f2": 0.009099455828058642, + "frequency_bias": 321.54410201912856, + "predicted_positive_rate": 0.07484799010513296 + }, + "0.7000": { + "threshold": 0.7, + "tp": 843, + "fp": 315079, + "fn": 1039, + "precision": 0.0026683801697887453, + "recall": 0.4479277364505845, + "far": 0.9973316198302112, + "csi": 0.002659633204085045, + "f1": 0.005305156637424324, + "f2": 0.013031380429741845, + "frequency_bias": 167.86503719447396, + "predicted_positive_rate": 0.03907507730364873 + }, + "0.9000": { + "threshold": 0.9, + "tp": 585, + "fp": 138799, + "fn": 1297, + "precision": 0.004197038397520519, + "recall": 0.3108395324123273, + "far": 0.9958029616024795, + "csi": 0.004158344054989657, + "f1": 0.008282247674599692, + "f2": 0.019909878022217383, + "frequency_bias": 74.06163655685441, + "predicted_positive_rate": 0.01723982683982684 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005553494207561016, + "recall": 0.2385759800672531, + "lift": 23.857599034176413 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0023129251785576344, + "recall": 0.4968118965625763, + "lift": 9.936238239152363 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001546073006466031, + "recall": 0.6641870141029358, + "lift": 6.641870593042289 + } + }, + "val_fss": { + "0.1000": { + "1": 0.001182645589865583, + "2": 0.001235610970657941, + "4": 0.0013156511427493411, + "8": 0.001399485420612856 + }, + "0.2000": { + "1": 0.001955440214953197, + "2": 0.002070129096730211, + "4": 0.0022438521231362696, + "8": 0.002400362116154686 + }, + "0.3000": { + "1": 0.0027211631165605654, + "2": 0.002910697745954649, + "4": 0.0032097807960153046, + "8": 0.00344582755765388 + }, + "0.5000": { + "1": 0.004202872161291116, + "2": 0.004619998632292344, + "4": 0.0053646052916888065, + "8": 0.00604022995299458 + }, + "0.7000": { + "1": 0.0065952032055480725, + "2": 0.00754062200693062, + "4": 0.009568511759124099, + "8": 0.011927180679107141 + }, + "0.9000": { + "1": 0.011117215021448135, + "2": 0.013052893418606493, + "4": 0.018349644962076805, + "8": 0.025929086293738135 + } + }, + "minutes": 0.34816946188608805 + }, + { + "epoch": 8, + "train_loss": 0.44826403022158967, + "train_main_loss": 0.42296392545852457, + "train_aux_spatial_loss": 0.050600208696345975, + "val_loss": 0.3248936382432779, + "val_pr_auc": 0.0044442059631609795, + "val_auroc": 0.8849275341349525, + "val_brier": 0.08548334985971451, + "val_brier_skill_score": -366.3153893738006, + "val_ece": 0.18183675751987904, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0015666706745220632, + "val_positive_recall": 0.6923485653560042, + "val_far": 0.9984333293254779, + "val_csi": 0.0015655807727937387, + "val_f1": 0.0031262671218908276, + "val_f2": 0.0077630870276015574, + "val_frequency_bias": 441.92348565356, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1772, + "fp": 3855746, + "fn": 110, + "precision": 0.0004593627301285438, + "recall": 0.9415515409139213, + "far": 0.9995406372698714, + "csi": 0.0004593496314315429, + "f1": 0.0009182774524537493, + "f2": 0.0022923401170387107, + "frequency_bias": 2049.6907545164718, + "predicted_positive_rate": 0.47712034632034633 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1618, + "fp": 2317858, + "fn": 264, + "precision": 0.0006975713480113612, + "recall": 0.8597236981934112, + "far": 0.9993024286519886, + "csi": 0.0006974919603058964, + "f1": 0.0013940116087221358, + "f2": 0.0034765733105744556, + "frequency_bias": 1232.4527098831031, + "predicted_positive_rate": 0.28688633271490416 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1481, + "fp": 1547894, + "fn": 401, + "precision": 0.0009558693021379589, + "recall": 0.7869287991498406, + "far": 0.999044130697862, + "csi": 0.0009556219737562073, + "f1": 0.0019094192645061393, + "f2": 0.0047562372222290025, + "frequency_bias": 823.259829968119, + "predicted_positive_rate": 0.19163574520717377 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1303, + "fp": 830397, + "fn": 579, + "precision": 0.0015666706745220632, + "recall": 0.6923485653560042, + "far": 0.9984333293254779, + "csi": 0.0015655807727937387, + "f1": 0.0031262671218908276, + "f2": 0.0077630870276015574, + "frequency_bias": 441.92348565356, + "predicted_positive_rate": 0.10286951144094002 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1100, + "fp": 493742, + "fn": 782, + "precision": 0.002222931764078231, + "recall": 0.5844845908607864, + "far": 0.9977770682359217, + "csi": 0.002219424402369538, + "f1": 0.004429018932042743, + "f2": 0.010948105977665863, + "frequency_bias": 262.93411264612115, + "predicted_positive_rate": 0.06120494743351886 + }, + "0.9000": { + "threshold": 0.9, + "tp": 839, + "fp": 211101, + "fn": 1043, + "precision": 0.0039586675474190805, + "recall": 0.44580233793836344, + "far": 0.996041332452581, + "csi": 0.003939281538902166, + "f1": 0.007847648979057346, + "f2": 0.019114403922211892, + "frequency_bias": 112.61424017003188, + "predicted_positive_rate": 0.026213976499690785 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.007482993416488171, + "recall": 0.3214665353298187, + "lift": 32.14665395038951 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002540507121011615, + "recall": 0.5456960797309875, + "lift": 10.913921572844114 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001606679055839777, + "recall": 0.6902231574058533, + "lift": 6.902231866677136 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0009645709705659034, + "2": 0.0009937018941826858, + "4": 0.001037849954013348, + "8": 0.0010919201251842203 + }, + "0.2000": { + "1": 0.0015015023453176513, + "2": 0.0015689402217066162, + "4": 0.0016632836621220948, + "8": 0.001740115974165457 + }, + "0.3000": { + "1": 0.0020927497428742114, + "2": 0.00219703009118255, + "4": 0.0023412446037914814, + "8": 0.0024308848199744437 + }, + "0.5000": { + "1": 0.003430910518313901, + "2": 0.0035887345058116527, + "4": 0.003812179025492024, + "8": 0.0039358805986901 + }, + "0.7000": { + "1": 0.004948203201738943, + "2": 0.00521956377670052, + "4": 0.005637850205958861, + "8": 0.0059804616256462495 + }, + "0.9000": { + "1": 0.00948030325480842, + "2": 0.01033684622192943, + "4": 0.01188914050876233, + "8": 0.014040969292779426 + } + }, + "minutes": 0.31795929272969564 + }, + { + "epoch": 9, + "train_loss": 0.4411211615542718, + "train_main_loss": 0.4163666974302027, + "train_aux_spatial_loss": 0.04950892778797695, + "val_loss": 0.2851397772630056, + "val_pr_auc": 0.0032182950594722933, + "val_auroc": 0.884300165942819, + "val_brier": 0.07394615560770035, + "val_brier_skill_score": -316.7409517094562, + "val_ece": 0.16132334364664333, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0016357551889862657, + "val_positive_recall": 0.6609989373007439, + "val_far": 0.9983642448110137, + "val_csi": 0.0016343840776306161, + "val_f1": 0.0032634344499578296, + "val_f2": 0.008098610346170022, + "val_frequency_bias": 404.0940488841658, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1733, + "fp": 3292718, + "fn": 149, + "precision": 0.0005260360527444481, + "recall": 0.9208289054197663, + "far": 0.9994739639472555, + "csi": 0.0005260122624901354, + "f1": 0.0010514714381101666, + "f2": 0.002624183860648417, + "frequency_bias": 1750.5053134962805, + "predicted_positive_rate": 0.4074769325912183 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1584, + "fp": 1951842, + "fn": 298, + "precision": 0.0008108830331939884, + "recall": 0.8416578108395324, + "far": 0.999189116966806, + "csi": 0.0008107593498365174, + "f1": 0.0016202051032369327, + "f2": 0.004038850477879644, + "frequency_bias": 1037.952178533475, + "predicted_positive_rate": 0.24161113172541743 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1453, + "fp": 1379073, + "fn": 429, + "precision": 0.0010524973814328741, + "recall": 0.7720510095642933, + "far": 0.9989475026185671, + "csi": 0.0010521704182974826, + "f1": 0.0021021290386051008, + "f2": 0.005233946229757632, + "frequency_bias": 733.5419766206163, + "predicted_positive_rate": 0.17075151515151515 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1244, + "fp": 759261, + "fn": 638, + "precision": 0.0016357551889862657, + "recall": 0.6609989373007439, + "far": 0.9983642448110137, + "csi": 0.0016343840776306161, + "f1": 0.0032634344499578296, + "f2": 0.008098610346170022, + "frequency_bias": 404.0940488841658, + "predicted_positive_rate": 0.09406369820655534 + }, + "0.7000": { + "threshold": 0.7, + "tp": 996, + "fp": 385640, + "fn": 886, + "precision": 0.0025760663776782295, + "recall": 0.5292242295430393, + "far": 0.9974239336223217, + "csi": 0.0025701766609379597, + "f1": 0.005127175574876839, + "f2": 0.012634334946874904, + "frequency_bias": 205.43889479277365, + "predicted_positive_rate": 0.04782139764996908 + }, + "0.9000": { + "threshold": 0.9, + "tp": 768, + "fp": 182254, + "fn": 1114, + "precision": 0.0041962168482477516, + "recall": 0.40807651434643993, + "far": 0.9958037831517522, + "csi": 0.004170830255897815, + "f1": 0.008307013369099642, + "f2": 0.020152191025977435, + "frequency_bias": 97.24867162592986, + "predicted_positive_rate": 0.02263722943722944 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005862708669155836, + "recall": 0.25185972452163696, + "lift": 25.18597255264594 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002503401366993785, + "recall": 0.5377258062362671, + "lift": 10.754516670609245 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015658626798540354, + "recall": 0.6726886034011841, + "lift": 6.726886274172471 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0011201829491246373, + "2": 0.0011650008395039935, + "4": 0.0012291472998500197, + "8": 0.001288563228972528 + }, + "0.2000": { + "1": 0.0017641369780564409, + "2": 0.0018498610763283407, + "4": 0.0019697883619944623, + "8": 0.0020608795760591603 + }, + "0.3000": { + "1": 0.0023264923565211593, + "2": 0.002448559375719639, + "4": 0.0026245463430637495, + "8": 0.002768340451214768 + }, + "0.5000": { + "1": 0.0037134975103053636, + "2": 0.003964199893520592, + "4": 0.004366414205446922, + "8": 0.004762426177904011 + }, + "0.7000": { + "1": 0.006053201961429555, + "2": 0.006659116708271573, + "4": 0.007824432033392337, + "8": 0.0091423801874152 + }, + "0.9000": { + "1": 0.01022660873826664, + "2": 0.011336714828667716, + "4": 0.01395971459290235, + "8": 0.017751013937979288 + } + }, + "minutes": 0.3203668077786764 + }, + { + "epoch": 10, + "train_loss": 0.4325153522077466, + "train_main_loss": 0.40816109493837044, + "train_aux_spatial_loss": 0.048708514710370465, + "val_loss": 0.19746975464125474, + "val_pr_auc": 0.0028221717878134725, + "val_auroc": 0.8620318173394532, + "val_brier": 0.044820670038461685, + "val_brier_skill_score": -191.59097700534562, + "val_ece": 0.13526803746421806, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.002322708563657786, + "val_positive_recall": 0.47555791710945805, + "val_far": 0.9976772914363422, + "val_csi": 0.0023167742219392046, + "val_f1": 0.004622838371108036, + "val_f2": 0.011391000218910844, + "val_frequency_bias": 204.74282678002126, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1694, + "fp": 3439838, + "fn": 188, + "precision": 0.000492222649680433, + "recall": 0.900106269925611, + "far": 0.9995077773503196, + "csi": 0.0004921957625838244, + "f1": 0.0009839072501883306, + "f2": 0.0024557415643682626, + "frequency_bias": 1828.6567481402762, + "predicted_positive_rate": 0.4256687693259122 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1437, + "fp": 1559537, + "fn": 445, + "precision": 0.0009205790743471704, + "recall": 0.7635494155154091, + "far": 0.9990794209256528, + "csi": 0.0009203167119139706, + "f1": 0.0018389410156789876, + "f2": 0.004580803849787887, + "frequency_bias": 829.422954303932, + "predicted_positive_rate": 0.1930703772418058 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1178, + "fp": 891195, + "fn": 704, + "precision": 0.0013200757979006537, + "recall": 0.6259298618490967, + "far": 0.9986799242020994, + "csi": 0.0013190352007721618, + "f1": 0.0026345952776333378, + "f2": 0.006545164412529823, + "frequency_bias": 474.16206163655687, + "predicted_positive_rate": 0.110373902288188 + }, + "0.5000": { + "threshold": 0.5, + "tp": 895, + "fp": 384431, + "fn": 987, + "precision": 0.002322708563657786, + "recall": 0.47555791710945805, + "far": 0.9976772914363422, + "csi": 0.0023167742219392046, + "f1": 0.004622838371108036, + "f2": 0.011391000218910844, + "frequency_bias": 204.74282678002126, + "predicted_positive_rate": 0.04765936920222635 + }, + "0.7000": { + "threshold": 0.7, + "tp": 659, + "fp": 170866, + "fn": 1223, + "precision": 0.0038420055385512317, + "recall": 0.35015940488841657, + "far": 0.9961579944614488, + "csi": 0.003814805381248987, + "f1": 0.007600615892092015, + "f2": 0.01840237248189084, + "frequency_bias": 91.13974495217853, + "predicted_positive_rate": 0.0212152133580705 + }, + "0.9000": { + "threshold": 0.9, + "tp": 327, + "fp": 52950, + "fn": 1555, + "precision": 0.006137732980460612, + "recall": 0.17375132837407015, + "far": 0.9938622670195394, + "csi": 0.0059636708491391886, + "f1": 0.011856632643811527, + "f2": 0.026889236082559, + "frequency_bias": 28.308714133900107, + "predicted_positive_rate": 0.00658961038961039 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005701917223632336, + "recall": 0.24495217204093933, + "lift": 24.49521864311588 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002246134914457798, + "recall": 0.4824654757976532, + "lift": 9.649309815222225 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001405071117915213, + "recall": 0.6036131978034973, + "lift": 6.036131864526638 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010545868717067863, + "2": 0.0011022352726061113, + "4": 0.001168632193011776, + "8": 0.0012236074960421117 + }, + "0.2000": { + "1": 0.0020329538883044895, + "2": 0.0021331716637861264, + "4": 0.0023056844278652733, + "8": 0.0024083609396829964 + }, + "0.3000": { + "1": 0.0030090804133009463, + "2": 0.00323794828427848, + "4": 0.0036126412524609686, + "8": 0.003827556736519133 + }, + "0.5000": { + "1": 0.005642367607543997, + "2": 0.006246436645517495, + "4": 0.007234446182773802, + "8": 0.007960937835548854 + }, + "0.7000": { + "1": 0.010066936508528523, + "2": 0.011589698733172615, + "4": 0.014878004135214007, + "8": 0.018806429552858606 + }, + "0.9000": { + "1": 0.016158702083284626, + "2": 0.01968974409486912, + "4": 0.028978356172675412, + "8": 0.04860085752482901 + } + }, + "minutes": 0.34885872999827067 + } +] diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/run_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/run_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..8c6a4867769ab5e7859e0895d105bf948ada1327 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/run_summary.json @@ -0,0 +1,172 @@ +{ + "run_name": "reference_fireprone_seed_1_20260504", + "device": "cuda", + "index_root": "", + "input_channels": 16, + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "train_positive_rate": 0.0018382099293520256, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight": 50.0, + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "aux_positive_rate": 0.004560296846011132, + "init_checkpoint": "", + "best_val_pr_auc": 0.007534030208235136, + "best_val_auroc": 0.8668780863986516, + "best_test_pr_auc": 0.0013275812482877569, + "best_test_auroc": 0.8211195324224633, + "best_test_brier": 0.14432816207408905, + "best_test_ece": 0.2856647948942866, + "best_test_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.001855287584476173, + "recall": 0.0764915868639946, + "lift": 7.649158522554341 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001625231932848692, + "recall": 0.3350331485271454, + "lift": 6.7006629022351945 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012022263836115599, + "recall": 0.4956654906272888, + "lift": 4.956654841647355 + } + }, + "best_test_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1902, + "fp": 5715039, + "fn": 59, + "precision": 0.00033269540476279185, + "recall": 0.969913309535951, + "far": 0.9996673045952372, + "csi": 0.000332691971313626, + "f1": 0.0006651626483545268, + "f2": 0.0016611977567716515, + "frequency_bias": 2915.3192248852624, + "predicted_positive_rate": 0.7071046382189239 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1820, + "fp": 4195336, + "fn": 141, + "precision": 0.0004336269607324579, + "recall": 0.9280979092299847, + "far": 0.9995663730392675, + "csi": 0.0004336123938811097, + "f1": 0.000866848911330644, + "f2": 0.0021640903686087993, + "frequency_bias": 2140.3141254462007, + "predicted_positive_rate": 0.5191287569573284 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1681, + "fp": 3187599, + "fn": 280, + "precision": 0.0005270782120102343, + "recall": 0.85721570627231, + "far": 0.9994729217879897, + "csi": 0.0005270319417098284, + "f1": 0.001053508650709865, + "f2": 0.002628925246565351, + "frequency_bias": 1626.353901070882, + "predicted_positive_rate": 0.3944687693259122 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1319, + "fp": 1642412, + "fn": 642, + "precision": 0.0008024427354597559, + "recall": 0.6726160122386537, + "far": 0.9991975572645403, + "csi": 0.0008021294438670545, + "f1": 0.0016029730958162284, + "f2": 0.0039931580461075025, + "frequency_bias": 838.2106068332483, + "predicted_positive_rate": 0.20330624613481757 + }, + "0.7000": { + "threshold": 0.7, + "tp": 863, + "fp": 666140, + "fn": 1098, + "precision": 0.0012938472540603265, + "recall": 0.44008159102498723, + "far": 0.9987061527459397, + "csi": 0.0012917208625641931, + "f1": 0.0025801089445769876, + "f2": 0.006394041908758577, + "frequency_bias": 340.13411524732277, + "predicted_positive_rate": 0.08249882498453927 + }, + "0.9000": { + "threshold": 0.9, + "tp": 444, + "fp": 213874, + "fn": 1517, + "precision": 0.002071687865694902, + "recall": 0.22641509433962265, + "far": 0.9979283121343051, + "csi": 0.002057126971992494, + "f1": 0.004105807776067025, + "f2": 0.009992708023874471, + "frequency_bias": 109.29015808261092, + "predicted_positive_rate": 0.02650810142238714 + } + }, + "best_test_fss": { + "0.1000": { + "1": 0.0006848046210931047, + "2": 0.0006972018571715921, + "4": 0.0007162254754048414, + "8": 0.0007405531411359689 + }, + "0.2000": { + "1": 0.0009004883576219092, + "2": 0.0009194891185105858, + "4": 0.0009493580716710692, + "8": 0.0009785076278155946 + }, + "0.3000": { + "1": 0.0011134993250828673, + "2": 0.0011478568434259984, + "4": 0.0011942054782714262, + "8": 0.0012221388013780032 + }, + "0.5000": { + "1": 0.0017304583364127213, + "2": 0.0017995263941176232, + "4": 0.0018910170119570413, + "8": 0.001927562442000319 + }, + "0.7000": { + "1": 0.0029083943170400417, + "2": 0.003109232173895693, + "4": 0.003414739147105883, + "8": 0.0036022619251578902 + }, + "0.9000": { + "1": 0.005409636856494537, + "2": 0.0063686345028376445, + "4": 0.008451726086051603, + "8": 0.010875712868847476 + } + }, + "epochs": 10 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/tile_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/tile_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..a780a7c2a1b050634c3f3137ba115235cc4fbf78 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_1_20260504/tile_summary.json @@ -0,0 +1,16 @@ +{ + "index_root": "", + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "input_channels": 16, + "train_positive_rate": 0.0018382099293520256, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight_source": "tiles", + "positive_rate_source": "tiles", + "pos_weight": 50.0, + "loss_type": "bce", + "train_target_mode": "dilate_max", + "train_target_radius": 2 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/history.json b/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/history.json new file mode 100644 index 0000000000000000000000000000000000000000..cd7156e95d7219330e7369e0995c44f8ad41ceb0 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/history.json @@ -0,0 +1,1682 @@ +[ + { + "epoch": 1, + "train_loss": 0.5732859523984933, + "train_main_loss": 0.5301023899695203, + "train_aux_spatial_loss": 0.08636712405139627, + "val_loss": 0.24509115666151046, + "val_pr_auc": 0.0017965274230398033, + "val_auroc": 0.8433017073874776, + "val_brier": 0.06070003658533096, + "val_brier_skill_score": -259.82339554936163, + "val_ece": 0.1579977289119908, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.001549417642141233, + "val_positive_recall": 0.4500531349628055, + "val_far": 0.9984505823578588, + "val_csi": 0.0015464896328593442, + "val_f1": 0.003088203391190052, + "val_f2": 0.007641852449994136, + "val_frequency_bias": 290.4659936238045, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1770, + "fp": 4182500, + "fn": 112, + "precision": 0.0004230128552889751, + "recall": 0.9404888416578109, + "far": 0.999576987144711, + "csi": 0.00042300153284284276, + "f1": 0.0008456453564036852, + "f2": 0.0021112658577536417, + "frequency_bias": 2223.3103081827844, + "predicted_positive_rate": 0.5175349412492269 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1456, + "fp": 2049649, + "fn": 426, + "precision": 0.0007098612699008583, + "recall": 0.7736450584484591, + "far": 0.9992901387300991, + "csi": 0.0007097138673507736, + "f1": 0.0014184210616043841, + "f2": 0.0035363272618286015, + "frequency_bias": 1089.8538788522849, + "predicted_positive_rate": 0.2536926406926407 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1161, + "fp": 1142810, + "fn": 721, + "precision": 0.0010148858668620096, + "recall": 0.6168969181721573, + "far": 0.998985114133138, + "csi": 0.0010142466270402868, + "f1": 0.0020264379462287046, + "f2": 0.005041254920759809, + "frequency_bias": 607.8485653560042, + "predicted_positive_rate": 0.1414930117501546 + }, + "0.5000": { + "threshold": 0.5, + "tp": 847, + "fp": 545810, + "fn": 1035, + "precision": 0.001549417642141233, + "recall": 0.4500531349628055, + "far": 0.9984505823578588, + "csi": 0.0015464896328593442, + "f1": 0.003088203391190052, + "f2": 0.007641852449994136, + "frequency_bias": 290.4659936238045, + "predicted_positive_rate": 0.06761372912801485 + }, + "0.7000": { + "threshold": 0.7, + "tp": 596, + "fp": 258780, + "fn": 1286, + "precision": 0.0022978224662266364, + "recall": 0.31668437832093516, + "far": 0.9977021775337733, + "csi": 0.00228648594731875, + "f1": 0.004562539711702608, + "f2": 0.011165063093846479, + "frequency_bias": 137.8193411264612, + "predicted_positive_rate": 0.03208113790970934 + }, + "0.9000": { + "threshold": 0.9, + "tp": 326, + "fp": 103328, + "fn": 1556, + "precision": 0.0031450788199201187, + "recall": 0.17321997874601489, + "far": 0.9968549211800799, + "csi": 0.0030985647752114816, + "f1": 0.006177986658580957, + "f2": 0.014660646507528196, + "frequency_bias": 55.07651434643996, + "predicted_positive_rate": 0.012820531849103277 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0032529374584555626, + "recall": 0.13974495232105255, + "lift": 13.974495095614635 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018429190386086702, + "recall": 0.39585548639297485, + "lift": 7.917109810921228 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012467532651498914, + "recall": 0.5356004238128662, + "lift": 5.356004415022078 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0008853317818074968, + "2": 0.0009089386381523346, + "4": 0.0009457052841735925, + "8": 0.0009852276473808486 + }, + "0.2000": { + "1": 0.0015577706818201698, + "2": 0.0016341648495771244, + "4": 0.0017485784120635506, + "8": 0.001831827612458703 + }, + "0.3000": { + "1": 0.0023003002423931385, + "2": 0.002464986024784954, + "4": 0.0027264302361427184, + "8": 0.002898080903875666 + }, + "0.5000": { + "1": 0.003590797802279555, + "2": 0.0038924124691716022, + "4": 0.004390693252639188, + "8": 0.004810087814569819 + }, + "0.7000": { + "1": 0.005772289535979636, + "2": 0.006391883882916782, + "4": 0.0077055917957817854, + "8": 0.009441454533165472 + }, + "0.9000": { + "1": 0.008049290674849052, + "2": 0.009074456823212063, + "4": 0.011774809503574457, + "8": 0.016814201894514325 + } + }, + "minutes": 0.3870505134264628 + }, + { + "epoch": 2, + "train_loss": 0.5283296908698704, + "train_main_loss": 0.49774899936544365, + "train_aux_spatial_loss": 0.06116138393438454, + "val_loss": 0.4062879979610443, + "val_pr_auc": 0.002054551518322294, + "val_auroc": 0.8653528186778066, + "val_brier": 0.1165778785943985, + "val_brier_skill_score": -499.9261913407209, + "val_ece": 0.24187411134523915, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0010914377868432427, + "val_positive_recall": 0.7231668437832094, + "val_far": 0.9989085622131567, + "val_csi": 0.0010909819639278557, + "val_f1": 0.002179586038798553, + "val_f2": 0.005424441633247164, + "val_frequency_bias": 662.5818278427205, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1812, + "fp": 4827814, + "fn": 70, + "precision": 0.00037518433104343895, + "recall": 0.9628055260361318, + "far": 0.9996248156689566, + "csi": 0.0003751788932471112, + "f1": 0.0007500763736704978, + "f2": 0.0018730021826884155, + "frequency_bias": 2566.219978746015, + "predicted_positive_rate": 0.597356338899196 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1715, + "fp": 3400877, + "fn": 167, + "precision": 0.0005040275178452192, + "recall": 0.9112646121147715, + "far": 0.9994959724821548, + "csi": 0.000504002781272491, + "f1": 0.001007497780861302, + "f2": 0.0025145742671812132, + "frequency_bias": 1807.9659936238045, + "predicted_positive_rate": 0.42085244279529993 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1594, + "fp": 2554643, + "fn": 288, + "precision": 0.0006235728533778362, + "recall": 0.8469713071200851, + "far": 0.9993764271466221, + "csi": 0.0006235026060766079, + "f1": 0.0012462281856317082, + "f2": 0.0031087092615742864, + "frequency_bias": 1358.2555791710945, + "predicted_positive_rate": 0.31617031539888685 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1361, + "fp": 1245618, + "fn": 521, + "precision": 0.0010914377868432427, + "recall": 0.7231668437832094, + "far": 0.9989085622131567, + "csi": 0.0010909819639278557, + "f1": 0.002179586038798553, + "f2": 0.005424441633247164, + "frequency_bias": 662.5818278427205, + "predicted_positive_rate": 0.15423364254792826 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1016, + "fp": 533253, + "fn": 866, + "precision": 0.0019016637686259169, + "recall": 0.5398512221041445, + "far": 0.9980983362313741, + "csi": 0.001898586338026853, + "f1": 0.0037899770773532084, + "f2": 0.009376205479173935, + "frequency_bias": 283.8836344314559, + "predicted_positive_rate": 0.06608150896722326 + }, + "0.9000": { + "threshold": 0.9, + "tp": 609, + "fp": 209521, + "fn": 1273, + "precision": 0.002898205872555085, + "recall": 0.3235919234856536, + "far": 0.9971017941274449, + "csi": 0.0028807538209013116, + "f1": 0.005744957832575515, + "f2": 0.01398983726764006, + "frequency_bias": 111.65249734325187, + "predicted_positive_rate": 0.025990105132962276 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0032900432124733925, + "recall": 0.1413390040397644, + "lift": 14.133899997849502 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002221397589892149, + "recall": 0.4771519601345062, + "lift": 9.543039213577943 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015114409616217017, + "recall": 0.6493092179298401, + "lift": 6.493092650948695 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007845684873319936, + "2": 0.0008058636007729492, + "4": 0.0008395871424885515, + "8": 0.0008863828630808479 + }, + "0.2000": { + "1": 0.001061890053744352, + "2": 0.0010969197383139395, + "4": 0.0011554186534993294, + "8": 0.0012230009768544603 + }, + "0.3000": { + "1": 0.0013362506016462161, + "2": 0.0013943592723939524, + "4": 0.0014886648608402053, + "8": 0.0015775059478972953 + }, + "0.5000": { + "1": 0.002468789757546941, + "2": 0.002622560260418405, + "4": 0.0028690079941975366, + "8": 0.0030831279713063875 + }, + "0.7000": { + "1": 0.0044548071137847645, + "2": 0.004899878032674221, + "4": 0.005742732261700079, + "8": 0.006554892590571404 + }, + "0.9000": { + "1": 0.0075190310894935886, + "2": 0.008848901158122424, + "4": 0.01236010905206364, + "8": 0.017285273473687246 + } + }, + "minutes": 0.3233311374982198 + }, + { + "epoch": 3, + "train_loss": 0.5136756693969899, + "train_main_loss": 0.4847016787342131, + "train_aux_spatial_loss": 0.057947980659368674, + "val_loss": 0.3151714046796163, + "val_pr_auc": 0.0024664154896628107, + "val_auroc": 0.8703901645976795, + "val_brier": 0.08613400906324387, + "val_brier_skill_score": -369.111222001865, + "val_ece": 0.1926309692158547, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0014452201914565972, + "val_positive_recall": 0.6567481402763018, + "val_far": 0.9985547798085435, + "val_csi": 0.001444129368754228, + "val_f1": 0.0028840937330463245, + "val_f2": 0.007163049790150459, + "val_frequency_bias": 454.42773645058446, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1758, + "fp": 4044915, + "fn": 124, + "precision": 0.00043443095105534843, + "recall": 0.9341126461211477, + "far": 0.9995655690489447, + "csi": 0.00043441763943187663, + "f1": 0.0008684580053871072, + "f2": 0.0021681214128258562, + "frequency_bias": 2150.1981934112646, + "predicted_positive_rate": 0.5005161410018553 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1623, + "fp": 2661790, + "fn": 259, + "precision": 0.0006093685057480759, + "recall": 0.8623804463336876, + "far": 0.9993906314942519, + "csi": 0.0006093092542925706, + "f1": 0.001217876445196498, + "f2": 0.0030382550569256306, + "frequency_bias": 1415.2035069075453, + "predicted_positive_rate": 0.3294264687693259 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1509, + "fp": 1763618, + "fn": 373, + "precision": 0.0008548959933194609, + "recall": 0.8018065887353879, + "far": 0.9991451040066806, + "csi": 0.0008547153780798641, + "f1": 0.0017079709271429857, + "f2": 0.00425632737334676, + "frequency_bias": 937.8995749202976, + "predicted_positive_rate": 0.2183212121212121 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1236, + "fp": 853997, + "fn": 646, + "precision": 0.0014452201914565972, + "recall": 0.6567481402763018, + "far": 0.9985547798085435, + "csi": 0.001444129368754228, + "f1": 0.0028840937330463245, + "f2": 0.007163049790150459, + "frequency_bias": 454.42773645058446, + "predicted_positive_rate": 0.10578021026592455 + }, + "0.7000": { + "threshold": 0.7, + "tp": 941, + "fp": 447133, + "fn": 941, + "precision": 0.0021000995371300278, + "recall": 0.5, + "far": 0.99789990046287, + "csi": 0.00209569836197009, + "f1": 0.004182631190605304, + "f2": 0.010326995930658776, + "frequency_bias": 238.08395324123273, + "predicted_positive_rate": 0.05542040816326531 + }, + "0.9000": { + "threshold": 0.9, + "tp": 549, + "fp": 144590, + "fn": 1333, + "precision": 0.003782580836301752, + "recall": 0.2917109458023379, + "far": 0.9962174191636982, + "csi": 0.003748156644273308, + "f1": 0.007468320852123166, + "f2": 0.017980310086659197, + "frequency_bias": 77.11955366631243, + "predicted_positive_rate": 0.017951638837353123 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004155843984335661, + "recall": 0.17853347957134247, + "lift": 17.85334704984204 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0021917130798101425, + "recall": 0.4707757830619812, + "lift": 9.415515691882671 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015003092121332884, + "recall": 0.6445270776748657, + "lift": 6.445271080255079 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0009126233899767744, + "2": 0.0009396067987926804, + "4": 0.0009843825552887653, + "8": 0.0010422517525249253 + }, + "0.2000": { + "1": 0.001297803743524062, + "2": 0.0013508992011256327, + "4": 0.001432314924131206, + "8": 0.001506973317854765 + }, + "0.3000": { + "1": 0.0018602197823269773, + "2": 0.0019451288490225371, + "4": 0.00207146808966463, + "8": 0.002172537528579155 + }, + "0.5000": { + "1": 0.003223589167235419, + "2": 0.0034269782605044208, + "4": 0.0037462044010848627, + "8": 0.0039810770352668445 + }, + "0.7000": { + "1": 0.004801884901590947, + "2": 0.005215736541223648, + "4": 0.00592818083960267, + "8": 0.006606712295029626 + }, + "0.9000": { + "1": 0.0097866399903066, + "2": 0.011010246340202201, + "4": 0.014217749893489784, + "8": 0.02042521801444397 + } + }, + "minutes": 0.3174507300059001 + }, + { + "epoch": 4, + "train_loss": 0.49877583144292464, + "train_main_loss": 0.470812586229202, + "train_aux_spatial_loss": 0.05592649097478617, + "val_loss": 0.4736149037877719, + "val_pr_auc": 0.0029667986240187873, + "val_auroc": 0.8830141091475292, + "val_brier": 0.131561741232872, + "val_brier_skill_score": -564.3106983635114, + "val_ece": 0.2541514431854187, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0010748343359518568, + "val_positive_recall": 0.781615302869288, + "val_far": 0.9989251656640481, + "val_csi": 0.001074511648699702, + "val_f1": 0.0021467166253789772, + "val_f2": 0.005344772333045809, + "val_frequency_bias": 727.1960680127523, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1852, + "fp": 4888523, + "fn": 30, + "precision": 0.00037870306469340285, + "recall": 0.9840595111583422, + "far": 0.9996212969353065, + "csi": 0.00037870074155412484, + "f1": 0.0007571147631859896, + "f2": 0.0018906050201484186, + "frequency_bias": 2598.498937300744, + "predicted_positive_rate": 0.6048701298701299 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1764, + "fp": 3534590, + "fn": 118, + "precision": 0.0004988188399690755, + "recall": 0.9373007438894793, + "far": 0.999501181160031, + "csi": 0.0004988021960869477, + "f1": 0.0009971070329961031, + "f2": 0.0024887961845230736, + "frequency_bias": 1879.0403825717322, + "predicted_positive_rate": 0.4373969078540507 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1675, + "fp": 2659269, + "fn": 207, + "precision": 0.0006294758551852275, + "recall": 0.8900106269925611, + "far": 0.9993705241448148, + "csi": 0.0006294268908453522, + "f1": 0.001258061923685588, + "f2": 0.0031385002353406747, + "frequency_bias": 1413.8916046758768, + "predicted_positive_rate": 0.32912108843537413 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1471, + "fp": 1367112, + "fn": 411, + "precision": 0.0010748343359518568, + "recall": 0.781615302869288, + "far": 0.9989251656640481, + "csi": 0.001074511648699702, + "f1": 0.0021467166253789772, + "f2": 0.005344772333045809, + "frequency_bias": 727.1960680127523, + "predicted_positive_rate": 0.1692743351886209 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1158, + "fp": 711745, + "fn": 724, + "precision": 0.001624344405900943, + "recall": 0.6153028692879915, + "far": 0.998375655594099, + "csi": 0.001622696450666805, + "f1": 0.003240135145533272, + "f2": 0.00803685571553695, + "frequency_bias": 378.8007438894793, + "predicted_positive_rate": 0.08817600494743352 + }, + "0.9000": { + "threshold": 0.9, + "tp": 796, + "fp": 316530, + "fn": 1086, + "precision": 0.0025084613299887182, + "recall": 0.42295430393198724, + "far": 0.9974915386700113, + "csi": 0.002499905782445385, + "f1": 0.004987343675597103, + "f2": 0.012251657667752283, + "frequency_bias": 168.61105207226356, + "predicted_positive_rate": 0.039248732220160794 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004007420968264341, + "recall": 0.17215727269649506, + "lift": 17.21572744090257 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002221397589892149, + "recall": 0.4771519601345062, + "lift": 9.543039213577943 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015114409616217017, + "recall": 0.6493092179298401, + "lift": 6.493092650948695 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007882371861520632, + "2": 0.0008068278383324134, + "4": 0.0008376526392290806, + "8": 0.0008803104743912993 + }, + "0.2000": { + "1": 0.0010456654051311798, + "2": 0.0010781494695234484, + "4": 0.00113452177870621, + "8": 0.0012067720438921903 + }, + "0.3000": { + "1": 0.001341867235808647, + "2": 0.0013990919727997486, + "4": 0.0014963756841404052, + "8": 0.0016008149955504258 + }, + "0.5000": { + "1": 0.002401290586052096, + "2": 0.002581335230269266, + "4": 0.0028727047230912017, + "8": 0.0031094243892790807 + }, + "0.7000": { + "1": 0.0037611063108259346, + "2": 0.004170442864082857, + "4": 0.004911188688805468, + "8": 0.005619346494556421 + }, + "0.9000": { + "1": 0.00620797412047025, + "2": 0.0072512678310659995, + "4": 0.009527837752599244, + "8": 0.012433301157436127 + } + }, + "minutes": 0.34307103554407753 + }, + { + "epoch": 5, + "train_loss": 0.482236114504778, + "train_main_loss": 0.45522032067156104, + "train_aux_spatial_loss": 0.05403158970131391, + "val_loss": 0.6312598725159962, + "val_pr_auc": 0.001904165446995025, + "val_auroc": 0.8616241410980919, + "val_brier": 0.17233318090438843, + "val_brier_skill_score": -739.5024434559806, + "val_ece": 0.2959978739301973, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0008378176445403336, + "val_positive_recall": 0.7954303931987248, + "val_far": 0.9991621823554596, + "val_csi": 0.0008376371581886446, + "val_f1": 0.0016738722188087553, + "val_f2": 0.004171512996896305, + "val_frequency_bias": 949.4075451647184, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1849, + "fp": 5330029, + "fn": 33, + "precision": 0.00034678212817322525, + "recall": 0.9824654622741764, + "far": 0.9996532178718268, + "csi": 0.00034677998188641936, + "f1": 0.000693319534437245, + "f2": 0.001731466009514916, + "frequency_bias": 2833.0913921360257, + "predicted_positive_rate": 0.6594777983920841 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1760, + "fp": 3845604, + "fn": 122, + "precision": 0.0004574560660233864, + "recall": 0.9351753453772582, + "far": 0.9995425439339766, + "csi": 0.000457441560541091, + "f1": 0.0009144648068738656, + "f2": 0.0022828136300575997, + "frequency_bias": 2044.2954303931988, + "predicted_positive_rate": 0.4758644403215832 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1689, + "fp": 2918064, + "fn": 193, + "precision": 0.0005784735900605291, + "recall": 0.8974495217853348, + "far": 0.9994215264099394, + "csi": 0.0005784353546264212, + "f1": 0.0011562019211845422, + "f2": 0.002884929735136463, + "frequency_bias": 1551.4096705632305, + "predicted_positive_rate": 0.36113209647495365 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1497, + "fp": 1785288, + "fn": 385, + "precision": 0.0008378176445403336, + "recall": 0.7954303931987248, + "far": 0.9991621823554596, + "csi": 0.0008376371581886446, + "f1": 0.0016738722188087553, + "f2": 0.004171512996896305, + "frequency_bias": 949.4075451647184, + "predicted_positive_rate": 0.221 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1281, + "fp": 1103534, + "fn": 601, + "precision": 0.0011594701375343384, + "recall": 0.6806588735387885, + "far": 0.9988405298624656, + "csi": 0.0011588397490175644, + "f1": 0.002314996787738649, + "f2": 0.005758115976816504, + "frequency_bias": 587.0430393198725, + "predicted_positive_rate": 0.13664996907854052 + }, + "0.9000": { + "threshold": 0.9, + "tp": 929, + "fp": 573131, + "fn": 953, + "precision": 0.0016182977389123088, + "recall": 0.49362380446333687, + "far": 0.9983817022610877, + "csi": 0.0016156156469505907, + "f1": 0.003226019286664282, + "f2": 0.007986753509357139, + "frequency_bias": 305.02656748140276, + "predicted_positive_rate": 0.07100309214594928 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.002671614056453109, + "recall": 0.1147715225815773, + "lift": 11.477151960678897 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0017588125774636865, + "recall": 0.37778958678245544, + "lift": 7.555791665770626 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0013654916547238827, + "recall": 0.586609959602356, + "lift": 5.866100002150498 + } + }, + "val_fss": { + "0.1000": { + "1": 0.000720903487296698, + "2": 0.0007379622256847851, + "4": 0.0007659140531370845, + "8": 0.0008043442027690917 + }, + "0.2000": { + "1": 0.0009597972490167361, + "2": 0.0009899780347129017, + "4": 0.0010392513227689415, + "8": 0.0010982538039768786 + }, + "0.3000": { + "1": 0.0012276114505155311, + "2": 0.0012735628537789978, + "4": 0.0013438876742029038, + "8": 0.0014172341902246588 + }, + "0.5000": { + "1": 0.0018178385262409558, + "2": 0.0018914580444638895, + "4": 0.002005861068742676, + "8": 0.0021171275849246607 + }, + "0.7000": { + "1": 0.0025095121564123746, + "2": 0.0026071244397790627, + "4": 0.002787665044875265, + "8": 0.003009586547953802 + }, + "0.9000": { + "1": 0.00349269641993466, + "2": 0.0036221493430278473, + "4": 0.004001940413619787, + "8": 0.004630463384076355 + } + }, + "minutes": 0.32958046595255536 + }, + { + "epoch": 6, + "train_loss": 0.4705298146818342, + "train_main_loss": 0.4441832022530821, + "train_aux_spatial_loss": 0.052693225361118125, + "val_loss": 0.3288325930635134, + "val_pr_auc": 0.0028623518804880635, + "val_auroc": 0.8795604343470619, + "val_brier": 0.08432164788246155, + "val_brier_skill_score": -361.32364519424686, + "val_ece": 0.1792877230218383, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0015739775487745805, + "val_positive_recall": 0.6923485653560042, + "val_far": 0.9984260224512255, + "val_csi": 0.001572877460412497, + "val_f1": 0.0031408148040124335, + "val_f2": 0.007798967399957145, + "val_frequency_bias": 439.8719447396387, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1758, + "fp": 3773348, + "fn": 124, + "precision": 0.0004656822881264791, + "recall": 0.9341126461211477, + "far": 0.9995343177118735, + "csi": 0.0004656669924746307, + "f1": 0.0009309004953153147, + "f2": 0.002323777558177714, + "frequency_bias": 2005.9011689691818, + "predicted_positive_rate": 0.46692714904143473 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1577, + "fp": 2267086, + "fn": 305, + "precision": 0.0006951230746920102, + "recall": 0.8379383634431455, + "far": 0.999304876925308, + "csi": 0.000695029634618029, + "f1": 0.001389093807874321, + "f2": 0.0034641205417295824, + "frequency_bias": 1205.4532412327312, + "predicted_positive_rate": 0.28060148423005565 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1469, + "fp": 1517955, + "fn": 413, + "precision": 0.0009668137399435576, + "recall": 0.7805526036131775, + "far": 0.9990331862600564, + "csi": 0.0009665510182999888, + "f1": 0.0019312353990584405, + "f2": 0.004810236340107613, + "frequency_bias": 807.3453772582359, + "predicted_positive_rate": 0.18793123067408782 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1303, + "fp": 826536, + "fn": 579, + "precision": 0.0015739775487745805, + "recall": 0.6923485653560042, + "far": 0.9984260224512255, + "csi": 0.001572877460412497, + "f1": 0.0031408148040124335, + "f2": 0.007798967399957145, + "frequency_bias": 439.8719447396387, + "predicted_positive_rate": 0.10239196042053185 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1048, + "fp": 466249, + "fn": 834, + "precision": 0.0022426850589667814, + "recall": 0.5568544102019128, + "far": 0.9977573149410333, + "csi": 0.002238689597569911, + "f1": 0.004467378122209221, + "f2": 0.011035644711209392, + "frequency_bias": 248.298087141339, + "predicted_positive_rate": 0.057798021026592455 + }, + "0.9000": { + "threshold": 0.9, + "tp": 742, + "fp": 226824, + "fn": 1140, + "precision": 0.0032605925313974847, + "recall": 0.3942614240170032, + "far": 0.9967394074686026, + "csi": 0.0032443398948868852, + "f1": 0.006467696384365957, + "f2": 0.015780921673883636, + "frequency_bias": 120.91710945802338, + "predicted_positive_rate": 0.028146691403834263 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004848484881222248, + "recall": 0.20828905701637268, + "lift": 20.82890589171393 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002466295612975955, + "recall": 0.5297555923461914, + "lift": 10.595111768374379 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0016029684338718653, + "recall": 0.6886290907859802, + "lift": 6.8862911764073385 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0009832075371584148, + "2": 0.0010169042550060015, + "4": 0.0010721484966499384, + "8": 0.0011368377314868727 + }, + "0.2000": { + "1": 0.0015207106192908215, + "2": 0.0016047733082307225, + "4": 0.0017274780230661646, + "8": 0.0018439819139227653 + }, + "0.3000": { + "1": 0.002164479861285651, + "2": 0.002303876071255262, + "4": 0.002502814516543217, + "8": 0.002677417485086986 + }, + "0.5000": { + "1": 0.0035466153498271424, + "2": 0.0038054359562748052, + "4": 0.004256125038073377, + "8": 0.004679745806242752 + }, + "0.7000": { + "1": 0.005259027399887928, + "2": 0.0058170650135721536, + "4": 0.006905022198597899, + "8": 0.008165048178681578 + }, + "0.9000": { + "1": 0.008040151750769331, + "2": 0.009007045143097692, + "4": 0.011282893901345625, + "8": 0.014929724649083487 + } + }, + "minutes": 0.33652125199635824 + }, + { + "epoch": 7, + "train_loss": 0.462470636422584, + "train_main_loss": 0.4368088399746645, + "train_aux_spatial_loss": 0.05132359440849709, + "val_loss": 0.2877284549176693, + "val_pr_auc": 0.0013908105247300113, + "val_auroc": 0.8529700408228243, + "val_brier": 0.07498431950807571, + "val_brier_skill_score": -321.20186226017756, + "val_ece": 0.16345792957978955, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.00137407744359688, + "val_positive_recall": 0.5425079702444209, + "val_far": 0.9986259225564031, + "val_csi": 0.001372487078323173, + "val_f1": 0.002741211878763797, + "val_f2": 0.006801479405040423, + "val_frequency_bias": 394.8161530286929, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1712, + "fp": 3476150, + "fn": 170, + "precision": 0.0004922564495083474, + "recall": 0.9096705632306057, + "far": 0.9995077435504917, + "csi": 0.000492232388891189, + "f1": 0.0009839804307443306, + "f2": 0.002455966190297212, + "frequency_bias": 1847.9606801275238, + "predicted_positive_rate": 0.4301622758194187 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1530, + "fp": 2143152, + "fn": 352, + "precision": 0.0007133924749683169, + "recall": 0.8129649309245484, + "far": 0.9992866075250317, + "csi": 0.0007132754072895814, + "f1": 0.0014255340162231363, + "f2": 0.003554485854075578, + "frequency_bias": 1139.575982996812, + "predicted_positive_rate": 0.2652667903525046 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1354, + "fp": 1448620, + "fn": 528, + "precision": 0.000933809847624854, + "recall": 0.7194473963868225, + "far": 0.9990661901523752, + "csi": 0.0009334699297208829, + "f1": 0.0018651987524933604, + "f2": 0.00464493359185785, + "frequency_bias": 770.4431455897981, + "predicted_positive_rate": 0.1793412492269635 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1021, + "fp": 742023, + "fn": 861, + "precision": 0.00137407744359688, + "recall": 0.5425079702444209, + "far": 0.9986259225564031, + "csi": 0.001372487078323173, + "f1": 0.002741211878763797, + "f2": 0.006801479405040423, + "frequency_bias": 394.8161530286929, + "predicted_positive_rate": 0.09190401978973407 + }, + "0.7000": { + "threshold": 0.7, + "tp": 669, + "fp": 387288, + "fn": 1213, + "precision": 0.0017244179122944037, + "recall": 0.3554729011689692, + "far": 0.9982755820877056, + "csi": 0.001719043091707994, + "f1": 0.0034321861075982648, + "f2": 0.008457969328798816, + "frequency_bias": 206.14080765143464, + "predicted_positive_rate": 0.0479847866419295 + }, + "0.9000": { + "threshold": 0.9, + "tp": 330, + "fp": 170442, + "fn": 1552, + "precision": 0.00193240109619844, + "recall": 0.17534537725823593, + "far": 0.9980675989038016, + "csi": 0.0019149973306097816, + "f1": 0.0038226742502345734, + "f2": 0.009254066180594505, + "frequency_bias": 90.73963868225292, + "predicted_positive_rate": 0.02112207792207792 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.002251082332804799, + "recall": 0.09670563042163849, + "lift": 9.67056373550477 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0017093382775783539, + "recall": 0.3671625852584839, + "lift": 7.343251962829395 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0013395176501944661, + "recall": 0.575451672077179, + "lift": 5.754516670609246 + } + }, + "val_fss": { + "0.1000": { + "1": 0.001043168774059744, + "2": 0.0010819388013025844, + "4": 0.0011386362023144692, + "8": 0.0011957179318650146 + }, + "0.2000": { + "1": 0.0015483993663276685, + "2": 0.001623910048448085, + "4": 0.001730854780802371, + "8": 0.0018125949671790043 + }, + "0.3000": { + "1": 0.0020767730914419413, + "2": 0.002216874591054907, + "4": 0.0024268280557646627, + "8": 0.002582330742552652 + }, + "0.5000": { + "1": 0.003179344493123515, + "2": 0.0034580954569070066, + "4": 0.004022383332455304, + "8": 0.004531362170723141 + }, + "0.7000": { + "1": 0.004287884524174168, + "2": 0.004893582597532031, + "4": 0.006272511260630553, + "8": 0.00800338321521643 + }, + "0.9000": { + "1": 0.00525683498068652, + "2": 0.006461524593853074, + "4": 0.009610762688246233, + "8": 0.014847628576755034 + } + }, + "minutes": 0.3354960521062215 + }, + { + "epoch": 8, + "train_loss": 0.45163499733658824, + "train_main_loss": 0.42649205012479513, + "train_aux_spatial_loss": 0.05028589440088308, + "val_loss": 0.389868001639843, + "val_pr_auc": 0.004338335499478623, + "val_auroc": 0.8726679507819024, + "val_brier": 0.11727461218833923, + "val_brier_skill_score": -502.9200020859495, + "val_ece": 0.24583306557241666, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0011231134016327171, + "val_positive_recall": 0.7454835281615303, + "val_far": 0.9988768865983673, + "val_csi": 0.0011226829160948559, + "val_f1": 0.0022428478252529, + "val_f2": 0.005581929031919244, + "val_frequency_bias": 663.7651434643996, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1825, + "fp": 5047814, + "fn": 57, + "precision": 0.0003614119742025123, + "recall": 0.9697130712008502, + "far": 0.9996385880257975, + "csi": 0.0003614078946534603, + "f1": 0.0007225546523512424, + "f2": 0.0018043699169910743, + "frequency_bias": 2683.123804463337, + "predicted_positive_rate": 0.6245688311688312 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1727, + "fp": 3624057, + "fn": 155, + "precision": 0.0004763107785791983, + "recall": 0.9176408076514346, + "far": 0.9995236892214208, + "csi": 0.00047629041746151824, + "f1": 0.0009521273457920327, + "f2": 0.002376619459050035, + "frequency_bias": 1926.5589798087142, + "predicted_positive_rate": 0.4484581323438466 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1618, + "fp": 2596717, + "fn": 264, + "precision": 0.0006227064639471046, + "recall": 0.8597236981934112, + "far": 0.9993772935360529, + "csi": 0.0006226432012018784, + "f1": 0.0012445115157696453, + "f2": 0.0031045377289596575, + "frequency_bias": 1380.624335812965, + "predicted_positive_rate": 0.32137724180581323 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1403, + "fp": 1247803, + "fn": 479, + "precision": 0.0011231134016327171, + "recall": 0.7454835281615303, + "far": 0.9988768865983673, + "csi": 0.0011226829160948559, + "f1": 0.0022428478252529, + "f2": 0.005581929031919244, + "frequency_bias": 663.7651434643996, + "predicted_positive_rate": 0.1545090909090909 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1064, + "fp": 572445, + "fn": 818, + "precision": 0.0018552455148916581, + "recall": 0.565356004250797, + "far": 0.9981447544851083, + "csi": 0.001852603133754812, + "f1": 0.0036983546840322493, + "f2": 0.009156043418921686, + "frequency_bias": 304.7337938363443, + "predicted_positive_rate": 0.07093494124922696 + }, + "0.9000": { + "threshold": 0.9, + "tp": 616, + "fp": 143363, + "fn": 1266, + "precision": 0.004278401711360684, + "recall": 0.3273113708820404, + "far": 0.9957215982886393, + "csi": 0.004241109848876037, + "f1": 0.008446397597712891, + "f2": 0.02032909370524134, + "frequency_bias": 76.50318809776833, + "predicted_positive_rate": 0.017808163265306123 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005640074145048857, + "recall": 0.24229542911052704, + "lift": 24.229543139236732 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002325293840840459, + "recall": 0.4994686543941498, + "lift": 9.989373539974505 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015052566304802895, + "recall": 0.6466525197029114, + "lift": 6.466525000537624 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007513765476421241, + "2": 0.0007711290741327925, + "4": 0.0008023612192573637, + "8": 0.0008460677286081797 + }, + "0.2000": { + "1": 0.0010087632973743155, + "2": 0.001044669279142485, + "4": 0.0010981556816972438, + "8": 0.0011592603845778093 + }, + "0.3000": { + "1": 0.00134257679793226, + "2": 0.0014032317950684048, + "4": 0.0014828012007669367, + "8": 0.0015567722214533397 + }, + "0.5000": { + "1": 0.002475717082535134, + "2": 0.0025902566873002852, + "4": 0.0027648772728405824, + "8": 0.0028595649359570707 + }, + "0.7000": { + "1": 0.004229141272194981, + "2": 0.004534755794878054, + "4": 0.004987145697110007, + "8": 0.005228016705880245 + }, + "0.9000": { + "1": 0.011542436340720164, + "2": 0.013278581972967496, + "4": 0.017111917801594156, + "8": 0.022356010708446727 + } + }, + "minutes": 0.34035869836807253 + }, + { + "epoch": 9, + "train_loss": 0.4406228200723544, + "train_main_loss": 0.4159060165082645, + "train_aux_spatial_loss": 0.04943360574618867, + "val_loss": 0.31516960461934407, + "val_pr_auc": 0.005532091282032133, + "val_auroc": 0.8821471865606735, + "val_brier": 0.07678419351577759, + "val_brier_skill_score": -328.9357826440623, + "val_ece": 0.18066359303804805, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0017832156643714674, + "val_positive_recall": 0.6514346439957492, + "val_far": 0.9982167843356285, + "val_csi": 0.0017815158287536074, + "val_f1": 0.003556695348445904, + "val_f2": 0.008819509387813826, + "val_frequency_bias": 365.31455897980874, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1795, + "fp": 4233910, + "fn": 87, + "precision": 0.00042377833206042444, + "recall": 0.9537725823591924, + "far": 0.9995762216679396, + "csi": 0.00042376962797040083, + "f1": 0.0008471802466828409, + "f2": 0.002115132494491818, + "frequency_bias": 2250.640276301807, + "predicted_positive_rate": 0.5238967223252937 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1595, + "fp": 2216109, + "fn": 287, + "precision": 0.0007192123024533482, + "recall": 0.8475026567481403, + "far": 0.9992807876975467, + "csi": 0.0007191192389869932, + "f1": 0.0014372049562395872, + "f2": 0.0035838959712964765, + "frequency_bias": 1178.376195536663, + "predicted_positive_rate": 0.2742985776128633 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1456, + "fp": 1317007, + "fn": 426, + "precision": 0.0011043161620766, + "recall": 0.7736450584484591, + "far": 0.9988956838379234, + "csi": 0.0011039594689166413, + "f1": 0.0022054841726972876, + "f2": 0.005490233342458584, + "frequency_bias": 700.5648246546227, + "predicted_positive_rate": 0.1630752009894867 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1226, + "fp": 686296, + "fn": 656, + "precision": 0.0017832156643714674, + "recall": 0.6514346439957492, + "far": 0.9982167843356285, + "csi": 0.0017815158287536074, + "f1": 0.003556695348445904, + "f2": 0.008819509387813826, + "frequency_bias": 365.31455897980874, + "predicted_positive_rate": 0.08503673469387756 + }, + "0.7000": { + "threshold": 0.7, + "tp": 959, + "fp": 404322, + "fn": 923, + "precision": 0.0023662594594861344, + "recall": 0.5095642933049946, + "far": 0.9976337405405139, + "csi": 0.0023608827091806087, + "f1": 0.004710644140061843, + "f2": 0.011615541327829578, + "frequency_bias": 215.34590860786398, + "predicted_positive_rate": 0.05012752009894867 + }, + "0.9000": { + "threshold": 0.9, + "tp": 736, + "fp": 206564, + "fn": 1146, + "precision": 0.0035504100337674866, + "recall": 0.39107332624867164, + "far": 0.9964495899662326, + "csi": 0.0035308904944206174, + "f1": 0.007036934344255241, + "f2": 0.01712998305621241, + "frequency_bias": 110.14877789585547, + "predicted_positive_rate": 0.025640074211502783 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.006629561074078083, + "recall": 0.2848033905029297, + "lift": 28.480341198987567 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0023698206059634686, + "recall": 0.5090329647064209, + "lift": 10.180658822517413 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015807050513103604, + "recall": 0.6790648102760315, + "lift": 6.790648535135884 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0008930744755271114, + "2": 0.000921422318230225, + "4": 0.0009668763668795322, + "8": 0.0010260119580676186 + }, + "0.2000": { + "1": 0.0015794562705196968, + "2": 0.0016669067534983428, + "4": 0.001799706921313815, + "8": 0.0019474635373948335 + }, + "0.3000": { + "1": 0.00246002599564632, + "2": 0.0026177117656231985, + "4": 0.002871564863597964, + "8": 0.003131757057791784 + }, + "0.5000": { + "1": 0.004021237206644379, + "2": 0.004332716953577953, + "4": 0.004884649782999739, + "8": 0.0054515150420901826 + }, + "0.7000": { + "1": 0.005615133103695769, + "2": 0.006360224351620203, + "4": 0.007663492450681542, + "8": 0.00913398414028943 + }, + "0.9000": { + "1": 0.00885985202765427, + "2": 0.01012085108726446, + "4": 0.012505376077093011, + "8": 0.015618568706976754 + } + }, + "minutes": 0.33895504077275596 + }, + { + "epoch": 10, + "train_loss": 0.4308571537754615, + "train_main_loss": 0.4065246517781404, + "train_aux_spatial_loss": 0.04866500381171554, + "val_loss": 0.2545071696241697, + "val_pr_auc": 0.00492243368142961, + "val_auroc": 0.8938939230518667, + "val_brier": 0.06364420056343079, + "val_brier_skill_score": -272.47424205655614, + "val_ece": 0.15454458198181653, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.002100446914367563, + "val_positive_recall": 0.6615302869287991, + "val_far": 0.9978995530856324, + "val_csi": 0.0020981920157473947, + "val_f1": 0.004187597647545546, + "val_f2": 0.010370523390736333, + "val_frequency_bias": 314.9473963868225, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1759, + "fp": 3554529, + "fn": 123, + "precision": 0.000494616858927061, + "recall": 0.934643995749203, + "far": 0.999505383141073, + "csi": 0.0004945997523908232, + "f1": 0.0009887104888186905, + "f2": 0.0024678602935729564, + "frequency_bias": 1889.6323060573857, + "predicted_positive_rate": 0.43986246134817564 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1593, + "fp": 1928992, + "fn": 289, + "precision": 0.0008251384942905907, + "recall": 0.8464399574920297, + "far": 0.9991748615057094, + "csi": 0.0008250149932103285, + "f1": 0.0016486698091092889, + "f2": 0.004109667496167664, + "frequency_bias": 1025.8156216790649, + "predicted_positive_rate": 0.23878602350030922 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1485, + "fp": 1212388, + "fn": 397, + "precision": 0.0012233569739173702, + "recall": 0.7890541976620616, + "far": 0.9987766430260826, + "csi": 0.0012229570029729797, + "f1": 0.0024429264119826774, + "f2": 0.006079084592201907, + "frequency_bias": 644.990967056323, + "predicted_positive_rate": 0.15013889919604206 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1245, + "fp": 591486, + "fn": 637, + "precision": 0.002100446914367563, + "recall": 0.6615302869287991, + "far": 0.9978995530856324, + "csi": 0.0020981920157473947, + "f1": 0.004187597647545546, + "f2": 0.010370523390736333, + "frequency_bias": 314.9473963868225, + "predicted_positive_rate": 0.07331243042671615 + }, + "0.7000": { + "threshold": 0.7, + "tp": 948, + "fp": 305265, + "fn": 934, + "precision": 0.0030958842374425644, + "recall": 0.5037194473963869, + "far": 0.9969041157625574, + "csi": 0.0030864699964512107, + "f1": 0.0061539460231422125, + "f2": 0.015108003098096838, + "frequency_bias": 162.70616365568543, + "predicted_positive_rate": 0.03787421150278293 + }, + "0.9000": { + "threshold": 0.9, + "tp": 697, + "fp": 143067, + "fn": 1185, + "precision": 0.004848223477365683, + "recall": 0.3703506907545165, + "far": 0.9951517765226343, + "csi": 0.004808587848139691, + "f1": 0.009571151971217886, + "f2": 0.023034925838775352, + "frequency_bias": 76.38894792773645, + "predicted_positive_rate": 0.017781570810142238 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.006716141011565924, + "recall": 0.2885228395462036, + "lift": 28.852285304047886 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0026839827187359333, + "recall": 0.5765143632888794, + "lift": 11.530287261501037 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001667285105213523, + "recall": 0.7162593007087708, + "lift": 7.162593140311982 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010543004451490923, + "2": 0.0010971495172527046, + "4": 0.0011617131253165436, + "8": 0.001235067155033076 + }, + "0.2000": { + "1": 0.0018153448265396666, + "2": 0.00192737425428402, + "4": 0.0020974439448524507, + "8": 0.0022549619868982296 + }, + "0.3000": { + "1": 0.0027408731997776714, + "2": 0.002930739084691436, + "4": 0.0032168364951372652, + "8": 0.0034672212918110956 + }, + "0.5000": { + "1": 0.0048490908581931835, + "2": 0.0052456592900168575, + "4": 0.005907669840747531, + "8": 0.006570117783041063 + }, + "0.7000": { + "1": 0.007577933898357814, + "2": 0.008558571828157113, + "4": 0.010329712940135849, + "8": 0.012440375731332498 + }, + "0.9000": { + "1": 0.012198325070619909, + "2": 0.013721391504927793, + "4": 0.017129504146446983, + "8": 0.022312787696272984 + } + }, + "minutes": 0.3265586694081624 + } +] diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/run_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/run_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..6e9588c714c891a0f6c0147756632a7ec7df4c48 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/run_summary.json @@ -0,0 +1,172 @@ +{ + "run_name": "reference_fireprone_seed_42_20260504", + "device": "cuda", + "index_root": "", + "input_channels": 16, + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "train_positive_rate": 0.0018277173713066195, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight": 50.0, + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "aux_positive_rate": 0.004560296846011132, + "init_checkpoint": "", + "best_val_pr_auc": 0.005532091282032133, + "best_val_auroc": 0.8821471865606735, + "best_test_pr_auc": 0.0016100872246761122, + "best_test_auroc": 0.8121210769028695, + "best_test_brier": 0.09570477902889252, + "best_test_ece": 0.2090970246477704, + "best_test_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0027087198104709387, + "recall": 0.11167771369218826, + "lift": 11.16777118374645 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018082869937643409, + "recall": 0.37276899814605713, + "lift": 7.455379955815317 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012504637707024813, + "recall": 0.5155532956123352, + "lift": 5.155532591738293 + } + }, + "best_test_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1806, + "fp": 4593586, + "fn": 155, + "precision": 0.00039300238151609263, + "recall": 0.9209586945436002, + "far": 0.9996069976184839, + "csi": 0.0003929891262128317, + "f1": 0.000785669492858173, + "f2": 0.0019616634906400626, + "frequency_bias": 2343.392146863845, + "predicted_positive_rate": 0.5683849103277675 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1550, + "fp": 2681763, + "fn": 411, + "precision": 0.0005776441287319072, + "recall": 0.7904130545639979, + "far": 0.999422355871268, + "csi": 0.0005775556651876273, + "f1": 0.0011544445743711814, + "f2": 0.002879802256055667, + "frequency_bias": 1368.3391126976032, + "predicted_positive_rate": 0.3318878169449598 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1334, + "fp": 1753061, + "fn": 627, + "precision": 0.0007603760840631671, + "recall": 0.6802651708312085, + "far": 0.9992396239159368, + "csi": 0.0007601044317393172, + "f1": 0.0015190542236312, + "f2": 0.003784957658978152, + "frequency_bias": 894.6430392656807, + "predicted_positive_rate": 0.21699381570810142 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1069, + "fp": 930711, + "fn": 892, + "precision": 0.0011472665221404193, + "recall": 0.5451300356960734, + "far": 0.9988527334778596, + "csi": 0.0011461692856652714, + "f1": 0.0022897141712744755, + "f2": 0.0056884455910023585, + "frequency_bias": 475.15553289138194, + "predicted_positive_rate": 0.11524799010513297 + }, + "0.7000": { + "threshold": 0.7, + "tp": 843, + "fp": 514028, + "fn": 1118, + "precision": 0.0016373033245220647, + "recall": 0.4298827129015808, + "far": 0.998362696675478, + "csi": 0.0016337557583591899, + "f1": 0.0032621819082409758, + "f2": 0.008063667581760615, + "frequency_bias": 262.5553289138195, + "predicted_positive_rate": 0.06368225108225108 + }, + "0.9000": { + "threshold": 0.9, + "tp": 494, + "fp": 230401, + "fn": 1467, + "precision": 0.0021395006388185106, + "recall": 0.2519122896481387, + "far": 0.9978604993611815, + "csi": 0.002125993062548954, + "f1": 0.004242965609647164, + "f2": 0.010346026413782414, + "frequency_bias": 117.74349821519633, + "predicted_positive_rate": 0.028558441558441557 + } + }, + "best_test_fss": { + "0.1000": { + "1": 0.0008268794294157278, + "2": 0.0008514076234114043, + "4": 0.0008867121111854059, + "8": 0.0009292544410690606 + }, + "0.2000": { + "1": 0.001245705784772344, + "2": 0.0012999708169816326, + "4": 0.0013747560191506025, + "8": 0.0014380244101879214 + }, + "0.3000": { + "1": 0.0016567546208199335, + "2": 0.0017403972985233773, + "4": 0.0018648856518644896, + "8": 0.001957178150997607 + }, + "0.5000": { + "1": 0.002557529963904237, + "2": 0.0027217819781574226, + "4": 0.0029546256500705415, + "8": 0.003125849885388443 + }, + "0.7000": { + "1": 0.0037552341559934366, + "2": 0.004108313787275497, + "4": 0.004633106027329581, + "8": 0.005223108606388305 + }, + "0.9000": { + "1": 0.005267983383194896, + "2": 0.006025153257028237, + "4": 0.007611176315223789, + "8": 0.009809039775823591 + } + }, + "epochs": 10 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/tile_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/tile_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..06b28297f37a2574afa4aa9a9b9e9d027adeb73e --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_42_20260504/tile_summary.json @@ -0,0 +1,16 @@ +{ + "index_root": "", + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "input_channels": 16, + "train_positive_rate": 0.0018277173713066195, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight_source": "tiles", + "positive_rate_source": "tiles", + "pos_weight": 50.0, + "loss_type": "bce", + "train_target_mode": "dilate_max", + "train_target_radius": 2 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/history.json b/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/history.json new file mode 100644 index 0000000000000000000000000000000000000000..06bea4d3b4fe37ea265d90d780631e02919bb805 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/history.json @@ -0,0 +1,1682 @@ +[ + { + "epoch": 1, + "train_loss": 0.5795233543991986, + "train_main_loss": 0.5336806391359897, + "train_aux_spatial_loss": 0.0916854308093406, + "val_loss": 0.5250215088327725, + "val_pr_auc": 0.00302956816406225, + "val_auroc": 0.8476268012299131, + "val_brier": 0.15357030928134918, + "val_brier_skill_score": -658.879825047864, + "val_ece": 0.28153358592477284, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0008681095747196535, + "val_positive_recall": 0.7412327311370882, + "val_far": 0.9991318904252804, + "val_csi": 0.0008678465647273562, + "val_f1": 0.001734188120251961, + "val_f2": 0.004320308609399505, + "val_frequency_bias": 853.84697130712, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1838, + "fp": 5383727, + "fn": 44, + "precision": 0.0003412826695063563, + "recall": 0.9766206163655685, + "far": 0.9996587173304936, + "csi": 0.00034127988125391207, + "f1": 0.0006823268980650761, + "f2": 0.0017040314342808473, + "frequency_bias": 2861.6179596174284, + "predicted_positive_rate": 0.6661181199752628 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1736, + "fp": 3897785, + "fn": 146, + "precision": 0.0004451828827181595, + "recall": 0.922422954303932, + "far": 0.9995548171172819, + "csi": 0.0004451662154742956, + "f1": 0.0008899362613910945, + "f2": 0.0022216255798174016, + "frequency_bias": 2072.009032943677, + "predicted_positive_rate": 0.4823155225726654 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1615, + "fp": 2934454, + "fn": 267, + "precision": 0.0005500551928445823, + "recall": 0.8581296493092455, + "far": 0.9994499448071554, + "csi": 0.0005500051765193084, + "f1": 0.0010994056742266976, + "f2": 0.002743242366397302, + "frequency_bias": 1560.0791710945803, + "predicted_positive_rate": 0.36315015460729744 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1395, + "fp": 1605545, + "fn": 487, + "precision": 0.0008681095747196535, + "recall": 0.7412327311370882, + "far": 0.9991318904252804, + "csi": 0.0008678465647273562, + "f1": 0.001734188120251961, + "f2": 0.004320308609399505, + "frequency_bias": 853.84697130712, + "predicted_positive_rate": 0.1987557204700062 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1088, + "fp": 910358, + "fn": 794, + "precision": 0.001193707581140298, + "recall": 0.5781083953241233, + "far": 0.9988062924188597, + "csi": 0.0011926685959835132, + "f1": 0.002382495664208258, + "f2": 0.005919645169504252, + "frequency_bias": 484.29649309245485, + "predicted_positive_rate": 0.11273296227581941 + }, + "0.9000": { + "threshold": 0.9, + "tp": 701, + "fp": 394270, + "fn": 1181, + "precision": 0.0017748138470925713, + "recall": 0.37247608926673753, + "far": 0.9982251861529075, + "csi": 0.0017695228094266847, + "f1": 0.0035327942588313558, + "f2": 0.008708096169183027, + "frequency_bias": 209.86769394261424, + "predicted_positive_rate": 0.048852319109461965 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0043290043249726295, + "recall": 0.18597236275672913, + "lift": 18.59723726042579 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0017637599958106875, + "recall": 0.3788522779941559, + "lift": 7.577045586053171 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012479900615289807, + "recall": 0.5361317992210388, + "lift": 5.361317645034825 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007067254219508268, + "2": 0.0007234213847714388, + "4": 0.0007495019182500728, + "8": 0.0007830543185699534 + }, + "0.2000": { + "1": 0.0009375066389885056, + "2": 0.0009701896382775521, + "4": 0.0010247521613826471, + "8": 0.0010816868830242354 + }, + "0.3000": { + "1": 0.0011841353931208998, + "2": 0.0012332094695163276, + "4": 0.0013160793352786637, + "8": 0.0013954395228068517 + }, + "0.5000": { + "1": 0.001895017558836809, + "2": 0.001973506551422588, + "4": 0.0021128830563802437, + "8": 0.0022158294695364678 + }, + "0.7000": { + "1": 0.0026256074379803485, + "2": 0.0027993105699366527, + "4": 0.0030546381753130403, + "8": 0.003188899049464955 + }, + "0.9000": { + "1": 0.004244499691496761, + "2": 0.004616982081301013, + "4": 0.005185018364351035, + "8": 0.005703623485958986 + } + }, + "minutes": 0.38692938486735023 + }, + { + "epoch": 2, + "train_loss": 0.5295870734150671, + "train_main_loss": 0.4984965211945434, + "train_aux_spatial_loss": 0.06218110369714074, + "val_loss": 0.1981954239308834, + "val_pr_auc": 0.002046157994282752, + "val_auroc": 0.8487165040319367, + "val_brier": 0.04589230194687843, + "val_brier_skill_score": -196.19569701633563, + "val_ece": 0.12395327587383405, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.001856401444928432, + "val_positive_recall": 0.3926673751328374, + "val_far": 0.9981435985550716, + "val_csi": 0.0018510864800551068, + "val_f1": 0.0036953325799321942, + "val_f2": 0.00910973595325559, + "val_frequency_bias": 211.52072263549417, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1653, + "fp": 3106762, + "fn": 229, + "precision": 0.0005317822748892925, + "recall": 0.8783209351753454, + "far": 0.9994682177251107, + "csi": 0.0005317431008504029, + "f1": 0.001062921000791886, + "f2": 0.0026524875455038815, + "frequency_bias": 1651.6551540913922, + "predicted_positive_rate": 0.3844669140383426 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1331, + "fp": 1490961, + "fn": 551, + "precision": 0.0008919165954116219, + "recall": 0.7072263549415515, + "far": 0.9991080834045883, + "csi": 0.0008915873939858378, + "f1": 0.0017815863480424637, + "f2": 0.0044371991305623346, + "frequency_bias": 792.9287991498406, + "predicted_positive_rate": 0.18457538651824365 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1067, + "fp": 834474, + "fn": 815, + "precision": 0.0012770169267576337, + "recall": 0.5669500531349628, + "far": 0.9987229830732424, + "csi": 0.0012757725179229898, + "f1": 0.0025482939924028835, + "f2": 0.006328070418909959, + "frequency_bias": 443.9643995749203, + "predicted_positive_rate": 0.10334458874458874 + }, + "0.5000": { + "threshold": 0.5, + "tp": 739, + "fp": 397343, + "fn": 1143, + "precision": 0.001856401444928432, + "recall": 0.3926673751328374, + "far": 0.9981435985550716, + "csi": 0.0018510864800551068, + "f1": 0.0036953325799321942, + "f2": 0.00910973595325559, + "frequency_bias": 211.52072263549417, + "predicted_positive_rate": 0.049237105751391466 + }, + "0.7000": { + "threshold": 0.7, + "tp": 617, + "fp": 206285, + "fn": 1265, + "precision": 0.0029820881383456903, + "recall": 0.32784272051009566, + "far": 0.9970179118616543, + "csi": 0.0029639664307983494, + "f1": 0.0059104145911564095, + "f2": 0.014386979433847877, + "frequency_bias": 109.93730074388948, + "predicted_positive_rate": 0.025590847247990104 + }, + "0.9000": { + "threshold": 0.9, + "tp": 348, + "fp": 86362, + "fn": 1534, + "precision": 0.004013377926421404, + "recall": 0.1849096705632306, + "far": 0.9959866220735786, + "csi": 0.003943610897058157, + "f1": 0.00785623984106917, + "f2": 0.018463889301555635, + "frequency_bias": 46.07332624867163, + "predicted_positive_rate": 0.010724799010513297 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004044526722282171, + "recall": 0.1737513244152069, + "lift": 17.375132343137437 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018429190386086702, + "recall": 0.39585548639297485, + "lift": 7.917109810921228 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0013036486925557256, + "recall": 0.5600425004959106, + "lift": 5.600424998387127 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0011313298295642094, + "2": 0.0011683798708436166, + "4": 0.0012267995062213988, + "8": 0.0012904870520601408 + }, + "0.2000": { + "1": 0.0019838003913644764, + "2": 0.0020951564940783207, + "4": 0.002271241452076689, + "8": 0.0024139105838063557 + }, + "0.3000": { + "1": 0.0029177942126825673, + "2": 0.0031946482222295902, + "4": 0.0036810675049008257, + "8": 0.004083577845361508 + }, + "0.5000": { + "1": 0.0046250396513106695, + "2": 0.005241562890837548, + "4": 0.006354549298866807, + "8": 0.00746207518491393 + }, + "0.7000": { + "1": 0.007309355074624491, + "2": 0.008090759551170335, + "4": 0.009921752700405628, + "8": 0.012786519539836116 + }, + "0.9000": { + "1": 0.01030998864826782, + "2": 0.011657290946334498, + "4": 0.016018656643936313, + "8": 0.025224938209435277 + } + }, + "minutes": 0.3169411182403564 + }, + { + "epoch": 3, + "train_loss": 0.5118337455830854, + "train_main_loss": 0.48268066439472623, + "train_aux_spatial_loss": 0.05830616311089252, + "val_loss": 0.31896051540970805, + "val_pr_auc": 0.001019430946186465, + "val_auroc": 0.8272157621144766, + "val_brier": 0.08598973602056503, + "val_brier_skill_score": -368.49129181739414, + "val_ece": 0.18084020724136254, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0010149243054251504, + "val_positive_recall": 0.4824654622741764, + "val_far": 0.9989850756945748, + "val_csi": 0.0010138205626927432, + "val_f1": 0.0020255875430827746, + "val_f2": 0.005032277515695386, + "val_frequency_bias": 475.3708820403826, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1684, + "fp": 3663042, + "fn": 198, + "precision": 0.0004595159365256775, + "recall": 0.8947927736450585, + "far": 0.9995404840634743, + "csi": 0.0004594911108661462, + "f1": 0.0009185601515078786, + "f2": 0.0022928697197960707, + "frequency_bias": 1947.2507970244421, + "predicted_positive_rate": 0.4532747062461348 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1507, + "fp": 2277895, + "fn": 375, + "precision": 0.0006611383161022057, + "recall": 0.8007438894792773, + "far": 0.9993388616838977, + "csi": 0.0006610295656110225, + "f1": 0.0013211857883542776, + "f2": 0.0032948100728924803, + "frequency_bias": 1211.1594048884165, + "predicted_positive_rate": 0.2819297464440322 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1334, + "fp": 1581254, + "fn": 548, + "precision": 0.000842923110752767, + "recall": 0.7088204038257173, + "far": 0.9991570768892473, + "csi": 0.0008426313342631334, + "f1": 0.0016838438089708232, + "f2": 0.00419466252776527, + "frequency_bias": 840.9075451647184, + "predicted_positive_rate": 0.19574372294372294 + }, + "0.5000": { + "threshold": 0.5, + "tp": 908, + "fp": 893740, + "fn": 974, + "precision": 0.0010149243054251504, + "recall": 0.4824654622741764, + "far": 0.9989850756945748, + "csi": 0.0010138205626927432, + "f1": 0.0020255875430827746, + "f2": 0.005032277515695386, + "frequency_bias": 475.3708820403826, + "predicted_positive_rate": 0.11065528756957328 + }, + "0.7000": { + "threshold": 0.7, + "tp": 621, + "fp": 497577, + "fn": 1261, + "precision": 0.0012464923584598895, + "recall": 0.3299681190223167, + "far": 0.9987535076415401, + "csi": 0.0012433452996141825, + "f1": 0.002483602623580227, + "f2": 0.006139688289706283, + "frequency_bias": 264.7173219978746, + "predicted_positive_rate": 0.06162003710575139 + }, + "0.9000": { + "threshold": 0.9, + "tp": 286, + "fp": 184589, + "fn": 1596, + "precision": 0.0015469912102772144, + "recall": 0.15196599362380447, + "far": 0.9984530087897228, + "csi": 0.001533750556386784, + "f1": 0.0030628035361459013, + "f2": 0.007432316543920833, + "frequency_bias": 98.23326248671626, + "predicted_positive_rate": 0.022866419294990725 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0017439703224226832, + "recall": 0.07492029666900635, + "lift": 7.4920299049229895 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001338280737400055, + "recall": 0.2874601483345032, + "lift": 5.749202940480721 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0010674088262021542, + "recall": 0.4585547149181366, + "lift": 4.585547554258811 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0009740815185863561, + "2": 0.001005812161545827, + "4": 0.001055261976948163, + "8": 0.0010992879644106113 + }, + "0.2000": { + "1": 0.0014229145916550001, + "2": 0.0014700339354011271, + "4": 0.0015364414825030925, + "8": 0.0015827285477550967 + }, + "0.3000": { + "1": 0.0018098067083015534, + "2": 0.0018650612200171723, + "4": 0.0019665999384206323, + "8": 0.002042833409939604 + }, + "0.5000": { + "1": 0.0023055038912407344, + "2": 0.0024273937090060205, + "4": 0.0026763229300285074, + "8": 0.0029077273131987713 + }, + "0.7000": { + "1": 0.002861966206934352, + "2": 0.0030903014392590356, + "4": 0.0036201637468585846, + "8": 0.004202406612231591 + }, + "0.9000": { + "1": 0.0038396208986245517, + "2": 0.004425201788265554, + "4": 0.005877068610142366, + "8": 0.008076184699332467 + } + }, + "minutes": 0.33083393971125286 + }, + { + "epoch": 4, + "train_loss": 0.5014545876805161, + "train_main_loss": 0.4733086560772328, + "train_aux_spatial_loss": 0.056291862974635516, + "val_loss": 0.714298881093661, + "val_pr_auc": 0.0030682622199010843, + "val_auroc": 0.8761108875200969, + "val_brier": 0.2028871327638626, + "val_brier_skill_score": -870.7904281054932, + "val_ece": 0.3354083814166639, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0007264174734926152, + "val_positive_recall": 0.844845908607864, + "val_far": 0.9992735825265073, + "val_csi": 0.0007263205787176193, + "val_f1": 0.0014515868400415207, + "val_f2": 0.003619638382190104, + "val_frequency_bias": 1163.0308182784272, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1871, + "fp": 5911512, + "fn": 11, + "precision": 0.0003164009501836766, + "recall": 0.9941551540913921, + "far": 0.9996835990498163, + "csi": 0.00031640036161973984, + "f1": 0.0006326005681909433, + "f2": 0.0015799933489964635, + "frequency_bias": 3142.0738575982996, + "predicted_positive_rate": 0.7314017316017316 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1790, + "fp": 4373008, + "fn": 92, + "precision": 0.00040916174872531255, + "recall": 0.9511158342189161, + "far": 0.9995908382512747, + "csi": 0.0004091531444219169, + "f1": 0.0008179716131862507, + "f2": 0.0020422944345080667, + "frequency_bias": 2324.547290116897, + "predicted_positive_rate": 0.5411005565862709 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1725, + "fp": 3415319, + "fn": 157, + "precision": 0.00050482229669855, + "recall": 0.9165781083953242, + "far": 0.9994951777033014, + "csi": 0.0005047991031256283, + "f1": 0.0010090888191203904, + "f2": 0.002518562903627081, + "frequency_bias": 1815.6450584484592, + "predicted_positive_rate": 0.42263995052566483 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1590, + "fp": 2187234, + "fn": 292, + "precision": 0.0007264174734926152, + "recall": 0.844845908607864, + "far": 0.9992735825265073, + "csi": 0.0007263205787176193, + "f1": 0.0014515868400415207, + "f2": 0.003619638382190104, + "frequency_bias": 1163.0308182784272, + "predicted_positive_rate": 0.2707265306122449 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1441, + "fp": 1361655, + "fn": 441, + "precision": 0.0010571522475306214, + "recall": 0.7656748140276302, + "far": 0.9989428477524693, + "csi": 0.0010568103395800774, + "f1": 0.002111389341073629, + "f2": 0.005256729781471797, + "frequency_bias": 724.2805526036132, + "predicted_positive_rate": 0.168595670995671 + }, + "0.9000": { + "threshold": 0.9, + "tp": 1115, + "fp": 634236, + "fn": 767, + "precision": 0.001754935460871235, + "recall": 0.5924548352816154, + "far": 0.9982450645391288, + "csi": 0.0017528194454488003, + "f1": 0.003499504890675781, + "f2": 0.008671927376691415, + "frequency_bias": 337.59351753453774, + "predicted_positive_rate": 0.07858392084106369 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005293753929436207, + "recall": 0.22741764783859253, + "lift": 22.74176471853234 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002278293250128627, + "recall": 0.48937299847602844, + "lift": 9.787460797174546 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0015337043441832066, + "recall": 0.6588735580444336, + "lift": 6.588735292220149 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0006518448132221133, + "2": 0.0006635725065556874, + "4": 0.0006826997042340688, + "8": 0.0007084853296767113 + }, + "0.2000": { + "1": 0.0008589763023596486, + "2": 0.0008854152202262267, + "4": 0.0009275047159825744, + "8": 0.000974054012801262 + }, + "0.3000": { + "1": 0.0010670370878855806, + "2": 0.0011091460766166072, + "4": 0.0011764363698075764, + "8": 0.0012380013792546718 + }, + "0.5000": { + "1": 0.001577796273116161, + "2": 0.001657621748606397, + "4": 0.001773829668805127, + "8": 0.0018645216828079292 + }, + "0.7000": { + "1": 0.002322673564303268, + "2": 0.0024379540533610022, + "4": 0.002617086020650139, + "8": 0.0027739584877713996 + }, + "0.9000": { + "1": 0.004049673487098726, + "2": 0.004396817960682808, + "4": 0.004958663276014907, + "8": 0.0055254589787200326 + } + }, + "minutes": 0.32502801418304444 + }, + { + "epoch": 5, + "train_loss": 0.4840681991702775, + "train_main_loss": 0.4566405527103148, + "train_aux_spatial_loss": 0.05485529401597806, + "val_loss": 0.32642485077182454, + "val_pr_auc": 0.00269643573270509, + "val_auroc": 0.8832613542277497, + "val_brier": 0.0865185409784317, + "val_brier_skill_score": -370.7635261100412, + "val_ece": 0.17086548094873133, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0014307592400306989, + "val_positive_recall": 0.6567481402763018, + "val_far": 0.9985692407599693, + "val_csi": 0.001429690129701581, + "val_f1": 0.0028552980679380753, + "val_f2": 0.007091995111343175, + "val_frequency_bias": 459.02072263549417, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1748, + "fp": 3381937, + "fn": 134, + "precision": 0.0005165965508018625, + "recall": 0.9287991498405951, + "far": 0.9994834034491982, + "csi": 0.0005165760934612637, + "f1": 0.0010326187607570608, + "f2": 0.002577248907691732, + "frequency_bias": 1797.9197662061636, + "predicted_positive_rate": 0.4185139146567718 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1614, + "fp": 2195701, + "fn": 268, + "precision": 0.0007345328275645504, + "recall": 0.8575982996811902, + "far": 0.9992654671724355, + "csi": 0.0007344432496975085, + "f1": 0.0014678084773669664, + "f2": 0.003660124553086093, + "frequency_bias": 1167.5425079702445, + "predicted_positive_rate": 0.27177674706246135 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1505, + "fp": 1542529, + "fn": 377, + "precision": 0.0009747194686127378, + "recall": 0.7996811902231669, + "far": 0.9990252805313873, + "csi": 0.0009744815337368097, + "f1": 0.0019470656879157727, + "f2": 0.00484995121045759, + "frequency_bias": 820.4218916046759, + "predicted_positive_rate": 0.19097513914656772 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1236, + "fp": 862641, + "fn": 646, + "precision": 0.0014307592400306989, + "recall": 0.6567481402763018, + "far": 0.9985692407599693, + "csi": 0.001429690129701581, + "f1": 0.0028552980679380753, + "f2": 0.007091995111343175, + "frequency_bias": 459.02072263549417, + "predicted_positive_rate": 0.10684935064935065 + }, + "0.7000": { + "threshold": 0.7, + "tp": 974, + "fp": 505344, + "fn": 908, + "precision": 0.0019236922250443398, + "recall": 0.5175345377258236, + "far": 0.9980763077749557, + "csi": 0.0019202485676996053, + "f1": 0.0038331365604092884, + "f2": 0.009477547747768785, + "frequency_bias": 269.03188097768333, + "predicted_positive_rate": 0.06262436611008039 + }, + "0.9000": { + "threshold": 0.9, + "tp": 767, + "fp": 250939, + "fn": 1115, + "precision": 0.003047205867162483, + "recall": 0.4075451647183847, + "far": 0.9969527941328375, + "csi": 0.0030337669734713493, + "f1": 0.006049182137956055, + "f2": 0.014793584174915327, + "frequency_bias": 133.74388947927736, + "predicted_positive_rate": 0.031132467532467532 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004836116451770067, + "recall": 0.20775771141052246, + "lift": 20.775771591123345 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002166975988075137, + "recall": 0.46546226739883423, + "lift": 9.309246090469944 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014780457131564617, + "recall": 0.634962797164917, + "lift": 6.349627938867848 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010873373288891042, + "2": 0.001124730156876863, + "4": 0.0011842569881339582, + "8": 0.0012454325913199282 + }, + "0.2000": { + "1": 0.0015967200495251577, + "2": 0.001672856211459739, + "4": 0.00177710787777563, + "8": 0.0018681221664564873 + }, + "0.3000": { + "1": 0.0021362061921686992, + "2": 0.002243340057739074, + "4": 0.0024038793817035, + "8": 0.0025597638125007816 + }, + "0.5000": { + "1": 0.0032197830545794837, + "2": 0.0034434210744950233, + "4": 0.003829941863601527, + "8": 0.004228975808156621 + }, + "0.7000": { + "1": 0.004541360883536072, + "2": 0.004996111235622602, + "4": 0.005838071075877749, + "8": 0.0067937173415755225 + }, + "0.9000": { + "1": 0.0074909006005016154, + "2": 0.00836342260268863, + "4": 0.010405964121525169, + "8": 0.013064023833393557 + } + }, + "minutes": 0.3387693285942078 + }, + { + "epoch": 6, + "train_loss": 0.4704399423556344, + "train_main_loss": 0.4438283516374887, + "train_aux_spatial_loss": 0.05322318294850407, + "val_loss": 0.5198802714546521, + "val_pr_auc": 0.0020561227899118244, + "val_auroc": 0.8650460273834328, + "val_brier": 0.1523989737033844, + "val_brier_skill_score": -653.8466860258955, + "val_ece": 0.3041266358289659, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0009245181088592227, + "val_positive_recall": 0.793836344314559, + "val_far": 0.9990754818911408, + "val_csi": 0.0009242961831022078, + "val_f1": 0.0018468852971736105, + "val_f2": 0.004601156140572404, + "val_frequency_bias": 858.6487778958555, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1849, + "fp": 6336772, + "fn": 33, + "precision": 0.00029170382643164814, + "recall": 0.9824654622741764, + "far": 0.9997082961735684, + "csi": 0.00029170230777701386, + "f1": 0.0005832344847088631, + "f2": 0.0014567889912449265, + "frequency_bias": 3368.0239107332627, + "predicted_positive_rate": 0.7839976499690785 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1762, + "fp": 4527029, + "fn": 120, + "precision": 0.00038906630930859916, + "recall": 0.9362380446333688, + "far": 0.9996109336906914, + "csi": 0.0003890560004380744, + "f1": 0.0007778093894659802, + "f2": 0.0019421032780102105, + "frequency_bias": 2406.3714133900107, + "predicted_positive_rate": 0.5601473098330241 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1682, + "fp": 3194137, + "fn": 200, + "precision": 0.0005263126603853348, + "recall": 0.8937300743889479, + "far": 0.9994736873396146, + "csi": 0.0005262797248702214, + "f1": 0.0010520058004172372, + "f2": 0.0026253790176337436, + "frequency_bias": 1698.0972369819342, + "predicted_positive_rate": 0.3952775510204082 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1494, + "fp": 1614483, + "fn": 388, + "precision": 0.0009245181088592227, + "recall": 0.793836344314559, + "far": 0.9990754818911408, + "csi": 0.0009242961831022078, + "f1": 0.0018468852971736105, + "f2": 0.004601156140572404, + "frequency_bias": 858.6487778958555, + "predicted_positive_rate": 0.1998734693877551 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1227, + "fp": 770825, + "fn": 655, + "precision": 0.0015892711889872702, + "recall": 0.6519659936238045, + "far": 0.9984107288110128, + "csi": 0.0015879240125946834, + "f1": 0.0031708130150633, + "f2": 0.00786962210420996, + "frequency_bias": 410.2295430393199, + "predicted_positive_rate": 0.09549189857761287 + }, + "0.9000": { + "threshold": 0.9, + "tp": 733, + "fp": 284579, + "fn": 1149, + "precision": 0.0025691173171825932, + "recall": 0.3894792773645058, + "far": 0.9974308826828174, + "csi": 0.0025588125434177775, + "f1": 0.005104563465810568, + "f2": 0.012515366753175796, + "frequency_bias": 151.60042507970243, + "predicted_positive_rate": 0.03528905380333952 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0032776747830212116, + "recall": 0.14080765843391418, + "lift": 14.080765697258915 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002169449580833316, + "recall": 0.46599361300468445, + "lift": 9.31987255049544 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.00154483609367162, + "recall": 0.663655698299408, + "lift": 6.636556862913764 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0006032450292200453, + "2": 0.0006159225446106253, + "4": 0.0006359492894152385, + "8": 0.0006629478573787484 + }, + "0.2000": { + "1": 0.0008224392220558263, + "2": 0.0008506113892607914, + "4": 0.0008909354914231615, + "8": 0.0009322944740023376 + }, + "0.3000": { + "1": 0.001132169119496873, + "2": 0.0011820837106975013, + "4": 0.0012528655821539836, + "8": 0.0013113866777990646 + }, + "0.5000": { + "1": 0.0020500064266273377, + "2": 0.0021609626362443235, + "4": 0.002330766029421505, + "8": 0.002448917281466567 + }, + "0.7000": { + "1": 0.0035578224345581555, + "2": 0.0038334909485748803, + "4": 0.004359088724627869, + "8": 0.004831371146690966 + }, + "0.9000": { + "1": 0.00637013151022392, + "2": 0.007172450990244439, + "4": 0.009163883225476899, + "8": 0.011610480962539693 + } + }, + "minutes": 0.32045116027196247 + }, + { + "epoch": 7, + "train_loss": 0.45992261002683726, + "train_main_loss": 0.4340675990611428, + "train_aux_spatial_loss": 0.05171002073115929, + "val_loss": 0.3403168035050233, + "val_pr_auc": 0.0019248177157325489, + "val_auroc": 0.8516727976804799, + "val_brier": 0.09707184135913849, + "val_brier_skill_score": -416.11024737072506, + "val_ece": 0.20351086646269692, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0011260303225245256, + "val_positive_recall": 0.6259298618490967, + "val_far": 0.9988739696774754, + "val_csi": 0.0011252730793221996, + "val_f1": 0.002248016526165634, + "val_f2": 0.005589927122155567, + "val_frequency_bias": 555.8730074388948, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1753, + "fp": 4223621, + "fn": 129, + "precision": 0.00041487451761666543, + "recall": 0.9314558979808714, + "far": 0.9995851254823833, + "csi": 0.00041486185194993354, + "f1": 0.0008293796259322832, + "f2": 0.0020706834223896517, + "frequency_bias": 2245.1509032943677, + "predicted_positive_rate": 0.5226189239332096 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1653, + "fp": 2706333, + "fn": 229, + "precision": 0.0006104167451382688, + "recall": 0.8783209351753454, + "far": 0.9993895832548617, + "csi": 0.0006103651297995174, + "f1": 0.00121998562291595, + "f2": 0.003043622680641676, + "frequency_bias": 1438.8873538788523, + "predicted_positive_rate": 0.3349395176252319 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1499, + "fp": 1884843, + "fn": 383, + "precision": 0.0007946597170608511, + "recall": 0.7964930924548352, + "far": 0.9992053402829392, + "csi": 0.0007944984033179186, + "f1": 0.0015877353534326437, + "f2": 0.003957505002983309, + "frequency_bias": 1002.307120085016, + "predicted_positive_rate": 0.23331379097093383 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1178, + "fp": 1044975, + "fn": 704, + "precision": 0.0011260303225245256, + "recall": 0.6259298618490967, + "far": 0.9988739696774754, + "csi": 0.0011252730793221996, + "f1": 0.002248016526165634, + "f2": 0.005589927122155567, + "frequency_bias": 555.8730074388948, + "predicted_positive_rate": 0.1293943104514533 + }, + "0.7000": { + "threshold": 0.7, + "tp": 819, + "fp": 535799, + "fn": 1063, + "precision": 0.0015262253595667683, + "recall": 0.4351753453772582, + "far": 0.9984737746404332, + "csi": 0.0015232079987948244, + "f1": 0.0030417827298050144, + "f2": 0.007525553803574776, + "frequency_bias": 285.1317747077577, + "predicted_positive_rate": 0.06637204700061843 + }, + "0.9000": { + "threshold": 0.9, + "tp": 476, + "fp": 160341, + "fn": 1406, + "precision": 0.0029598860816953433, + "recall": 0.25292242295430395, + "far": 0.9970401139183046, + "csi": 0.002934232507104418, + "f1": 0.005851295951419493, + "f2": 0.01413763402536458, + "frequency_bias": 85.4500531349628, + "predicted_positive_rate": 0.01989078540507112 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0037847866769880056, + "recall": 0.16259299218654633, + "lift": 16.259299027724918 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0017884972039610147, + "recall": 0.38416579365730286, + "lift": 7.6833156875816755 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001277674688026309, + "recall": 0.5488841533660889, + "lift": 5.488841666845874 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0008760009616006759, + "2": 0.0009054218131856473, + "4": 0.0009476146471629754, + "8": 0.0009927278131620643 + }, + "0.2000": { + "1": 0.001305241106265398, + "2": 0.001353720436246486, + "4": 0.0014217692397042736, + "8": 0.0014697761316361246 + }, + "0.3000": { + "1": 0.0017153860355670902, + "2": 0.00177860122228235, + "4": 0.001884522312124659, + "8": 0.0019437440305635212 + }, + "0.5000": { + "1": 0.0024635323345161275, + "2": 0.0025797735042438763, + "4": 0.0028109999434535737, + "8": 0.002947214909635787 + }, + "0.7000": { + "1": 0.003567303280486267, + "2": 0.0038553882148504304, + "4": 0.004379270130838386, + "8": 0.004750678685008225 + }, + "0.9000": { + "1": 0.007761321421334344, + "2": 0.009021772361332503, + "4": 0.011815395230424186, + "8": 0.015149874578981093 + } + }, + "minutes": 0.32094937562942505 + }, + { + "epoch": 8, + "train_loss": 0.4501411836177224, + "train_main_loss": 0.42512462227910447, + "train_aux_spatial_loss": 0.0500331219766449, + "val_loss": 0.2779005537430445, + "val_pr_auc": 0.0033511312666555885, + "val_auroc": 0.8727357646440189, + "val_brier": 0.07455155998468399, + "val_brier_skill_score": -319.34232782334607, + "val_ece": 0.1583937377225246, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0015191347299669793, + "val_positive_recall": 0.6211477151965994, + "val_far": 0.998480865270033, + "val_csi": 0.0015177284707165392, + "val_f1": 0.0030308569235894785, + "val_f2": 0.0075220868804252005, + "val_frequency_bias": 408.8825717321998, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1718, + "fp": 3294835, + "fn": 164, + "precision": 0.0005211504259145842, + "recall": 0.9128586609989373, + "far": 0.9994788495740854, + "csi": 0.0005211245005258262, + "f1": 0.001041706142458469, + "f2": 0.0025998151982351524, + "frequency_bias": 1751.6222104144526, + "predicted_positive_rate": 0.4077369202226345 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1536, + "fp": 1918537, + "fn": 346, + "precision": 0.0007999695844897564, + "recall": 0.8161530286928799, + "far": 0.9992000304155102, + "csi": 0.0007998254547575295, + "f1": 0.0015983724905109641, + "f2": 0.00398422702623624, + "frequency_bias": 1020.230074388948, + "predicted_positive_rate": 0.23748583797155226 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1408, + "fp": 1333909, + "fn": 474, + "precision": 0.0010544312698782387, + "recall": 0.7481402763018066, + "far": 0.9989455687301217, + "csi": 0.0010540571092334055, + "f1": 0.002105894485413166, + "f2": 0.005242600597984131, + "frequency_bias": 709.5201912858661, + "predicted_positive_rate": 0.16515980210265924 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1169, + "fp": 768348, + "fn": 713, + "precision": 0.0015191347299669793, + "recall": 0.6211477151965994, + "far": 0.998480865270033, + "csi": 0.0015177284707165392, + "f1": 0.0030308569235894785, + "f2": 0.0075220868804252005, + "frequency_bias": 408.8825717321998, + "predicted_positive_rate": 0.09517835497835497 + }, + "0.7000": { + "threshold": 0.7, + "tp": 867, + "fp": 440719, + "fn": 1015, + "precision": 0.0019633774621478036, + "recall": 0.46068012752391074, + "far": 0.9980366225378522, + "csi": 0.001958874923463797, + "f1": 0.003910090468759866, + "f2": 0.009652337713809855, + "frequency_bias": 234.6365568544102, + "predicted_positive_rate": 0.054617934446505874 + }, + "0.9000": { + "threshold": 0.9, + "tp": 599, + "fp": 158162, + "fn": 1283, + "precision": 0.0037729669125288958, + "recall": 0.31827842720510097, + "far": 0.9962270330874711, + "csi": 0.0037427207517932567, + "f1": 0.007457530051107114, + "f2": 0.018010812501127557, + "frequency_bias": 84.35759829968119, + "predicted_positive_rate": 0.019636487322201607 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005343228112906218, + "recall": 0.2295430451631546, + "lift": 22.954303921357795 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002095238072797656, + "recall": 0.4500531256198883, + "lift": 9.001062746025704 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014743352076038718, + "recall": 0.633368730545044, + "lift": 6.333687748713827 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0011050920489504845, + "2": 0.0011474319184742177, + "4": 0.001207237050294463, + "8": 0.001263614219587872 + }, + "0.2000": { + "1": 0.0017407768081587616, + "2": 0.0018298937606811005, + "4": 0.001941458698715981, + "8": 0.0019852967362636864 + }, + "0.3000": { + "1": 0.00230278489686786, + "2": 0.0024155683935852545, + "4": 0.002561184010625994, + "8": 0.0025929800638113676 + }, + "0.5000": { + "1": 0.0034017699482070674, + "2": 0.003590432517582065, + "4": 0.0038659187657282823, + "8": 0.003947451170566385 + }, + "0.7000": { + "1": 0.004745738291732748, + "2": 0.005164124226629263, + "4": 0.005773230761917714, + "8": 0.006170780849613977 + }, + "0.9000": { + "1": 0.009755112945959765, + "2": 0.011279501877038078, + "4": 0.014849528591105998, + "8": 0.020210872188599516 + } + }, + "minutes": 0.3122890591621399 + }, + { + "epoch": 9, + "train_loss": 0.4357784079228708, + "train_main_loss": 0.41138888008416336, + "train_aux_spatial_loss": 0.04877905696682909, + "val_loss": 0.2466988556087017, + "val_pr_auc": 0.0024715249735166854, + "val_auroc": 0.8593465923841782, + "val_brier": 0.06473856419324875, + "val_brier_skill_score": -277.17663852865024, + "val_ece": 0.14665763289695394, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0013977547551646229, + "val_positive_recall": 0.5042507970244421, + "val_far": 0.9986022452448354, + "val_csi": 0.001395836612103036, + "val_f1": 0.0027877819361130864, + "val_f2": 0.0069121335986504955, + "val_frequency_bias": 360.7577045696068, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1671, + "fp": 3056649, + "fn": 211, + "precision": 0.0005463784038295535, + "recall": 0.88788522848034, + "far": 0.9994536215961705, + "csi": 0.0005463407106221908, + "f1": 0.0010920847708746023, + "f2": 0.002725184027388181, + "frequency_bias": 1625.0371944739638, + "predicted_positive_rate": 0.3782708719851577 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1469, + "fp": 1790064, + "fn": 413, + "precision": 0.000819968150181995, + "recall": 0.7805526036131775, + "far": 0.999180031849818, + "csi": 0.0008197791674525906, + "f1": 0.001638215360081186, + "f2": 0.004082685356416487, + "frequency_bias": 951.9303931987248, + "predicted_positive_rate": 0.22158726035868892 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1278, + "fp": 1237670, + "fn": 604, + "precision": 0.0010315202897942448, + "recall": 0.6790648246546227, + "far": 0.9989684797102057, + "csi": 0.0010310176579925651, + "f1": 0.0020599115108435482, + "f2": 0.005126452494873548, + "frequency_bias": 658.3145589798087, + "predicted_positive_rate": 0.1532403215831787 + }, + "0.5000": { + "threshold": 0.5, + "tp": 949, + "fp": 677997, + "fn": 933, + "precision": 0.0013977547551646229, + "recall": 0.5042507970244421, + "far": 0.9986022452448354, + "csi": 0.001395836612103036, + "f1": 0.0027877819361130864, + "f2": 0.0069121335986504955, + "frequency_bias": 360.7577045696068, + "predicted_positive_rate": 0.08397600494743351 + }, + "0.7000": { + "threshold": 0.7, + "tp": 780, + "fp": 337462, + "fn": 1102, + "precision": 0.002306041236747654, + "recall": 0.41445270988310307, + "far": 0.9976939587632524, + "csi": 0.0022985525012966194, + "f1": 0.004586562547776693, + "f2": 0.011279174017410416, + "frequency_bias": 179.72476089266738, + "predicted_positive_rate": 0.04183574520717378 + }, + "0.9000": { + "threshold": 0.9, + "tp": 575, + "fp": 126493, + "fn": 1307, + "precision": 0.004525136147574527, + "recall": 0.3055260361317747, + "far": 0.9954748638524255, + "csi": 0.0044790652385589096, + "f1": 0.008918185343156262, + "f2": 0.021360218728639784, + "frequency_bias": 67.51753453772582, + "predicted_positive_rate": 0.015716512059369202 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.005256648175418377, + "recall": 0.22582359611988068, + "lift": 22.582359816297473 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002011131728067994, + "recall": 0.4319872558116913, + "lift": 8.63974510099088 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012838590191677213, + "recall": 0.5515409111976624, + "lift": 5.515409317256945 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0011651445757413503, + "2": 0.0012111852278434698, + "4": 0.0012755769277128515, + "8": 0.001326139591474207 + }, + "0.2000": { + "1": 0.0017847015451787351, + "2": 0.0018660988212818586, + "4": 0.001969892674721363, + "8": 0.0020256766859216846 + }, + "0.3000": { + "1": 0.0022836423061386446, + "2": 0.0024096079587074826, + "4": 0.0025866385088719746, + "8": 0.002677494067034991 + }, + "0.5000": { + "1": 0.003257597509393695, + "2": 0.0035462700407361547, + "4": 0.003995590535502691, + "8": 0.0042497358267622465 + }, + "0.7000": { + "1": 0.0056228500540735915, + "2": 0.006273475118964256, + "4": 0.00754737401459038, + "8": 0.008762055377144917 + }, + "0.9000": { + "1": 0.01119749514237256, + "2": 0.012904909248986685, + "4": 0.017159283576349882, + "8": 0.023161111656246902 + } + }, + "minutes": 0.3215821862220764 + }, + { + "epoch": 10, + "train_loss": 0.43129921461825527, + "train_main_loss": 0.4073479366069134, + "train_aux_spatial_loss": 0.04790255693317092, + "val_loss": 0.31038872922460237, + "val_pr_auc": 0.003182743786360499, + "val_auroc": 0.8601447316055663, + "val_brier": 0.08384887129068375, + "val_brier_skill_score": -359.29216048780205, + "val_ece": 0.17778525591728558, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0013001670051705871, + "val_positive_recall": 0.6200850159404888, + "val_far": 0.9986998329948295, + "val_csi": 0.0012991321307548103, + "val_f1": 0.002594893152439411, + "val_f2": 0.006446765844846732, + "val_frequency_bias": 476.9272051009564, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1708, + "fp": 3565572, + "fn": 174, + "precision": 0.00047879616963064295, + "recall": 0.9075451647183846, + "far": 0.9995212038303694, + "csi": 0.00047877281669224045, + "f1": 0.0009570874059513128, + "f2": 0.00238893948989708, + "frequency_bias": 1895.4729011689692, + "predicted_positive_rate": 0.44122201607915895 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1545, + "fp": 2195984, + "fn": 337, + "precision": 0.0007030623941709075, + "recall": 0.8209351753453773, + "far": 0.999296937605829, + "csi": 0.0007029545932281585, + "f1": 0.0014049215903712403, + "f2": 0.0035033107987684675, + "frequency_bias": 1167.6562167906482, + "predicted_positive_rate": 0.27180321583178724 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1434, + "fp": 1603689, + "fn": 448, + "precision": 0.000893389478563325, + "recall": 0.7619553666312433, + "far": 0.9991066105214367, + "csi": 0.000893140197474917, + "f1": 0.001784686419768451, + "f2": 0.004446095280379946, + "frequency_bias": 852.8815090329437, + "predicted_positive_rate": 0.19853098330241187 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1167, + "fp": 896410, + "fn": 715, + "precision": 0.0013001670051705871, + "recall": 0.6200850159404888, + "far": 0.9986998329948295, + "csi": 0.0012991321307548103, + "f1": 0.002594893152439411, + "f2": 0.006446765844846732, + "frequency_bias": 476.9272051009564, + "predicted_positive_rate": 0.11101756338899196 + }, + "0.7000": { + "threshold": 0.7, + "tp": 924, + "fp": 456494, + "fn": 958, + "precision": 0.0020200341919207375, + "recall": 0.4909670563230606, + "far": 0.9979799658080792, + "csi": 0.002015812346196136, + "f1": 0.0040235140431090785, + "f2": 0.009936637803099713, + "frequency_bias": 243.0488841657811, + "predicted_positive_rate": 0.05657612863327149 + }, + "0.9000": { + "threshold": 0.9, + "tp": 734, + "fp": 175677, + "fn": 1148, + "precision": 0.004160738275957848, + "recall": 0.3900106269925611, + "far": 0.9958392617240421, + "csi": 0.004133837203408444, + "f1": 0.008233637888195275, + "f2": 0.0199522667840969, + "frequency_bias": 93.73591923485654, + "predicted_positive_rate": 0.021819542362399504 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.006431663408875465, + "recall": 0.2763018012046814, + "lift": 27.630180386759534 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002209029160439968, + "recall": 0.4744952321052551, + "lift": 9.489904912987356 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0013927025720477104, + "recall": 0.5982996821403503, + "lift": 5.9829970638202745 + } + }, + "val_fss": { + "0.1000": { + "1": 0.001019473269986193, + "2": 0.0010557422939804217, + "4": 0.0011052621270040142, + "8": 0.0011453423660473527 + }, + "0.2000": { + "1": 0.0015099904182619417, + "2": 0.0015686609647825867, + "4": 0.0016378638383718336, + "8": 0.0016768349605363886 + }, + "0.3000": { + "1": 0.0019186672475260957, + "2": 0.0019898217973708388, + "4": 0.002076219424972159, + "8": 0.0021161819887003386 + }, + "0.5000": { + "1": 0.002904085129755374, + "2": 0.0030756505922767685, + "4": 0.0033142291130332513, + "8": 0.003457418818581859 + }, + "0.7000": { + "1": 0.004749695098450801, + "2": 0.005118148709556514, + "4": 0.005725380054942164, + "8": 0.006258756516320685 + }, + "0.9000": { + "1": 0.010015907694898685, + "2": 0.011072008375778264, + "4": 0.013502674103421874, + "8": 0.017073185913380362 + } + }, + "minutes": 0.31329720417658485 + } +] diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/run_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/run_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..5b3e2fd02cb4fc1889b50952fb25ac63d69605b4 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/run_summary.json @@ -0,0 +1,172 @@ +{ + "run_name": "reference_fireprone_seed_7_20260504", + "device": "cuda", + "index_root": "", + "input_channels": 16, + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "train_positive_rate": 0.0018423063389998895, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight": 50.0, + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "aux_positive_rate": 0.004560296846011132, + "init_checkpoint": "", + "best_val_pr_auc": 0.0033511312666555885, + "best_val_auroc": 0.8727357646440189, + "best_test_pr_auc": 0.0016046388753321321, + "best_test_auroc": 0.8223907600143328, + "best_test_brier": 0.09998881816864014, + "best_test_ece": 0.19384167839217642, + "best_test_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.003426097799092531, + "recall": 0.1412544548511505, + "lift": 14.125446318034083 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0017266542417928576, + "recall": 0.3559408485889435, + "lift": 7.11881657572967 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0011168831260874867, + "recall": 0.46047934889793396, + "lift": 4.60479343153765 + } + }, + "best_test_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1768, + "fp": 3684053, + "fn": 193, + "precision": 0.000479676034186142, + "recall": 0.901580826109128, + "far": 0.9995203239658138, + "csi": 0.000479650918309046, + "f1": 0.0009588419272071939, + "f2": 0.0023932868844359196, + "frequency_bias": 1879.5619581845997, + "predicted_positive_rate": 0.4558838589981447 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1571, + "fp": 2394479, + "fn": 390, + "precision": 0.0006556624444398072, + "recall": 0.8011218765935747, + "far": 0.9993443375555602, + "csi": 0.0006555557410158402, + "f1": 0.0013102525384579137, + "f2": 0.0032676149613918085, + "frequency_bias": 1221.8510963793983, + "predicted_positive_rate": 0.2963574520717378 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1390, + "fp": 1792642, + "fn": 571, + "precision": 0.0007747910851088498, + "recall": 0.7088220295767466, + "far": 0.9992252089148912, + "csi": 0.0007745445650096428, + "f1": 0.0015478902200621052, + "f2": 0.0038570911649858258, + "frequency_bias": 914.8556858745538, + "predicted_positive_rate": 0.22189635126777985 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1063, + "fp": 1120631, + "fn": 898, + "precision": 0.0009476737862554315, + "recall": 0.5420703722590515, + "far": 0.9990523262137446, + "csi": 0.0009469157093583421, + "f1": 0.001892039816491717, + "f2": 0.0047054636497399814, + "frequency_bias": 572.0010198878124, + "predicted_positive_rate": 0.13873766233766233 + }, + "0.7000": { + "threshold": 0.7, + "tp": 807, + "fp": 603572, + "fn": 1154, + "precision": 0.0013352548649109251, + "recall": 0.41152473227944925, + "far": 0.9986647451350891, + "csi": 0.0013327101908566503, + "f1": 0.002661872876603886, + "f2": 0.006590735728647894, + "frequency_bias": 308.1993880673126, + "predicted_positive_rate": 0.07475312306740878 + }, + "0.9000": { + "threshold": 0.9, + "tp": 547, + "fp": 202779, + "fn": 1414, + "precision": 0.0026902609602313526, + "recall": 0.27893931667516575, + "far": 0.9973097390397686, + "csi": 0.0026716811565888443, + "f1": 0.005329124591425663, + "f2": 0.01295165032911872, + "frequency_bias": 103.68485466598675, + "predicted_positive_rate": 0.025148546691403834 + } + }, + "best_test_fss": { + "0.1000": { + "1": 0.001006653180254391, + "2": 0.001036172017983894, + "4": 0.0010764843206666486, + "8": 0.0011088189513154267 + }, + "0.2000": { + "1": 0.0013931809574536658, + "2": 0.0014436935674144724, + "4": 0.0015037854705144404, + "8": 0.0015046820466139144 + }, + "0.3000": { + "1": 0.0016631737390239154, + "2": 0.0017265932312411358, + "4": 0.0017994774155269955, + "8": 0.0017754472066875415 + }, + "0.5000": { + "1": 0.00209939111569335, + "2": 0.002211811539862274, + "4": 0.0023188574757858627, + "8": 0.0022717093175803615 + }, + "0.7000": { + "1": 0.0031227648502994976, + "2": 0.0033730806856239193, + "4": 0.003653859775083146, + "8": 0.003690592345308774 + }, + "0.9000": { + "1": 0.0066600329138472025, + "2": 0.0075581235730202145, + "4": 0.009383123791337655, + "8": 0.011475141202696215 + } + }, + "epochs": 10 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/tile_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/tile_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..7ef825f81c3bcef8a18cffe05b9d41285027d3b1 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_7_20260504/tile_summary.json @@ -0,0 +1,16 @@ +{ + "index_root": "", + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "input_channels": 16, + "train_positive_rate": 0.0018423063389998895, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight_source": "tiles", + "positive_rate_source": "tiles", + "pos_weight": 50.0, + "loss_type": "bce", + "train_target_mode": "dilate_max", + "train_target_radius": 2 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/history.json b/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/history.json new file mode 100644 index 0000000000000000000000000000000000000000..5a495399872b53c3de3f81fcacef0a49636ca48d --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/history.json @@ -0,0 +1,1682 @@ +[ + { + "epoch": 1, + "train_loss": 0.5778533777167002, + "train_main_loss": 0.5341317258517807, + "train_aux_spatial_loss": 0.08744330333986021, + "val_loss": 0.5268240878979366, + "val_pr_auc": 0.001801559151297299, + "val_auroc": 0.8300020902264806, + "val_brier": 0.16021513938903809, + "val_brier_skill_score": -687.4321497091457, + "val_ece": 0.3149934093473418, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0007911901010445063, + "val_positive_recall": 0.7454835281615303, + "val_far": 0.9992088098989554, + "val_csi": 0.0007909764415306043, + "val_f1": 0.0015807025845557582, + "val_f2": 0.003939227518325972, + "val_frequency_bias": 942.230605738576, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1861, + "fp": 6674626, + "fn": 21, + "precision": 0.00027873940292252495, + "recall": 0.9888416578108395, + "far": 0.9997212605970774, + "csi": 0.0002787385261876418, + "f1": 0.0005573217053445235, + "f2": 0.0013921273366382329, + "frequency_bias": 3547.548884165781, + "predicted_positive_rate": 0.825786889301175 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1722, + "fp": 4433958, + "fn": 160, + "precision": 0.0003882155610864625, + "recall": 0.9149840595111584, + "far": 0.9996117844389135, + "csi": 0.0003882015582167075, + "f1": 0.0007761018324927066, + "f2": 0.0019377890929256523, + "frequency_bias": 2356.8969181721573, + "predicted_positive_rate": 0.5486307977736549 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1631, + "fp": 3209108, + "fn": 251, + "precision": 0.0005079827416678839, + "recall": 0.8666312433581297, + "far": 0.9994920172583321, + "csi": 0.0005079430331455407, + "f1": 0.001015370316013, + "f2": 0.0025339724764912295, + "frequency_bias": 1706.0249734325187, + "predicted_positive_rate": 0.3971229437229437 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1403, + "fp": 1771875, + "fn": 479, + "precision": 0.0007911901010445063, + "recall": 0.7454835281615303, + "far": 0.9992088098989554, + "csi": 0.0007909764415306043, + "f1": 0.0015807025845557582, + "f2": 0.003939227518325972, + "frequency_bias": 942.230605738576, + "predicted_positive_rate": 0.21932937538651825 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1007, + "fp": 873767, + "fn": 875, + "precision": 0.0011511544696115796, + "recall": 0.5350690754516472, + "far": 0.9988488455303884, + "csi": 0.0011500041683368564, + "f1": 0.0022973663557883596, + "f2": 0.0057066627979988705, + "frequency_bias": 464.81083953241233, + "predicted_positive_rate": 0.10819715522572665 + }, + "0.9000": { + "threshold": 0.9, + "tp": 585, + "fp": 243959, + "fn": 1297, + "precision": 0.002392207537293902, + "recall": 0.3108395324123273, + "far": 0.9976077924627061, + "csi": 0.002379586806106386, + "f1": 0.004747875630006573, + "f2": 0.011603827477863468, + "frequency_bias": 129.9383634431456, + "predicted_positive_rate": 0.030246629560915277 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0034137291368097067, + "recall": 0.146652489900589, + "lift": 14.665250005376244 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.00172912806738168, + "recall": 0.37141337990760803, + "lift": 7.4282681440753535 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0011861472157761455, + "recall": 0.5095642805099487, + "lift": 5.095643141387231 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0005767564103924538, + "2": 0.0005890134794819257, + "4": 0.0006078227289899596, + "8": 0.0006326282137129047 + }, + "0.2000": { + "1": 0.00081986363926001, + "2": 0.0008498172851243435, + "4": 0.0008980338142677846, + "8": 0.0009442365839407163 + }, + "0.3000": { + "1": 0.001086040351521933, + "2": 0.0011281453218618953, + "4": 0.0011947800946939813, + "8": 0.0012519072968685263 + }, + "0.5000": { + "1": 0.001722969678203623, + "2": 0.0017971917246983882, + "4": 0.001911957912350437, + "8": 0.0020062945625652917 + }, + "0.7000": { + "1": 0.002627765306140284, + "2": 0.0028429364479983787, + "4": 0.0031580131392605626, + "8": 0.003416834699769833 + }, + "0.9000": { + "1": 0.005918451186751605, + "2": 0.006697606371421805, + "4": 0.00873029919299595, + "8": 0.01141916176600577 + } + }, + "minutes": 0.4064649740854899 + }, + { + "epoch": 2, + "train_loss": 0.5286303263821187, + "train_main_loss": 0.49828399360398123, + "train_aux_spatial_loss": 0.06069266609487724, + "val_loss": 0.2564812108874321, + "val_pr_auc": 0.0016945588134657274, + "val_auroc": 0.8576418217604199, + "val_brier": 0.06716381013393402, + "val_brier_skill_score": -287.597734081082, + "val_ece": 0.15346273963817214, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0013487785097515384, + "val_positive_recall": 0.4957492029755579, + "val_far": 0.9986512214902484, + "val_csi": 0.0013469306438992558, + "val_f1": 0.0026902377241684555, + "val_f2": 0.006671290569383565, + "val_frequency_bias": 367.55419766206165, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1706, + "fp": 3331373, + "fn": 176, + "precision": 0.0005118390533197683, + "recall": 0.9064824654622742, + "far": 0.9994881609466802, + "csi": 0.0005118120275826482, + "f1": 0.0010231004200648825, + "f2": 0.0025534281644024577, + "frequency_bias": 1771.0302869287991, + "predicted_positive_rate": 0.41225466914038345 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1510, + "fp": 1809417, + "fn": 372, + "precision": 0.0008338270951838479, + "recall": 0.8023379383634431, + "far": 0.9991661729048161, + "csi": 0.0008336558458873991, + "f1": 0.001665922885422568, + "f2": 0.004151876180603864, + "frequency_bias": 962.2353878852285, + "predicted_positive_rate": 0.22398602350030922 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1315, + "fp": 1226566, + "fn": 567, + "precision": 0.0010709506865893357, + "recall": 0.6987247608926673, + "far": 0.9989290493134106, + "csi": 0.0010704563807340645, + "f1": 0.0021386234583411603, + "f2": 0.005322124090078671, + "frequency_bias": 652.4341126461211, + "predicted_positive_rate": 0.15187149041434755 + }, + "0.5000": { + "threshold": 0.5, + "tp": 933, + "fp": 690804, + "fn": 949, + "precision": 0.0013487785097515384, + "recall": 0.4957492029755579, + "far": 0.9986512214902484, + "csi": 0.0013469306438992558, + "f1": 0.0026902377241684555, + "f2": 0.006671290569383565, + "frequency_bias": 367.55419766206165, + "predicted_positive_rate": 0.08555807050092765 + }, + "0.7000": { + "threshold": 0.7, + "tp": 701, + "fp": 351788, + "fn": 1181, + "precision": 0.0019887145414466834, + "recall": 0.37247608926673753, + "far": 0.9980112854585533, + "csi": 0.001982073684508157, + "f1": 0.003956305679640828, + "f2": 0.009735651372018544, + "frequency_bias": 187.29489904357067, + "predicted_positive_rate": 0.043597897340754484 + }, + "0.9000": { + "threshold": 0.9, + "tp": 356, + "fp": 142050, + "fn": 1526, + "precision": 0.0024998946673595213, + "recall": 0.1891604675876727, + "far": 0.9975001053326404, + "csi": 0.0024733902120445766, + "f1": 0.004934575293856731, + "f2": 0.01187189029839796, + "frequency_bias": 75.6673751328374, + "predicted_positive_rate": 0.017613605442176872 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.002770562656223774, + "recall": 0.11902231723070145, + "lift": 11.90223136656136 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001855287584476173, + "recall": 0.39851221442222595, + "lift": 7.970244611627591 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012615955201908946, + "recall": 0.5419766306877136, + "lift": 5.4197661758697135 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010905469804065593, + "2": 0.001132628631186794, + "4": 0.0012021196677818313, + "8": 0.0012747952224614334 + }, + "0.2000": { + "1": 0.0018261282348317476, + "2": 0.0019093635460114555, + "4": 0.0020469217446071886, + "8": 0.0021685131925710532 + }, + "0.3000": { + "1": 0.002352647456016599, + "2": 0.0024661544218815834, + "4": 0.0026671970273386414, + "8": 0.0028544633354051774 + }, + "0.5000": { + "1": 0.0031238784505325246, + "2": 0.0033962414112024275, + "4": 0.0039018832969852646, + "8": 0.004433729474684234 + }, + "0.7000": { + "1": 0.004722874230304819, + "2": 0.005319378699667143, + "4": 0.006670035900161353, + "8": 0.008381867678164179 + }, + "0.9000": { + "1": 0.00686789744070182, + "2": 0.008074405364309811, + "4": 0.011220289911567538, + "8": 0.016138933098775343 + } + }, + "minutes": 0.3335001548131307 + }, + { + "epoch": 3, + "train_loss": 0.5119181981534846, + "train_main_loss": 0.48301050484862534, + "train_aux_spatial_loss": 0.05781538778911272, + "val_loss": 0.24042269537846248, + "val_pr_auc": 0.0024715155000571394, + "val_auroc": 0.8559554621686548, + "val_brier": 0.06131188943982124, + "val_brier_skill_score": -262.4524802758656, + "val_ece": 0.15086109867404504, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0017179984871622876, + "val_positive_recall": 0.5116896918172157, + "val_far": 0.9982820015128377, + "val_csi": 0.0017151864352441424, + "val_f1": 0.003424499215885694, + "val_f2": 0.008476157616043265, + "val_frequency_bias": 297.8405951115834, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1692, + "fp": 3465302, + "fn": 190, + "precision": 0.0004880308417032161, + "recall": 0.8990435706695006, + "far": 0.9995119691582968, + "csi": 0.00048800409785001314, + "f1": 0.0009755321320220153, + "f2": 0.0024348672997321647, + "frequency_bias": 1842.1859723698194, + "predicted_positive_rate": 0.42881805813234386 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1457, + "fp": 2011263, + "fn": 425, + "precision": 0.0007238960213045033, + "recall": 0.7741764080765143, + "far": 0.9992761039786955, + "csi": 0.0007237431978322475, + "f1": 0.0014464395448828104, + "f2": 0.0036059929276009675, + "frequency_bias": 1069.4580233793836, + "predicted_positive_rate": 0.24894495980210266 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1286, + "fp": 1194152, + "fn": 596, + "precision": 0.0010757563336618043, + "recall": 0.6833156216790648, + "far": 0.9989242436663381, + "csi": 0.001075220269657886, + "f1": 0.002148130825510306, + "f2": 0.0053451219735221114, + "frequency_bias": 635.1955366631244, + "predicted_positive_rate": 0.14785875077303648 + }, + "0.5000": { + "threshold": 0.5, + "tp": 963, + "fp": 559573, + "fn": 919, + "precision": 0.0017179984871622876, + "recall": 0.5116896918172157, + "far": 0.9982820015128377, + "csi": 0.0017151864352441424, + "f1": 0.003424499215885694, + "f2": 0.008476157616043265, + "frequency_bias": 297.8405951115834, + "predicted_positive_rate": 0.06933036487322201 + }, + "0.7000": { + "threshold": 0.7, + "tp": 755, + "fp": 299786, + "fn": 1127, + "precision": 0.002512136447273417, + "recall": 0.40116896918172157, + "far": 0.9974878635527266, + "csi": 0.0025027513690547222, + "f1": 0.0049930064842951755, + "f2": 0.012253748348584247, + "frequency_bias": 159.692348565356, + "predicted_positive_rate": 0.03717266542980829 + }, + "0.9000": { + "threshold": 0.9, + "tp": 487, + "fp": 107029, + "fn": 1395, + "precision": 0.004529558391309201, + "recall": 0.2587672688629118, + "far": 0.9954704416086908, + "csi": 0.004471540983004472, + "f1": 0.008903270626519681, + "f2": 0.021165814818678075, + "frequency_bias": 57.12858660998937, + "predicted_positive_rate": 0.013298206555349412 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004947433713823557, + "recall": 0.21253985166549683, + "lift": 21.253986297827947 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0020754483994096518, + "recall": 0.44580233097076416, + "lift": 8.916047064895523 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001385281328111887, + "recall": 0.595111608505249, + "lift": 5.95111568328068 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0010285527774587244, + "2": 0.0010653427122698789, + "4": 0.001122755300464795, + "8": 0.0011787351272731783 + }, + "0.2000": { + "1": 0.0015860647314260845, + "2": 0.0016723213614568966, + "4": 0.0017956498996868353, + "8": 0.0018829515791326523 + }, + "0.3000": { + "1": 0.0023788924877019557, + "2": 0.0025265549899461393, + "4": 0.002761744000741362, + "8": 0.002927857083833363 + }, + "0.5000": { + "1": 0.003953657459880633, + "2": 0.004350779441774422, + "4": 0.005033153433681603, + "8": 0.0055588513866320355 + }, + "0.7000": { + "1": 0.006268758494930693, + "2": 0.007051153593185755, + "4": 0.008535853615263655, + "8": 0.01020087247019108 + }, + "0.9000": { + "1": 0.012090833894108921, + "2": 0.013963613985153223, + "4": 0.018853160860571982, + "8": 0.02710607924875852 + } + }, + "minutes": 0.3254421591758728 + }, + { + "epoch": 4, + "train_loss": 0.4973915634129131, + "train_main_loss": 0.46941266194494213, + "train_aux_spatial_loss": 0.0559578032103662, + "val_loss": 0.2442722608645757, + "val_pr_auc": 0.0015287010547867983, + "val_auroc": 0.861693387637483, + "val_brier": 0.06107700988650322, + "val_brier_skill_score": -261.4432208736014, + "val_ece": 0.14442989307660722, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0016247604629118164, + "val_positive_recall": 0.5063761955366631, + "val_far": 0.9983752395370882, + "val_csi": 0.0016221911666329065, + "val_f1": 0.0032391278486820865, + "val_f2": 0.008020859283997334, + "val_frequency_bias": 311.66206163655687, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1686, + "fp": 3178518, + "fn": 196, + "precision": 0.0005301546693231, + "recall": 0.895855472901169, + "far": 0.999469845330677, + "csi": 0.0005301219972330525, + "f1": 0.0010596822336039944, + "f2": 0.002644513403259747, + "frequency_bias": 1689.8002125398511, + "predicted_positive_rate": 0.3933461966604824 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1503, + "fp": 1763618, + "fn": 379, + "precision": 0.0008514996988874984, + "recall": 0.7986184909670563, + "far": 0.9991485003011125, + "csi": 0.0008513169073916737, + "f1": 0.0017011855667477644, + "f2": 0.004239417955838973, + "frequency_bias": 937.8963868225293, + "predicted_positive_rate": 0.2183204700061843 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1346, + "fp": 1144434, + "fn": 536, + "precision": 0.0011747455881582851, + "recall": 0.7151965993623804, + "far": 0.9988252544118417, + "csi": 0.0011741962949134445, + "f1": 0.00234563834996715, + "f2": 0.005835388291765947, + "frequency_bias": 608.8097768331562, + "predicted_positive_rate": 0.14171675943104514 + }, + "0.5000": { + "threshold": 0.5, + "tp": 953, + "fp": 585595, + "fn": 929, + "precision": 0.0016247604629118164, + "recall": 0.5063761955366631, + "far": 0.9983752395370882, + "csi": 0.0016221911666329065, + "f1": 0.0032391278486820865, + "f2": 0.008020859283997334, + "frequency_bias": 311.66206163655687, + "predicted_positive_rate": 0.07254768089053804 + }, + "0.7000": { + "threshold": 0.7, + "tp": 640, + "fp": 321536, + "fn": 1242, + "precision": 0.001986491855383393, + "recall": 0.34006376195536664, + "far": 0.9980135081446166, + "csi": 0.0019788632667322166, + "f1": 0.003949910201260268, + "f2": 0.009705675393686459, + "frequency_bias": 171.18809776833157, + "predicted_positive_rate": 0.03984860853432282 + }, + "0.9000": { + "threshold": 0.9, + "tp": 318, + "fp": 123828, + "fn": 1564, + "precision": 0.0025615001691556716, + "recall": 0.1689691817215728, + "far": 0.9974384998308443, + "csi": 0.0025296316919894996, + "f1": 0.0050464976037071126, + "f2": 0.012075276820025214, + "frequency_bias": 65.96493092454836, + "predicted_positive_rate": 0.015355102040816327 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0020408162381500006, + "recall": 0.08767268806695938, + "lift": 8.767268622686151 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0018330240854993463, + "recall": 0.3937300741672516, + "lift": 7.874601470240361 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014223870821297169, + "recall": 0.6110520958900452, + "lift": 6.110520585515546 + } + }, + "val_fss": { + "0.1000": { + "1": 0.001124098184984046, + "2": 0.001166554522149088, + "4": 0.0012322749538367095, + "8": 0.0012949795537359199 + }, + "0.2000": { + "1": 0.0018536996365665637, + "2": 0.0019364847225513682, + "4": 0.0020742894927048283, + "8": 0.0021806406439595083 + }, + "0.3000": { + "1": 0.002565767874730618, + "2": 0.0026879762763847292, + "4": 0.002921815146029516, + "8": 0.0031007743459819936 + }, + "0.5000": { + "1": 0.003750084728529779, + "2": 0.004078126887792521, + "4": 0.004664788826923449, + "8": 0.0051470259043820965 + }, + "0.7000": { + "1": 0.004897024677909645, + "2": 0.005528912754412585, + "4": 0.006720608815280804, + "8": 0.007938644694797992 + }, + "0.9000": { + "1": 0.006531676310613244, + "2": 0.0076338310683981, + "4": 0.010603661046574286, + "8": 0.01568061290465117 + } + }, + "minutes": 0.33346703052520754 + }, + { + "epoch": 5, + "train_loss": 0.4875196878293424, + "train_main_loss": 0.4603805759908716, + "train_aux_spatial_loss": 0.05427822470582733, + "val_loss": 0.29524567822615305, + "val_pr_auc": 0.0014329584864629465, + "val_auroc": 0.8570874266988384, + "val_brier": 0.0725969523191452, + "val_brier_skill_score": -310.9435287950134, + "val_ece": 0.1495643223012075, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0013688255199105052, + "val_positive_recall": 0.5233793836344315, + "val_far": 0.9986311744800895, + "val_csi": 0.0013671213559623145, + "val_f1": 0.0027305097737003402, + "val_f2": 0.00677326944684737, + "val_frequency_bias": 382.3565356004251, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1662, + "fp": 2908424, + "fn": 220, + "precision": 0.0005711171422425317, + "recall": 0.8831030818278427, + "far": 0.9994288828577574, + "csi": 0.0005710739695413472, + "f1": 0.001141496060396268, + "f2": 0.0028482177560157033, + "frequency_bias": 1546.2731137088203, + "predicted_positive_rate": 0.3599364254792826 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1500, + "fp": 1678289, + "fn": 382, + "precision": 0.000892969295548429, + "recall": 0.7970244420828906, + "far": 0.9991070307044516, + "csi": 0.0008927662720044567, + "f1": 0.00178393990263256, + "f2": 0.004444926472026299, + "frequency_bias": 892.5552603613178, + "predicted_positive_rate": 0.2077661100803958 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1327, + "fp": 1182582, + "fn": 555, + "precision": 0.0011208631744500633, + "recall": 0.7051009564293305, + "far": 0.99887913682555, + "csi": 0.001120337975658188, + "f1": 0.002238168446210167, + "f2": 0.005568905447791197, + "frequency_bias": 629.0696068012752, + "predicted_positive_rate": 0.14643277674706245 + }, + "0.5000": { + "threshold": 0.5, + "tp": 985, + "fp": 718610, + "fn": 897, + "precision": 0.0013688255199105052, + "recall": 0.5233793836344315, + "far": 0.9986311744800895, + "csi": 0.0013671213559623145, + "f1": 0.0027305097737003402, + "f2": 0.00677326944684737, + "frequency_bias": 382.3565356004251, + "predicted_positive_rate": 0.08900371057513914 + }, + "0.7000": { + "threshold": 0.7, + "tp": 718, + "fp": 439397, + "fn": 1164, + "precision": 0.0016313917953262216, + "recall": 0.38150903294367694, + "far": 0.9983686082046738, + "csi": 0.0016270885312919944, + "f1": 0.0032488908295757665, + "f2": 0.008019783622216812, + "frequency_bias": 233.85494155154092, + "predicted_positive_rate": 0.05443599257884972 + }, + "0.9000": { + "threshold": 0.9, + "tp": 505, + "fp": 234301, + "fn": 1377, + "precision": 0.0021507116513206645, + "recall": 0.2683315621679065, + "far": 0.9978492883486794, + "csi": 0.002138172518767227, + "f1": 0.004267220982897315, + "f2": 0.010419503660237521, + "frequency_bias": 124.76408076514346, + "predicted_positive_rate": 0.0290421768707483 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0025108226109296083, + "recall": 0.10786397755146027, + "lift": 10.786398051148842 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0016870747786015272, + "recall": 0.3623804450035095, + "lift": 7.247608821442164 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.00131601735483855, + "recall": 0.5653560161590576, + "lift": 5.653560299209267 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0012350949695771885, + "2": 0.0012982024046330753, + "4": 0.0013891551578664174, + "8": 0.0014700961990706807 + }, + "0.2000": { + "1": 0.0019807045037021354, + "2": 0.0020895005644333553, + "4": 0.002273115825633387, + "8": 0.0024347403778096632 + }, + "0.3000": { + "1": 0.0024896008041010997, + "2": 0.002650367743360804, + "4": 0.0029762248158465843, + "8": 0.003287343877128479 + }, + "0.5000": { + "1": 0.003163524553950414, + "2": 0.00349047890930243, + "4": 0.004130384099124695, + "8": 0.004813550356828178 + }, + "0.7000": { + "1": 0.003977167875739984, + "2": 0.0045831981839108415, + "4": 0.005821352827483972, + "8": 0.007370126727222481 + }, + "0.9000": { + "1": 0.005249887506590167, + "2": 0.006010307870879972, + "4": 0.008156507820926917, + "8": 0.011534026564462896 + } + }, + "minutes": 0.34152236382166545 + }, + { + "epoch": 6, + "train_loss": 0.4729130111568113, + "train_main_loss": 0.44654953663776475, + "train_aux_spatial_loss": 0.052726947147827274, + "val_loss": 0.3501560365160306, + "val_pr_auc": 0.0013937340471345924, + "val_auroc": 0.86562865600496, + "val_brier": 0.09354882687330246, + "val_brier_skill_score": -400.97212468650577, + "val_ece": 0.19507035652011379, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0012431519742859437, + "val_positive_recall": 0.6333687566418703, + "val_far": 0.9987568480257141, + "val_csi": 0.0012422580332512457, + "val_f1": 0.002481433485820752, + "val_f2": 0.006167339796622656, + "val_frequency_bias": 509.48618490967056, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1802, + "fp": 4151030, + "fn": 80, + "precision": 0.0004339207557637776, + "recall": 0.9574920297555791, + "far": 0.9995660792442362, + "csi": 0.0004339123968916269, + "f1": 0.000867448397170058, + "f2": 0.0021656779701756577, + "frequency_bias": 2206.605738575983, + "predicted_positive_rate": 0.5136465058750773 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1629, + "fp": 2545183, + "fn": 253, + "precision": 0.000639623183807835, + "recall": 0.8655685441020191, + "far": 0.9993603768161922, + "csi": 0.0006395596500285623, + "f1": 0.0012783017498373677, + "f2": 0.0031886906206691355, + "frequency_bias": 1353.2476089266738, + "predicted_positive_rate": 0.31500457637600493 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1501, + "fp": 1707825, + "fn": 381, + "precision": 0.0008781238921071814, + "recall": 0.7975557917109458, + "far": 0.9991218761078928, + "csi": 0.0008779282064119759, + "f1": 0.0017543162491058947, + "f2": 0.004371367629396559, + "frequency_bias": 908.249734325186, + "predicted_positive_rate": 0.21141941867656153 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1192, + "fp": 957661, + "fn": 690, + "precision": 0.0012431519742859437, + "recall": 0.6333687566418703, + "far": 0.9987568480257141, + "csi": 0.0012422580332512457, + "f1": 0.002481433485820752, + "f2": 0.006167339796622656, + "frequency_bias": 509.48618490967056, + "predicted_positive_rate": 0.1185965367965368 + }, + "0.7000": { + "threshold": 0.7, + "tp": 855, + "fp": 519311, + "fn": 1027, + "precision": 0.0016437060476847776, + "recall": 0.4543039319872476, + "far": 0.9983562939523152, + "csi": 0.001640467158998682, + "f1": 0.003275560867966164, + "f2": 0.008101285972552275, + "frequency_bias": 276.39001062699253, + "predicted_positive_rate": 0.06433716759431045 + }, + "0.9000": { + "threshold": 0.9, + "tp": 460, + "fp": 219580, + "fn": 1422, + "precision": 0.002090528994728231, + "recall": 0.24442082890541977, + "far": 0.9979094710052717, + "csi": 0.002077105778869513, + "f1": 0.0041456007065545555, + "f2": 0.010106869155593053, + "frequency_bias": 116.91817215727949, + "predicted_positive_rate": 0.027215831787260357 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0018676561303436756, + "recall": 0.08023379743099213, + "lift": 8.023379412333956 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001798392040655017, + "recall": 0.38629117608070374, + "lift": 7.7258235281467655 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0013593073235824704, + "recall": 0.5839532613754272, + "lift": 5.839532351739428 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0009193882112880081, + "2": 0.0009544238669535465, + "4": 0.0010101364252731448, + "8": 0.0010646885958944807 + }, + "0.2000": { + "1": 0.0014043880513923135, + "2": 0.0014935574288226228, + "4": 0.0016362737109093528, + "8": 0.0017557443359328406 + }, + "0.3000": { + "1": 0.001958875397916504, + "2": 0.0020919404579550216, + "4": 0.002317532927398114, + "8": 0.002500376612063371 + }, + "0.5000": { + "1": 0.0028749173593225485, + "2": 0.0031265486723459635, + "4": 0.0035980207015140486, + "8": 0.004012266867574943 + }, + "0.7000": { + "1": 0.003977407204865502, + "2": 0.0044937585845883365, + "4": 0.005528768396049211, + "8": 0.006620101577705317 + }, + "0.9000": { + "1": 0.00520751276135345, + "2": 0.006124343725859349, + "4": 0.008435537832026108, + "8": 0.011484696864351607 + } + }, + "minutes": 0.33149927854537964 + }, + { + "epoch": 7, + "train_loss": 0.46297245338802046, + "train_main_loss": 0.43753425775094246, + "train_aux_spatial_loss": 0.050876390929891935, + "val_loss": 0.22136119877298674, + "val_pr_auc": 0.0020734326588658226, + "val_auroc": 0.8708066066652015, + "val_brier": 0.0556364431977272, + "val_brier_skill_score": -238.065523638041, + "val_ece": 0.12623748259798678, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0018673864945086484, + "val_positive_recall": 0.5462274176408076, + "val_far": 0.9981326135054913, + "val_csi": 0.0018644940836773337, + "val_f1": 0.003722048430077627, + "val_f2": 0.00921097432037704, + "val_frequency_bias": 292.5090329436769, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1637, + "fp": 2555675, + "fn": 245, + "precision": 0.0006401252565193453, + "recall": 0.8698193411264612, + "far": 0.9993598747434806, + "csi": 0.0006400639360139383, + "f1": 0.0012793090324531864, + "f2": 0.003191232201618814, + "frequency_bias": 1358.826780021254, + "predicted_positive_rate": 0.31630327767470623 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1451, + "fp": 1472886, + "fn": 431, + "precision": 0.0009841711901688692, + "recall": 0.7709883103081828, + "far": 0.9990158288098311, + "csi": 0.0009838835667711802, + "f1": 0.0019658329827755908, + "f2": 0.004895857584867718, + "frequency_bias": 783.3884165781084, + "predicted_positive_rate": 0.18235460729746444 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1316, + "fp": 1011772, + "fn": 566, + "precision": 0.0012989987049496193, + "recall": 0.6992561105207227, + "far": 0.9987010012950504, + "csi": 0.0012982733753332005, + "f1": 0.0025931800939929255, + "f2": 0.006447086857348895, + "frequency_bias": 538.3039319872477, + "predicted_positive_rate": 0.12530463821892393 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1028, + "fp": 549474, + "fn": 854, + "precision": 0.0018673864945086484, + "recall": 0.5462274176408076, + "far": 0.9981326135054913, + "csi": 0.0018644940836773337, + "f1": 0.003722048430077627, + "f2": 0.00921097432037704, + "frequency_bias": 292.5090329436769, + "predicted_positive_rate": 0.06808930117501547 + }, + "0.7000": { + "threshold": 0.7, + "tp": 769, + "fp": 311654, + "fn": 1113, + "precision": 0.002461406490559274, + "recall": 0.4086078639744952, + "far": 0.9975385935094407, + "csi": 0.0024526689120228616, + "f1": 0.004893336090739887, + "f2": 0.012017465174354824, + "frequency_bias": 166.0058448459086, + "predicted_positive_rate": 0.03864230055658627 + }, + "0.9000": { + "threshold": 0.9, + "tp": 413, + "fp": 132533, + "fn": 1469, + "precision": 0.0031065244535375264, + "recall": 0.21944739638682254, + "far": 0.9968934755464625, + "csi": 0.0030725737454897145, + "f1": 0.006126323908980331, + "f2": 0.014700229223913322, + "frequency_bias": 70.64080765143464, + "predicted_positive_rate": 0.016443537414965986 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.003426097799092531, + "recall": 0.1471838504076004, + "lift": 14.718385306198385 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.002142238663509488, + "recall": 0.46014878153800964, + "lift": 9.202975488825663 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.001512677758000791, + "recall": 0.6498405933380127, + "lift": 6.4984058809614424 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0014021766263225821, + "2": 0.001488854806077189, + "4": 0.001617522044606745, + "8": 0.0017209109810044643 + }, + "0.2000": { + "1": 0.0022323244436344147, + "2": 0.0024036083494088922, + "4": 0.0026669563437204946, + "8": 0.0028496477215000926 + }, + "0.3000": { + "1": 0.0030061288845402157, + "2": 0.0032706824694820336, + "4": 0.0037414303607975485, + "8": 0.004090089080580284 + }, + "0.5000": { + "1": 0.004510319885638658, + "2": 0.005095488079194688, + "4": 0.006224726885447285, + "8": 0.007265877228674111 + }, + "0.7000": { + "1": 0.0061981922002846135, + "2": 0.0071663426719156265, + "4": 0.009389004622336272, + "8": 0.011978850515245498 + }, + "0.9000": { + "1": 0.00831885386783937, + "2": 0.009977206958581508, + "4": 0.01488323805412095, + "8": 0.022524780923326215 + } + }, + "minutes": 0.33211313088734945 + }, + { + "epoch": 8, + "train_loss": 0.45174517886643994, + "train_main_loss": 0.4269228792569537, + "train_aux_spatial_loss": 0.04964459997631244, + "val_loss": 0.44725767920414605, + "val_pr_auc": 0.0027931174903348286, + "val_auroc": 0.8702263757214033, + "val_brier": 0.12641970813274384, + "val_brier_skill_score": -542.2157770315075, + "val_ece": 0.24778940444824657, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0010423633917768275, + "val_positive_recall": 0.7369819341126461, + "val_far": 0.9989576366082231, + "val_csi": 0.0010419757723729927, + "val_f1": 0.002081782377944814, + "val_f2": 0.005182497134120186, + "val_frequency_bias": 707.0297555791711, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1809, + "fp": 4950872, + "fn": 73, + "precision": 0.00036525671651374276, + "recall": 0.961211477151966, + "far": 0.9996347432834862, + "csi": 0.00036525133289478946, + "f1": 0.0007302359461369248, + "f2": 0.0018235118721811926, + "frequency_bias": 2631.605207226355, + "predicted_positive_rate": 0.6125764996907854 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1728, + "fp": 3373367, + "fn": 154, + "precision": 0.0005119855885538036, + "recall": 0.9181721572794899, + "far": 0.9994880144114462, + "csi": 0.0005119622285644703, + "f1": 0.0010234005147207101, + "f2": 0.002554230843933835, + "frequency_bias": 1793.355472901169, + "predicted_positive_rate": 0.4174514533085962 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1624, + "fp": 2396644, + "fn": 258, + "precision": 0.0006771553471088302, + "recall": 0.8629117959617428, + "far": 0.9993228446528911, + "csi": 0.0006770825081737701, + "f1": 0.001353248755286128, + "f2": 0.0033751822681557368, + "frequency_bias": 1274.3188097768332, + "predicted_positive_rate": 0.2966317872603587 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1387, + "fp": 1329243, + "fn": 495, + "precision": 0.0010423633917768275, + "recall": 0.7369819341126461, + "far": 0.9989576366082231, + "csi": 0.0010419757723729927, + "f1": 0.002081782377944814, + "f2": 0.005182497134120186, + "frequency_bias": 707.0297555791711, + "predicted_positive_rate": 0.16458008658008658 + }, + "0.7000": { + "threshold": 0.7, + "tp": 1109, + "fp": 713032, + "fn": 773, + "precision": 0.0015529146205021137, + "recall": 0.5892667375132837, + "far": 0.9984470853794979, + "csi": 0.0015512355332249752, + "f1": 0.0030976658571023555, + "f2": 0.007683577928385451, + "frequency_bias": 379.45855472901167, + "predicted_positive_rate": 0.0883291280148423 + }, + "0.9000": { + "threshold": 0.9, + "tp": 772, + "fp": 287094, + "fn": 1110, + "precision": 0.0026818033390535873, + "recall": 0.410201912858661, + "far": 0.9973181966609465, + "csi": 0.0026715021316649134, + "f1": 0.005328768447064346, + "f2": 0.013067293174539767, + "frequency_bias": 152.95749202975557, + "predicted_positive_rate": 0.035604947433518865 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0053803338669240475, + "recall": 0.23113708198070526, + "lift": 23.113708823592663 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0021595547441393137, + "recall": 0.46386823058128357, + "lift": 9.27736470993035 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014545454178005457, + "recall": 0.6248671412467957, + "lift": 6.248671567467869 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007674856988975076, + "2": 0.000791355414216266, + "4": 0.0008260328883219481, + "8": 0.0008700381800748058 + }, + "0.2000": { + "1": 0.0010883550972506928, + "2": 0.0011302905318852918, + "4": 0.0011919848085577556, + "8": 0.001259132880186864 + }, + "0.3000": { + "1": 0.0014676516650308224, + "2": 0.0015372280258556525, + "4": 0.0016419540607538474, + "8": 0.0017449860353127011 + }, + "0.5000": { + "1": 0.0023237192035810983, + "2": 0.002464085798703608, + "4": 0.002708399276045159, + "8": 0.0029286220283636366 + }, + "0.7000": { + "1": 0.0035873573947993176, + "2": 0.003932072661929298, + "4": 0.004541864058129508, + "8": 0.005095613653573028 + }, + "0.9000": { + "1": 0.006636955504364939, + "2": 0.007636333009147123, + "4": 0.009658326859772859, + "8": 0.012242707807434083 + } + }, + "minutes": 0.3327596187591553 + }, + { + "epoch": 9, + "train_loss": 0.44174556909476165, + "train_main_loss": 0.41717497324598773, + "train_aux_spatial_loss": 0.04914119221815917, + "val_loss": 0.35248868018388746, + "val_pr_auc": 0.0014017929970181718, + "val_auroc": 0.826237549108265, + "val_brier": 0.09342223405838013, + "val_brier_skill_score": -400.42816508289366, + "val_ece": 0.21254881950345395, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0011612270298949224, + "val_positive_recall": 0.5547290116896918, + "val_far": 0.998838772970105, + "val_csi": 0.00116014566273321, + "val_f1": 0.002317602568898173, + "val_f2": 0.00575792238276506, + "val_frequency_bias": 477.70935175345375, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1744, + "fp": 4810537, + "fn": 138, + "precision": 0.0003624061022205478, + "recall": 0.926673751328374, + "far": 0.9996375938977794, + "csi": 0.00036239570993298794, + "f1": 0.000724528853717666, + "f2": 0.0018092003230833422, + "frequency_bias": 2557.003719447396, + "predicted_positive_rate": 0.5952110080395795 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1569, + "fp": 2798322, + "fn": 313, + "precision": 0.0005603789576094213, + "recall": 0.8336875664187035, + "far": 0.9994396210423906, + "csi": 0.0005603163198109853, + "f1": 0.0011200050824959767, + "f2": 0.0027943816010363976, + "frequency_bias": 1487.7210414452709, + "predicted_positive_rate": 0.34630686456400744 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1407, + "fp": 1861031, + "fn": 475, + "precision": 0.0007554613898556623, + "recall": 0.7476089266737513, + "far": 0.9992445386101443, + "csi": 0.0007552687645638847, + "f1": 0.0015093975283213183, + "f2": 0.003762100487388541, + "frequency_bias": 989.605738575983, + "predicted_positive_rate": 0.23035720470006185 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1044, + "fp": 898005, + "fn": 838, + "precision": 0.0011612270298949224, + "recall": 0.5547290116896918, + "far": 0.998838772970105, + "csi": 0.00116014566273321, + "f1": 0.002317602568898173, + "f2": 0.00575792238276506, + "frequency_bias": 477.70935175345375, + "predicted_positive_rate": 0.11119962894248608 + }, + "0.7000": { + "threshold": 0.7, + "tp": 749, + "fp": 463515, + "fn": 1133, + "precision": 0.0016133062223217824, + "recall": 0.39798087141339, + "far": 0.9983866937776782, + "csi": 0.0016093786595100528, + "f1": 0.0032135854431873277, + "f2": 0.007937820056295994, + "frequency_bias": 246.6865037194474, + "predicted_positive_rate": 0.057422881880024736 + }, + "0.9000": { + "threshold": 0.9, + "tp": 453, + "fp": 189367, + "fn": 1429, + "precision": 0.0023864713939521654, + "recall": 0.24070138150903295, + "far": 0.9976135286060478, + "csi": 0.0023686398360252864, + "f1": 0.004726085278192194, + "f2": 0.011477187506333988, + "frequency_bias": 100.86078639744952, + "predicted_positive_rate": 0.02347804576376005 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0027334571350365877, + "recall": 0.1174282655119896, + "lift": 11.742827464558045 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0017217068234458566, + "recall": 0.3698193430900574, + "lift": 7.396386763535759 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012269634753465652, + "recall": 0.5270988345146179, + "lift": 5.270988233776119 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0007761889316962867, + "2": 0.0008088700048963782, + "4": 0.0008539254426118692, + "8": 0.0008953386395748053 + }, + "0.2000": { + "1": 0.0012289524564135235, + "2": 0.0012947806940213358, + "4": 0.0013901778317579883, + "8": 0.0014569956840254639 + }, + "0.3000": { + "1": 0.001693693219550152, + "2": 0.0017980729206314994, + "4": 0.001959161549363153, + "8": 0.0020775186421173597 + }, + "0.5000": { + "1": 0.002685166736102307, + "2": 0.0029421415350803226, + "4": 0.0034770871818433635, + "8": 0.003947886788339461 + }, + "0.7000": { + "1": 0.003915058495870372, + "2": 0.00452728637874078, + "4": 0.005929248077012117, + "8": 0.007441641374286712 + }, + "0.9000": { + "1": 0.006463210719885204, + "2": 0.00788266562330997, + "4": 0.011779975914961827, + "8": 0.01823609329615261 + } + }, + "minutes": 0.32817530234654746 + }, + { + "epoch": 10, + "train_loss": 0.43139428173367955, + "train_main_loss": 0.4073724075672085, + "train_aux_spatial_loss": 0.048043749015301965, + "val_loss": 0.2240987864633401, + "val_pr_auc": 0.003116102503753257, + "val_auroc": 0.874721890484618, + "val_brier": 0.05645371973514557, + "val_brier_skill_score": -241.57729815390343, + "val_ece": 0.1303528208549411, + "val_positive_rate": 0.00023277674335986376, + "val_precision": 0.0018323169746849528, + "val_positive_recall": 0.5425079702444209, + "val_far": 0.998167683025315, + "val_csi": 0.0018294900901126901, + "val_f1": 0.003652298336612413, + "val_f2": 0.009039461988221253, + "val_frequency_bias": 296.07757704569605, + "val_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1658, + "fp": 2737917, + "fn": 224, + "precision": 0.0006052033618353212, + "recall": 0.8809776833156217, + "far": 0.9993947966381647, + "csi": 0.0006051538817263602, + "f1": 0.0012095757839718077, + "f2": 0.0030177244901265082, + "frequency_bias": 1455.67215727949, + "predicted_positive_rate": 0.33884662956091527 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1447, + "fp": 1498772, + "fn": 435, + "precision": 0.0009645258458931663, + "recall": 0.7688629117959618, + "far": 0.9990354741541069, + "csi": 0.0009642462552993562, + "f1": 0.0019266347602458157, + "f2": 0.004798550419931196, + "frequency_bias": 797.1408076514347, + "predicted_positive_rate": 0.18555584415584417 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1304, + "fp": 1006923, + "fn": 578, + "precision": 0.0012933595311373331, + "recall": 0.6928799149840595, + "far": 0.9987066404688627, + "csi": 0.0012926184941589307, + "f1": 0.0025818995771743447, + "f2": 0.0064188706922437, + "frequency_bias": 535.721041445271, + "predicted_positive_rate": 0.12470340136054421 + }, + "0.5000": { + "threshold": 0.5, + "tp": 1021, + "fp": 556197, + "fn": 861, + "precision": 0.0018323169746849528, + "recall": 0.5425079702444209, + "far": 0.998167683025315, + "csi": 0.0018294900901126901, + "f1": 0.003652298336612413, + "f2": 0.009039461988221253, + "frequency_bias": 296.07757704569605, + "predicted_positive_rate": 0.0689199752628324 + }, + "0.7000": { + "threshold": 0.7, + "tp": 820, + "fp": 311338, + "fn": 1062, + "precision": 0.0026268748518378515, + "recall": 0.4357066950053135, + "far": 0.9973731251481621, + "csi": 0.002617968201264287, + "f1": 0.0052222646796586435, + "f2": 0.012825084614277761, + "frequency_bias": 165.86503719447396, + "predicted_positive_rate": 0.03860952380952381 + }, + "0.9000": { + "threshold": 0.9, + "tp": 560, + "fp": 134308, + "fn": 1322, + "precision": 0.004152208084942314, + "recall": 0.2975557917109458, + "far": 0.9958477919150577, + "csi": 0.004111902489169542, + "f1": 0.008190127970749542, + "f2": 0.019663473693081267, + "frequency_bias": 71.66206163655686, + "predicted_positive_rate": 0.01668126159554731 + } + }, + "val_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.004922696389257908, + "recall": 0.2114771455526352, + "lift": 21.147715696183667 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.0022238714154809713, + "recall": 0.47768330574035645, + "lift": 9.553666673834993 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0014669140800833702, + "recall": 0.6301806569099426, + "lift": 6.301806868290009 + } + }, + "val_fss": { + "0.1000": { + "1": 0.0013048518501964645, + "2": 0.0013652965653525584, + "4": 0.0014515110692925326, + "8": 0.0015328291894178836 + }, + "0.2000": { + "1": 0.0021372784576736414, + "2": 0.002257479647874039, + "4": 0.002436579126223748, + "8": 0.0025690244682270436 + }, + "0.3000": { + "1": 0.00287961415383009, + "2": 0.0030574405787181513, + "4": 0.0033325979533164185, + "8": 0.00352065311784866 + }, + "0.5000": { + "1": 0.004248566368253326, + "2": 0.004625449177877106, + "4": 0.005259187972128654, + "8": 0.00579578478616205 + }, + "0.7000": { + "1": 0.006428046413327948, + "2": 0.007130644667450725, + "4": 0.008581068668154113, + "8": 0.010327699790465106 + }, + "0.9000": { + "1": 0.01054567134777984, + "2": 0.01212271654616559, + "4": 0.015815071561349092, + "8": 0.02133905715867812 + } + }, + "minutes": 0.32832499742507937 + } +] diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/run_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/run_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..7dd55d9d4d721878a50a4d04faecf601d81fd41e --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/run_summary.json @@ -0,0 +1,172 @@ +{ + "run_name": "reference_fireprone_seed_99_20260504", + "device": "cuda", + "index_root": "", + "input_channels": 16, + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "train_positive_rate": 0.0018334667181808147, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight": 50.0, + "use_aux_spatial_head": true, + "aux_spatial_radius": 2, + "aux_spatial_loss_weight": 0.5, + "aux_positive_rate": 0.004560296846011132, + "init_checkpoint": "", + "best_val_pr_auc": 0.003116102503753257, + "best_val_auroc": 0.874721890484618, + "best_test_pr_auc": 0.001365278660864068, + "best_test_auroc": 0.82665163680193, + "best_test_brier": 0.07600810378789902, + "best_test_ece": 0.15982505350344936, + "best_test_topk_area_metrics": { + "0.0100": { + "fraction": 0.01, + "cells": 80850, + "precision": 0.0023623995948582888, + "recall": 0.0973992869257927, + "lift": 9.73992880989997 + }, + "0.0500": { + "fraction": 0.05, + "cells": 404250, + "precision": 0.001677179941907525, + "recall": 0.34574195742607117, + "lift": 6.914839162318504 + }, + "0.1000": { + "fraction": 0.1, + "cells": 808500, + "precision": 0.0012034631799906492, + "recall": 0.4961754083633423, + "lift": 4.961754024999269 + } + }, + "best_test_threshold_metrics": { + "0.1000": { + "threshold": 0.1, + "tp": 1701, + "fp": 3149821, + "fn": 260, + "precision": 0.0005397392117205592, + "recall": 0.8674145843957165, + "far": 0.9994602607882794, + "csi": 0.0005396946870056368, + "f1": 0.0010788071475254504, + "f2": 0.0026919957991571727, + "frequency_bias": 1607.0994390617032, + "predicted_positive_rate": 0.3897986394557823 + }, + "0.2000": { + "threshold": 0.2, + "tp": 1452, + "fp": 1964811, + "fn": 509, + "precision": 0.0007384566561034816, + "recall": 0.7404385517593065, + "far": 0.9992615433438965, + "csi": 0.0007382655437437588, + "f1": 0.001475441819630286, + "f2": 0.0036776122064305534, + "frequency_bias": 1002.6838347781744, + "predicted_positive_rate": 0.24319888682745824 + }, + "0.3000": { + "threshold": 0.3, + "tp": 1287, + "fp": 1409052, + "fn": 674, + "precision": 0.000912546557955215, + "recall": 0.6562978072412035, + "far": 0.9990874534420447, + "csi": 0.0009121106609223303, + "f1": 0.00182255894639949, + "f2": 0.00453749621875315, + "frequency_bias": 719.1937786843447, + "predicted_positive_rate": 0.17443896103896103 + }, + "0.5000": { + "threshold": 0.5, + "tp": 968, + "fp": 802322, + "fn": 993, + "precision": 0.0012050442554992593, + "recall": 0.49362570117287097, + "far": 0.9987949557445007, + "csi": 0.001203556459604393, + "f1": 0.0024042193055333057, + "f2": 0.005966954905108157, + "frequency_bias": 409.6328403875574, + "predicted_positive_rate": 0.09935559678416821 + }, + "0.7000": { + "threshold": 0.7, + "tp": 707, + "fp": 432996, + "fn": 1254, + "precision": 0.001630147820051971, + "recall": 0.36053034166241715, + "far": 0.998369852179948, + "csi": 0.0016254480327940464, + "f1": 0.00324562047816666, + "f2": 0.008005942742222232, + "frequency_bias": 221.16420193778686, + "predicted_positive_rate": 0.05364291898577613 + }, + "0.9000": { + "threshold": 0.9, + "tp": 409, + "fp": 163803, + "fn": 1552, + "precision": 0.002490682775923806, + "recall": 0.2085670576236614, + "far": 0.9975093172240762, + "csi": 0.0024673632393040707, + "f1": 0.004922580683985966, + "f2": 0.011885665132282513, + "frequency_bias": 83.7389087200408, + "predicted_positive_rate": 0.020310698824984538 + } + }, + "best_test_fss": { + "0.1000": { + "1": 0.001143681210237446, + "2": 0.0011847434725281714, + "4": 0.001236591045744806, + "8": 0.0012702633695319543 + }, + "0.2000": { + "1": 0.0015924077611619847, + "2": 0.0016590472832728187, + "4": 0.0017424757489505893, + "8": 0.0017626104493195793 + }, + "0.3000": { + "1": 0.0019615416657199347, + "2": 0.002042614896579975, + "4": 0.0021496045193630797, + "8": 0.0021595197888091278 + }, + "0.5000": { + "1": 0.0026900758912903333, + "2": 0.0028522394616976587, + "4": 0.0030564017994608728, + "8": 0.003127168867652075 + }, + "0.7000": { + "1": 0.0038445992739185586, + "2": 0.004171973927427075, + "4": 0.00463101415464473, + "8": 0.004986515076567888 + }, + "0.9000": { + "1": 0.0064130574398052476, + "2": 0.007473703614246263, + "4": 0.00958738894872413, + "8": 0.012579819257860225 + } + }, + "epochs": 10 +} diff --git a/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/tile_summary.json b/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/tile_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..38db43a1e57d30bf3cbb4d9873a51294d8ef8055 --- /dev/null +++ b/models/wildfire_fm/metrics/reference_fireprone_seed_99_20260504/tile_summary.json @@ -0,0 +1,16 @@ +{ + "index_root": "", + "tile_size": 32, + "num_train_tiles": 27177, + "positive_train_tiles": 6724, + "negative_train_tiles": 20453, + "input_channels": 16, + "train_positive_rate": 0.0018334667181808147, + "metric_reference_positive_rate": 0.0002787771234976204, + "pos_weight_source": "tiles", + "positive_rate_source": "tiles", + "pos_weight": 50.0, + "loss_type": "bce", + "train_target_mode": "dilate_max", + "train_target_radius": 2 +} diff --git a/models/wildfire_fm/modeling_unet.py b/models/wildfire_fm/modeling_unet.py new file mode 100644 index 0000000000000000000000000000000000000000..ab17549f49ac5219e151633ac6aac728ea3d7695 --- /dev/null +++ b/models/wildfire_fm/modeling_unet.py @@ -0,0 +1,109 @@ +"""Compact U-Net architecture for the released WildFIRE-FM checkpoints.""" + +from __future__ import annotations + +import math + +import torch +import torch.nn as nn +import torch.nn.functional as F + + +def make_norm(norm_type: str, num_channels: int, norm_groups: int) -> nn.Module: + if norm_type == "batch": + return nn.BatchNorm2d(num_channels) + if norm_type == "group": + groups = max(1, min(int(norm_groups), num_channels)) + while num_channels % groups != 0 and groups > 1: + groups -= 1 + return nn.GroupNorm(groups, num_channels) + if norm_type == "instance": + return nn.InstanceNorm2d(num_channels, affine=True) + if norm_type in {"none", "identity"}: + return nn.Identity() + raise ValueError(f"Unsupported norm_type: {norm_type}") + + +class ConvBlock(nn.Module): + def __init__(self, in_ch: int, out_ch: int, norm_type: str, norm_groups: int): + super().__init__() + self.net = nn.Sequential( + nn.Conv2d(in_ch, out_ch, 3, padding=1, bias=False), + make_norm(norm_type, out_ch, norm_groups), + nn.ReLU(inplace=True), + nn.Conv2d(out_ch, out_ch, 3, padding=1, bias=False), + make_norm(norm_type, out_ch, norm_groups), + nn.ReLU(inplace=True), + ) + + def forward(self, x: torch.Tensor) -> torch.Tensor: + return self.net(x) + + +class UNetSmallFlex(nn.Module): + def __init__( + self, + in_ch: int, + base: int = 32, + dropout: float = 0.1, + norm_type: str = "group", + norm_groups: int = 8, + prior_prob: float | None = None, + use_aux_spatial_head: bool = False, + aux_prior_prob: float | None = None, + ): + super().__init__() + self.enc1 = ConvBlock(in_ch, base, norm_type, norm_groups) + self.enc2 = ConvBlock(base, base * 2, norm_type, norm_groups) + self.enc3 = ConvBlock(base * 2, base * 4, norm_type, norm_groups) + self.enc4 = ConvBlock(base * 4, base * 8, norm_type, norm_groups) + self.pool = nn.MaxPool2d(2) + self.bottleneck = ConvBlock(base * 8, base * 16, norm_type, norm_groups) + self.up4 = nn.ConvTranspose2d(base * 16, base * 8, 2, stride=2) + self.dec4 = ConvBlock(base * 16, base * 8, norm_type, norm_groups) + self.up3 = nn.ConvTranspose2d(base * 8, base * 4, 2, stride=2) + self.dec3 = ConvBlock(base * 8, base * 4, norm_type, norm_groups) + self.up2 = nn.ConvTranspose2d(base * 4, base * 2, 2, stride=2) + self.dec2 = ConvBlock(base * 4, base * 2, norm_type, norm_groups) + self.up1 = nn.ConvTranspose2d(base * 2, base, 2, stride=2) + self.dec1 = ConvBlock(base * 2, base, norm_type, norm_groups) + self.drop = nn.Dropout2d(p=dropout) + self.head = nn.Conv2d(base, 1, kernel_size=1) + self.use_aux_spatial_head = bool(use_aux_spatial_head) + self.aux_head = nn.Conv2d(base, 1, kernel_size=1) if self.use_aux_spatial_head else None + if prior_prob is not None: + prior_prob = float(min(max(prior_prob, 1e-6), 1.0 - 1e-6)) + nn.init.constant_(self.head.bias, math.log(prior_prob / (1.0 - prior_prob))) + if self.aux_head is not None and aux_prior_prob is not None: + aux_prior_prob = float(min(max(aux_prior_prob, 1e-6), 1.0 - 1e-6)) + nn.init.constant_(self.aux_head.bias, math.log(aux_prior_prob / (1.0 - aux_prior_prob))) + + @staticmethod + def _match_hw(x: torch.Tensor, ref: torch.Tensor) -> torch.Tensor: + diff_y = ref.size(2) - x.size(2) + diff_x = ref.size(3) - x.size(3) + if diff_y > 0 or diff_x > 0: + x = F.pad(x, [diff_x // 2, diff_x - diff_x // 2, diff_y // 2, diff_y - diff_y // 2]) + if diff_y < 0: + y0 = (-diff_y) // 2 + x = x[:, :, y0 : y0 + ref.size(2), :] + if diff_x < 0: + x0 = (-diff_x) // 2 + x = x[:, :, :, x0 : x0 + ref.size(3)] + return x + + def forward(self, x: torch.Tensor, return_aux: bool = False): + e1 = self.enc1(x) + e2 = self.enc2(self.pool(e1)) + e3 = self.enc3(self.pool(e2)) + e4 = self.enc4(self.pool(e3)) + b = self.bottleneck(self.pool(e4)) + d4 = self.dec4(torch.cat([self._match_hw(self.up4(b), e4), e4], dim=1)) + d3 = self.dec3(torch.cat([self._match_hw(self.up3(d4), e3), e3], dim=1)) + d2 = self.dec2(torch.cat([self._match_hw(self.up2(d3), e2), e2], dim=1)) + d1 = self.dec1(torch.cat([self._match_hw(self.up1(d2), e1), e1], dim=1)) + features = self.drop(d1) + logits = self.head(features) + if return_aux and self.aux_head is not None: + return logits, self.aux_head(features) + return logits diff --git a/paper/main.tex b/paper/main.tex new file mode 100644 index 0000000000000000000000000000000000000000..5e561b399ae8d4f8ab4d529ade8a44656ca3fa72 --- /dev/null +++ b/paper/main.tex @@ -0,0 +1,141 @@ +% !TeX root = main.tex +% !TeX program = pdflatex +\documentclass{article} +\usepackage[preprint]{neurips_2026} +\usepackage[utf8]{inputenc} % allow utf-8 input +\usepackage[T1]{fontenc} % use 8-bit T1 fonts +\usepackage{hyperref} % hyperlinks +\usepackage{url} % simple URL typesetting +\usepackage{booktabs} % professional-quality tables +\usepackage{amsfonts} % blackboard math symbols +\usepackage{nicefrac} % compact symbols for 1/2, etc. +\usepackage{microtype} % microtypography +\usepackage[table]{xcolor} % colors +\usepackage{placeins} +\usepackage[utf8]{inputenc} +\usepackage[T1]{fontenc} +\usepackage{hyperref} +\setcitestyle{numbers,square} +\definecolor{tocblue}{RGB}{31, 73, 125} +\hypersetup{ + colorlinks=false, + citebordercolor=green, + linkbordercolor=green, + urlbordercolor=blue, + pdfauthor={Yangshuang Xu, Yuyang Dai, Liling Chang, Qi Wang, Yushun Dong}, + pdftitle={Does Your Wildfire Prediction Model Actually Work, or Just Score Well?}, + pdfsubject={}, + pdfkeywords={} +} + +\usepackage{url} +\usepackage{booktabs} +\usepackage{amsfonts} +\usepackage{nicefrac} +\usepackage{microtype} +\usepackage{amsmath} +\usepackage{amssymb} +\usepackage{graphicx} +\usepackage{tabularx} +\usepackage{longtable} +\usepackage{multirow} +\usepackage{array} +\usepackage{float} +\usepackage{adjustbox} +\usepackage{placeins} +\usepackage{enumitem} +\usepackage{siunitx} +\usepackage{tikz} +\usepackage{subcaption} +\usepackage{wrapfig} +\usepackage[normalem]{ulem} +\usepackage{pifont} +\usepackage{hyperref} +\usepackage{xcolor} +\usepackage{tabularx} +\usepackage{xspace} + + +\sisetup{detect-all} + +\definecolor{wfblue}{RGB}{42,111,151} +\definecolor{wforange}{RGB}{231,111,81} +\definecolor{wfgreen}{RGB}{42,157,143} +\definecolor{wfgold}{RGB}{233,196,106} +\definecolor{wfslate}{RGB}{38,70,83} +\definecolor{wfgray}{RGB}{108,117,125} +\definecolor{wfpurple}{RGB}{116,81,164} +\definecolor{wfindigo}{RGB}{77,100,166} +\definecolor{wfrose}{RGB}{188,80,144} +\definecolor{wfolive}{RGB}{120,143,64} +\definecolor{primarybg}{RGB}{219,234,254} +\definecolor{primaryrule}{RGB}{147,197,253} +\definecolor{headerbg}{RGB}{30,58,138} +\definecolor{regbg}{RGB}{240,240,240} +\definecolor{retrbg}{RGB}{232,232,232} +\definecolor{refbg}{RGB}{255,243,205} % amber – reference row +\definecolor{alphabg}{RGB}{220,237,220} % green – AlphaEarth row +\definecolor{subheadbg}{RGB}{241,245,249} % near-white – column subheader +\definecolor{bestval}{RGB}{0,100,0} % dark green – best frozen value +\definecolor{warnval}{RGB}{180,0,0} % dark red – anomalous value + +\newcolumntype{L}[1]{>{\raggedright\arraybackslash}p{#1}} +\newcolumntype{Y}{>{\raggedright\arraybackslash}X} +\newcommand{\ms}[2] +{\ensuremath{#1{\mkern1mu}_{\scriptscriptstyle \pm #2}}} +\newcommand{\msb}[2]{\ensuremath{\mathbf{#1 \pm #2}}} + +\newcommand{\best}[1]{\textbf{#1}} +\newcommand{\ourfm}{\textsc{Wild}{\textbf{FIRE}}\textsc{-FM}\xspace} +% \title{Ranking Is Not Decision Quality: Evaluation Contracts for Wildfire-Centric Transfer} +\title{Does Your Wildfire Prediction Model Actually Work,\\ or Just Score Well?} + + +\author{% + Yangshuang Xu\thanks{Equal contribution.} \\ + Florida State University \\ + \texttt{yx21e@fsu.edu} \\ + \And + Yuyang Dai\footnotemark[1] \\ + Florida State University \\ + \texttt{yd26@fsu.edu} \\ + \And + Liling Chang \\ + Florida State University \\ + \texttt{liling.chang@fsu.edu} \\ + \And + Qi Wang \\ + Northeastern University \\ + \texttt{wangqi@vt.edu} \\ + \And + Yushun Dong\thanks{Corresponding author.} \\ + Florida State University \\ + \texttt{yd24f@fsu.edu} \\ +} + +\begin{document} + +\maketitle + +\input{sections/0_abstract} +\input{sections/1_intro} +\input{sections/2_backbone} +% \input{sections/background} +\input{sections/3_prelim} +% \input{sections/methodology} +\input{sections/4_experiments} +\input{sections/5_conclusion} + +% \bibliographystyle{abbrvnat} +\bibliographystyle{plain} +\bibliography{references} + +\newpage +\input{sections/appendix} + +\clearpage +\input{checklist_filled} +\clearpage + + +\end{document} diff --git a/paper/manuscript_final.pdf b/paper/manuscript_final.pdf new file mode 100644 index 0000000000000000000000000000000000000000..eff930d4691b73ed1b0ad00de027ca595562c910 --- /dev/null +++ b/paper/manuscript_final.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c342978b2f0f25cf6e430b860702895bbb3b512145c8c6e38aa2233b416d835e +size 297362 diff --git a/paper/references.bib b/paper/references.bib new file mode 100644 index 0000000000000000000000000000000000000000..ab486de31637739b4f2c5c5f3e38dfa2bc897f46 --- /dev/null +++ b/paper/references.bib @@ -0,0 +1,465 @@ +@article{lam2023graphcast, + title = {Learning skillful medium-range global weather forecasting}, + author = {Lam, Remi and Sanchez-Gonzalez, Alvaro and Willson, Matthew and Wirnsberger, Peter and Fortunato, Meire and Alet, Ferran and Ravuri, Suman and Ewalds, Timo and Eaton-Rosen, Zach and Hu, Weihua and others}, + journal = {Science}, + volume = {382}, + number = {6677}, + pages = {1416--1421}, + year = {2023} +} + +@article{ebert2009neighborhood, + title = {Neighborhood Verification: A Strategy for Rewarding Close Forecasts}, + author = {Ebert, Elizabeth E.}, + journal = {Weather and Forecasting}, + volume = {24}, + number = {6}, + pages = {1498--1510}, + year = {2009}, + doi = {10.1175/2009WAF2222251.1} +} + +@inproceedings{ronneberger2015unet, + title = {U-Net: Convolutional Networks for Biomedical Image Segmentation}, + author = {Ronneberger, Olaf and Fischer, Philipp and Brox, Thomas}, + booktitle = {Medical Image Computing and Computer-Assisted Intervention}, + pages = {234--241}, + year = {2015}, + doi = {10.1007/978-3-319-24574-4_28} +} + +@misc{noaa_hrrr_ncei, + title = {{Rapid Refresh / High-Resolution Rapid Refresh}}, + author = {{NOAA National Centers for Environmental Information}}, + year = {2026}, + howpublished = {\url{https://www.ncei.noaa.gov/products/weather-climate-models/rapid-refresh-update}}, + note = {Accessed: 2026-05-05} +} + +@misc{noaa_hrrr_emc, + title = {{High-Resolution Rapid Refresh (HRRR)}}, + author = {{NOAA National Centers for Environmental Prediction Environmental Modeling Center}}, + year = {2026}, + howpublished = {\url{https://rapidrefresh.noaa.gov/hrrr/}}, + note = {Accessed: 2026-05-05} +} + +@misc{nasa_firms, + title = {{Fire Information for Resource Management System (FIRMS)}}, + author = {{NASA Earthdata}}, + year = {2026}, + howpublished = {\url{https://www.earthdata.nasa.gov/data/tools/firms}}, + note = {Accessed: 2026-05-05} +} + +@misc{landfire_fbfm40, + title = {{LANDFIRE 40 Fire Behavior Fuel Models}}, + author = {{LANDFIRE}}, + year = {2026}, + howpublished = {\url{https://landfire.gov/fuel/fbfm40}}, + note = {Accessed: 2026-05-05} +} + +@misc{landfire_canopy_cover, + title = {{LANDFIRE Forest Canopy Cover}}, + author = {{LANDFIRE}}, + year = {2026}, + howpublished = {\url{https://landfire.gov/fuel/cc}}, + note = {Accessed: 2026-05-05} +} + +@misc{usfs_wrc_housing_density, + title = {{Wildfire Risk to Communities: Housing Unit Density Image Service}}, + author = {{USDA Forest Service}}, + year = {2026}, + howpublished = {\url{https://catalog.data.gov/dataset/wildfire-risk-to-communities-housing-unit-density-image-service-fac22}}, + note = {Accessed: 2026-05-05} +} + +@misc{ornl_landscan_2024, + title = {{LandScan Global 2024}}, + author = {{Oak Ridge National Laboratory}}, + year = {2024}, + howpublished = {\url{https://landscan.ornl.gov/}}, + note = {Accessed: 2026-05-05} +} + +@misc{nifc_wfigs_perimeters, + title = {{Wildland Fire Interagency Geospatial Services (WFIGS): Current Perimeters}}, + author = {{National Interagency Fire Center}}, + year = {2026}, + howpublished = {\url{https://data-nifc.opendata.arcgis.com/datasets/nifc::wfigs-current-perimeters/about}}, + note = {Accessed: 2026-05-05} +} + +@misc{mtbs_usgs_2025, + title = {{Monitoring Trends in Burn Severity (MTBS)}}, + author = {{U.S. Geological Survey and USDA Forest Service}}, + year = {2025}, + howpublished = {\url{https://www.mtbs.gov/}}, + note = {Accessed: 2026-05-05} +} + +@article{pickell2017early, + title={An early warning system to forecast the close of the spring burning window from satellite-observed greenness}, + author={Pickell, Paul D and Coops, Nicholas C and Ferster, Colin J and Bater, Christopher W and Blouin, Karen D and Flannigan, Mike D and Zhang, Jinkai}, + journal={Scientific Reports}, + volume={7}, + number={1}, + pages={14190}, + year={2017}, + publisher={Nature Publishing Group} +} + +@article{khosravi2025assessing, + title={Assessing Pan-Canada wildfire susceptibility by integrating satellite data with novel hybrid deep learning and black widow optimizer algorithms}, + author={Khosravi, Khabat and Mosallanejad, Ashkan and Bateni, Sayed M. and Kim, Dongkyun and Jun, Changhyun and Shahvaran, Ali Reza and Farooque, Aitazaz A. and Karbasi, Massoud and Ali, Mumtaz}, + journal={Science of the Total Environment}, + volume={977}, + year={2025}, + publisher={Elsevier}, + doi={10.1016/j.scitotenv.2025.179369} +} + +@article{vetrita2025drought, + title={Drought and Fine Fuel Moisture Code Evaluation: An Early Warning System for Forest/Land Fire Using Remote Sensing Approach}, + author={Vetrita, Yenni and Prasasti, Indah and Haryani, Nanik Suryo and Priyatna, M. and Khomarudin, M. Rokhis}, + journal={International Journal of Remote Sensing and Earth Sciences}, + volume={9}, + number={2}, + year={2025}, + doi={10.30536/ijreses.v9i2.13954} +} + +@inproceedings{xu2026bcwildfire, + title={BCWildfire: A Long-term Multi-factor Dataset and Deep Learning Benchmark for Boreal Wildfire Risk Prediction}, + author={Xu, Zhengsen and Cheng, Sibo and Wang, Lanying and He, Hongjie and Sun, Wentao and Li, Jonathan and Xu, Lincoln Linlin}, + booktitle={Proceedings of the AAAI Conference on Artificial Intelligence}, + volume={40}, + number={46}, + pages={39486--39494}, + year={2026}, + doi={10.1609/aaai.v40i46.41299} +} + +@article{krstinic2026spatio, + title={Spatio-temporal data model for early wildfire detection}, + author={Krstini{\'c}, Damir and Bejo, Jakov and Sikora, Toma and Bugari{\'c}, Marin}, + journal={Fire}, + volume={9}, + number={4}, + pages={175}, + year={2026}, + doi={10.3390/fire9040175} +} + +@article{kotroni2020disarm, + title={DISARM Early Warning System for Wildfires in the Eastern Mediterranean}, + author={Kotroni, Vassiliki and Cartalis, Constantinos and Michaelides, Silas and Stoyanova, Julia and Tymvios, Filippos and Bezes, Antonis and Christoudias, Theodoros and Dafis, Stavros and Giannakopoulos, Christos and Giannaros, Theodore M. and others}, + journal={Sustainability}, + volume={12}, + number={16}, + pages={6670}, + year={2020}, + publisher={MDPI}, + doi={10.3390/su12166670} +} + +@article{ts2025satfire, + title={TS-SatFire: A Multi-Task Satellite Image Time-Series Dataset for Wildfire Detection and Prediction}, + author={TS-SatFire Consortium}, + journal={Scientific Data}, + volume={12}, + pages={1817}, + year={2025}, + doi={10.1038/s41597-025-06271-3} +} + +@article{nujjoo2025modelling, + title={Modelling Spatial-Temporal Wildfire Susceptibility Using Geospatial Techniques Over the Table Mountain Nature Reserve, South Africa}, + author={Nujjoo, Syed Tanweer Raza and Odera, Patroba Achola}, + journal={Geoplanning: Journal of Geomatics and Planning}, + volume={12}, + number={2}, + pages={197--214}, + year={2025}, + doi={10.14710/geoplanning.12.2.197-214} +} + +@article{singh2025benchmarking, + title={Benchmarking Artificial Neural Networks and U-Net Convolutional Architectures for Wildfire Susceptibility Prediction: Innovations in Geospatial Intelligence}, + author={Singh, Harikesh and Ang, Li-Minn and Srivastava, Sanjeev Kumar}, + journal={IEEE Transactions on Geoscience and Remote Sensing}, + volume={63}, + year={2025}, + publisher={IEEE}, + doi={10.1109/TGRS.2025.3529134} +} + +@inproceedings{goldammer1999early, + title={Early warning systems for the prediction of and appropriate response to wildfires and related environmental hazards}, + author={Goldammer, Johann Georg}, + booktitle={Early Warning Systems for Natural Disaster Reduction}, + year={1999} +} + +@article{huot2022nextday, + title={Next day wildfire prediction using deep learning}, + author={Huot, Fantine and Hu, R Lily and Goyal, Nita and Sankar, Tharun and Ihme, Matthias and Chen, Yi-Fan}, + journal={arXiv preprint arXiv:2206.08930}, + year={2022} +} + +@inproceedings{radke2019firecast, + title={FireCast: Leveraging deep learning to predict wildfire spread}, + author={Radke, David and Hessler, Anna and Ellsworth, David}, + booktitle={Proceedings of the 28th International Joint Conference on Artificial Intelligence}, + pages={4575--4581}, + year={2019} +} + +@article{nguyen2023climax, + title={ClimaX: A foundation model for weather and climate}, + author={Nguyen, Tung and Brandstetter, Johannes and Kapoor, Aditya and Gupta, Jayesh K and Grover, Aditya}, + journal={arXiv preprint arXiv:2301.10343}, + year={2023} +} + +@article{reed2023scalemae, + title={Scale-MAE: A scale-aware masked autoencoder for multiscale geospatial representation learning}, + author={Reed, Colorado J and Gupta, Ritwik and Li, Shufan and Brockman, Sarah and Funk, Christopher and Clipp, Brian and Keutzer, Kurt and Ermon, Stefano and Salakhutdinov, Ruslan}, + journal={arXiv preprint arXiv:2212.14532}, + year={2023} +} + +@article{bi2023panguweather, + title={Pangu-Weather: A 3D high-resolution model for fast and accurate global weather forecast}, + author={Bi, Kaifeng and Xie, Lingxi and Zhang, Hengheng and Chen, Xin and Gu, Xiaotao and Tian, Qi}, + journal={Nature}, + volume={619}, + number={7970}, + pages={533--538}, + year={2023}, + publisher={Nature Publishing Group} +} + +@article{cong2022satmae, + title={SatMAE: Pre-training transformers for temporal and multi-spectral satellite imagery}, + author={Cong, Yezhen and Khanna, Samar and Meng, Chenlin and Liu, Patrick and Rozi, Erik and He, Yutong and Burke, Marshall and Lobell, David and Ermon, Stefano}, + journal={Advances in Neural Information Processing Systems}, + volume={35}, + pages={197--211}, + year={2022} +} + +@article{guo2024skysense, + title={SkySense: A multi-modal remote sensing foundation model towards universal interpretation}, + author={Guo, Xin and Lao, Jianwei and Dang, Bo and Zhang, Yingying and Yu, Lei and Zhang, Ruixiang and Zhan, Siyu and Li, Wei and Hao, Yonggang and Zhang, Shuai and others}, + journal={arXiv preprint arXiv:2403.11916}, + year={2024} +} + +@article{bodnar2025aurora, + title={Aurora: A foundation model of the atmosphere}, + author={Bodnar, Cristian and others}, + journal={arXiv preprint arXiv:2405.13063}, + year={2024} +} + +@article{schmude2024prithviwxc, + title={Prithvi WxC: Foundation model for weather and climate}, + author={Schmude, Johannes and Roy, Sujit and Trofimova, Paulina and Ramesh, Karthik and Lusch, Bethany and Kesa, Harikumar and Singh, Shraddha and Chen, Phil and Liu, Zhuohan and Parashar, Shubhankar and others}, + journal={arXiv preprint arXiv:2409.13598}, + year={2024} +} + +@article{farahmand2020fdeo, + title={Introducing spatially distributed fire danger from earth observations (FDEO) using satellite-based data in the contiguous United States}, + author={Farahmand, Alireza and Stavros, E Natasha and Reager, John T and Behrangi, Ali}, + journal={Remote Sensing}, + volume={12}, + number={8}, + pages={1252}, + year={2020}, + publisher={MDPI} +} + +@article{gilleland2009intercomparison, + title={Intercomparison of spatial forecast verification methods}, + author={Gilleland, Eric and Ahijevych, David and Brown, Barbara G and Ebert, Elizabeth E}, + journal={Weather and Forecasting}, + volume={24}, + number={5}, + pages={1416--1430}, + year={2009}, + publisher={American Meteorological Society} +} + +@article{gilleland2009spatialverification, + title = {Intercomparison of Spatial Forecast Verification Methods}, + author = {Gilleland, Eric and Ahijevych, David and Brown, Barbara G. and Casati, Barbara and Ebert, Elizabeth E.}, + journal = {Weather and Forecasting}, + volume = {24}, + number = {5}, + pages = {1416--1430}, + year = {2009}, + doi = {10.1175/2009WAF2222269.1} +} + +@article{bi2023pangu, + title = {Accurate medium-range global weather forecasting with 3D neural networks}, + author = {Bi, Kaifeng and Xie, Lingxi and Zhang, Hengheng and Chen, Xin and Gu, Xiaotao and Tian, Qi}, + journal = {Nature}, + volume = {619}, + pages = {533--538}, + year = {2023} +} + +@inproceedings{reed2023scale, + title = {Scale-{MAE}: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning}, + author = {Reed, Colorado J. and Gupta, Ritwik and Li, Shufan and Brockman, Sarah and Funk, Christopher and Clipp, Brian and Keutzer, Kurt and Candido, Salvatore and Uyttendaele, Matt and Darrell, Trevor}, + booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision}, + pages = {4088--4099}, + year = {2023} +} + +@inproceedings{mcdermott2024aurocauprc, + title = {A Closer Look at {AUROC} and {AUPRC} under Class Imbalance}, + author = {McDermott, Matthew B. and Zhang, Haoran and Hansen, Lasse Hyldig and Angelotti, Giovanni and Gallifant, Jack}, + booktitle = {Advances in Neural Information Processing Systems}, + year = {2024} +} + +@inproceedings{lacoste2023geobench, + title = {{GEO-Bench}: Toward Foundation Models for Earth Monitoring}, + author = {Lacoste, Alexandre and Lehmann, Nils and Rodriguez, Pau and Sherwin, Evan and Kerner, Hannah and L{\"u}tjens, Bj{\"o}rn and Irvin, Jeremy and Dao, David and Alemohammad, Hamed and Drouin, Alexandre and others}, + booktitle = {Advances in Neural Information Processing Systems}, + year = {2023} +} + +@article{rasp2024weatherbench2, + title = {{WeatherBench} 2: A benchmark for the next generation of data-driven global weather models}, + author = {Rasp, Stephan and Hoyer, Stephan and Merose, Alex and Langmore, Ian and Battaglia, Peter and Russell, Tyler and Sanchez-Gonzalez, Alvaro and Yang, Vivian and Carver, Rob and Agrawal, Shreya and others}, + journal = {Journal of Advances in Modeling Earth Systems}, + volume = {16}, + number = {6}, + pages = {e2023MS004019}, + year = {2024} +} + +@inproceedings{koh2021wilds, + title = {{WILDS}: A benchmark of in-the-wild distribution shifts}, + author = {Koh, Pang Wei and Sagawa, Shiori and Marklund, Henrik and Xie, Sang Michael and Zhang, Marvin and Balsubramani, Akshay and Hu, Weihua and Yasunaga, Michihiro and Phillips, Richard Lanas and Gao, Irena and others}, + booktitle = {Proceedings of the International Conference on Machine Learning}, + pages = {5637--5664}, + year = {2021} +} + +@inproceedings{yeh2021sustainbench, + title = {{SustainBench}: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning}, + author = {Yeh, Christopher and Meng, Chenlin and Wang, Sijing and Driscoll, Anne and Rozi, Erik and Liu, Peng and Lee, Jae Yong and Burke, Marshall and Lobell, David B. and Ermon, Stefano}, + booktitle = {Advances in Neural Information Processing Systems}, + year = {2021} +} + +@inproceedings{torchgeo2022, + title = {TorchGeo: Deep Learning With Geospatial Data}, + author = {Stewart, Adam J. and Robinson, Caleb and Corley, Isaac A. and Ortiz, Anthony and Lavista Ferres, Juan M. and Banerjee, Arindam}, + booktitle = {Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems}, + year = {2022} +} + +@inproceedings{schaeffer2023mirage, + title = {Are Emergent Abilities of Large Language Models a Mirage?}, + author = {Schaeffer, Rylan and Miranda, Brando and Koyejo, Sanmi}, + booktitle = {Advances in Neural Information Processing Systems}, + year = {2023} +} + +@inproceedings{luth2023activelearning, + title = {Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment}, + author = {L{\"u}th, Carsten T. and Bungert, Till J. and Klein, Lukas and J{\"a}ger, Paul F.}, + booktitle = {Advances in Neural Information Processing Systems}, + year = {2023} +} + +@inproceedings{traub2024selectiveclassification, + title = {Overcoming Common Flaws in the Evaluation of Selective Classification Systems}, + author = {Traub, Jeremias and Bungert, Till J. and L{\"u}th, Carsten T. and Baumgartner, Michael and Maier-Hein, Klaus H. and Maier-Hein, Lena and J{\"a}ger, Paul F.}, + booktitle = {Advances in Neural Information Processing Systems}, + year = {2024} +} + + +# new---4.30 +@article{pathak2024stormcast, + title={Kilometer-scale convection-allowing model emulation using generative diffusion modeling}, + author={Pathak, Jaideep and Cohen, Yair and Garg, Piyush and Harrington, Peter and Brenowitz, Noah and Durran, Dale and Mardani, Morteza and Vahdat, Arash and Xu, Shaoming and Kashinath, Karthik and others}, + journal={Science Advances}, + volume={12}, + number={5}, + pages={eadv0423}, + year={2026}, + publisher={American Association for the Advancement of Science} +} + +@article{brown2025alphaearth, + title={Alphaearth foundations: An embedding field model for accurate and efficient global mapping from sparse label data}, + author={Brown, Christopher F and Kazmierski, Michal R and Pasquarella, Valerie J and Rucklidge, William J and Samsikova, Masha and Zhang, Chenhui and Shelhamer, Evan and Lahera, Estefania and Wiles, Olivia and Ilyushchenko, Simon and others}, + journal={arXiv preprint arXiv:2507.22291}, + year={2025} +} + +@article{weyn2020dlwp, + title={Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere}, + author={Weyn, Jonathan A and Durran, Dale R and Caruana, Rich}, + journal={Journal of Advances in Modeling Earth Systems}, + volume={12}, + number={9}, + pages={e2020MS002109}, + year={2020}, + publisher={Wiley Online Library} +} + +@article{pathak2022fourcastnet, + title={Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators}, + author={Pathak, Jaideep and Subramanian, Shashank and Harrington, Peter and Raja, Sanjeev and Chattopadhyay, Ashesh and Mardani, Morteza and Kurth, Thorsten and Hall, David and Li, Zongyi and Azizzadenesheli, Kamyar and others}, + journal={arXiv preprint arXiv:2202.11214}, + year={2022} +} + +@article{chen2023fengwu, + title={Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead}, + author={Chen, Kang and Han, Tao and Gong, Junchao and Bai, Lei and Ling, Fenghua and Luo, Jing-Jia and Chen, Xi and Ma, Leiming and Zhang, Tianning and Su, Rui and others}, + journal={arXiv preprint arXiv:2304.02948}, + year={2023} +} + +@article{chen2023fuxi, + title={FuXi: A cascade machine learning forecasting system for 15-day global weather forecast}, + author={Chen, Lei and Zhong, Xiaohui and Zhang, Feng and Cheng, Yuan and Xu, Yinghui and Qi, Yuan and Li, Hao}, + journal={npj climate and atmospheric science}, + volume={6}, + number={1}, + pages={190}, + year={2023}, + publisher={Nature Publishing Group UK London} +} + +@article{marsocci2024pangaea, + title={Pangaea: A global and inclusive benchmark for geospatial foundation models}, + author={Marsocci, Valerio and Jia, Yuru and Bellier, Georges Le and Kerekes, David and Zeng, Liang and Hafner, Sebastian and Gerard, Sebastian and Brune, Eric and Yadav, Ritu and Shibli, Ali and others}, + journal={arXiv preprint arXiv:2412.04204}, + year={2024} +} + +@article{gerard2023wildfirespreadts, + title={Wildfirespreadts: A dataset of multi-modal time series for wildfire spread prediction}, + author={Gerard, Sebastian and Zhao, Yu and Sullivan, Josephine}, + journal={Advances in Neural Information Processing Systems}, + volume={36}, + pages={74515--74529}, + year={2023} +} + + diff --git a/paper/sections/0_abstract.tex b/paper/sections/0_abstract.tex new file mode 100644 index 0000000000000000000000000000000000000000..29fe695c34d16dbb2068025d889271d64fc8d883 --- /dev/null +++ b/paper/sections/0_abstract.tex @@ -0,0 +1,4 @@ +\begin{abstract} +Wildfire prediction is important for early warning and resource allocation, yet existing Earth foundation models (Earth FMs) are pretrained for general atmospheric and geophysical objectives rather than wildfire forecasting. To address this gap, we introduce \ourfm, the first foundation model pretrained specifically for wildfire prediction using weather, active-fire observations, topography, vegetation, and static environmental data. However, introducing a domain-specific backbone alone does not solve the evaluation problem: wildfire events are sparse in space and time, making transfer conclusions highly sensitive to matching rules and evaluation settings. +To address this problem, we introduce a fixed-contract evaluation framework with two controlled checks: a fixed-output check for matching-rule effects and a fixed-feature check for head-selection effects. Under matched contracts, we compare \ourfm\ with ten Earth-FM baselines across occupancy, spread, retrieval, and regression tasks. Our results show that wildfire transfer conclusions depend strongly on evaluation design and task formulation. We hope this framework and \ourfm\ provide a foundation for future wildfire-specific Earth-FM research and benchmarking. Our code is available at https://anonymous.4open.science/r/Wildfire-fm-evaluation-contracts-5AE9/. +\end{abstract} \ No newline at end of file diff --git a/paper/sections/1_intro.tex b/paper/sections/1_intro.tex new file mode 100644 index 0000000000000000000000000000000000000000..7587beb83277dfa09b78f74571efcfe7e78e1fb8 --- /dev/null +++ b/paper/sections/1_intro.tex @@ -0,0 +1,77 @@ +\section{Introduction} + +Wildfire prediction is critical for early warning and resource allocation in disaster response~\cite{goldammer1999early, farahmand2020fdeo}. As extreme fire events grow more frequent and severe, accurate forecasting of wildfire occurrence and spread is becoming increasingly important~\cite{pickell2017early, kotroni2020disarm}. Recent Earth foundation models (Earth FMs), pretrained on large-scale atmospheric and geophysical data~\cite{bodnar2025aurora, schmude2024prithviwxc, nguyen2023climax}, provide transferable representations for Earth-system dynamics and have shown strong performance across weather and remote-sensing tasks. However, wildfire dynamics depend on complex interactions among weather, vegetation, topography, fuel conditions, and active-fire behavior, which are not explicitly modeled during pretraining in existing general-purpose Earth FMs. This mismatch raises a natural question: can representations learned for general atmospheric or geophysical objectives transfer reliably to wildfire forecasting, and how should we measure that transfer? + +Answering this question requires solving two intertwined problems. The first is that existing Earth FMs are not pretrained specifically for wildfire dynamics, but instead adapted to wildfire tasks after general-purpose pretraining. To address this limitation, we introduce \textbf{\ourfm}, the first foundation model pretrained specifically for wildfire prediction using fire-relevant multimodal data, including regional weather dynamics, active-fire observations, topography, vegetation, and static environmental context. By incorporating wildfire-specific signals directly during pretraining, \ourfm\ learns representations aligned with the physical processes underlying fire behavior rather than relying on transfer from general atmospheric or geophysical objectives. + +The second problem is evaluation: even with a domain-specific model, reliably comparing \ourfm\ against transferred general-purpose Earth FMs remains difficult. Wildfire events are sparse in space and time~\cite{ebert2009neighborhood, gilleland2009intercomparison}, making transfer conclusions highly sensitive to three sources of evaluation variability. +\textit{First,} matching rules determine what counts as a correct prediction. Early warning systems tolerate spatial offsets that post-fire damage assessment cannot, so different matching rules can produce substantially different F1 scores from the same model outputs~\cite{ebert2009neighborhood, gilleland2009intercomparison}. +\textit{Second,} head-selection metrics determine which lightweight adapter is chosen on top of a frozen representation. Ranking metrics such as PR-AUC and decision metrics such as F1 can favor different heads from the same frozen features~\cite{mcdermott2024aurocauprc, traub2024selectiveclassification}. +\textit{Third,} wildfire task forms operate over different prediction units and metric families. Occupancy prediction, spread forecasting, burned-area regression, and analog retrieval therefore produce scores that are not directly comparable even under the same backbone~\cite{schaeffer2023mirage, gerard2023wildfirespreadts}. +Related protocol sensitivity has also been observed in active learning~\cite{luth2023activelearning} and selective classification~\cite{traub2024selectiveclassification}. We show that these effects become particularly severe in wildfire transfer evaluation, where sparse events and heterogeneous task forms amplify evaluation instability~\cite{marsocci2024pangaea}. + +\begin{figure*} + \centering + \includegraphics[width=0.96\linewidth]{figures/overview_wildfire.pdf} + \caption{Overview of \textbf{\ourfm} and \textbf{Evaluation Protocol} in this paper.} + \vspace{-8mm} + \label{fig:overview} +\end{figure*} + + +This evaluation instability makes reliable comparison between +\ourfm\ and existing Earth FMs fundamentally difficult. Standard +geospatial benchmarks such as GEO-Bench~\cite{lacoste2023geobench}, +WeatherBench2~\cite{rasp2024weatherbench2}, WILDS~\cite{koh2021wilds}, +SustainBench~\cite{yeh2021sustainbench}, and +TorchGeo~\cite{torchgeo2022} standardize datasets, splits, and +metrics, but do so for tasks with dense, balanced labels where +matching-rule sensitivity is not a primary concern. Wildfire +studies such as FireCast~\cite{radke2019firecast} and Next Day +Wildfire Spread~\cite{huot2022nextday} apply the same +report-and-compare paradigm directly to sparse fire events +without controlling for matching-rule choice, head-selection +metric, or task-form comparability, the three sources of +instability identified above. As a result, scores reported under +different implicit protocol choices are not directly comparable, +even when the underlying predictions are identical. + +Based on the \textit{limitations of prior work}, our contributions are as follows (see Figure~\ref{fig:overview}). +\begin{itemize} + +\item \textbf{Wildfire-specific foundation model.} +We introduce \ourfm, the first foundation model pretrained +specifically for wildfire prediction using multimodal wildfire +data spanning weather, active-fire observations, topography, +vegetation, and static environmental context. Unlike general +Earth FMs adapted after pretraining, \ourfm learns wildfire +representations directly from fire-relevant processes during +pretraining. + +\item \textbf{Fixed-contract evaluation framework.} +We formulate wildfire Earth-FM transfer as a fixed-contract +evaluation problem, defining a contract +$\mathcal{C} = (\mathcal{T}, M, \Lambda, \Omega, \mathcal{A})$ +that specifies the task, metric, matching rule, evaluation +scope, and lightweight-head family before comparison. We +introduce two controlled checks: a \emph{fixed-output check} +for matching-rule effects and a \emph{fixed-feature check} +for head-selection effects, enabling evaluation artifacts to +be separated from representation quality. + +\item \textbf{Systematic wildfire transfer benchmark.} +Under fixed contracts, we compare \ourfm\ against ten +general-purpose Earth FMs across six wildfire task forms. +Our results show that wildfire transfer conclusions are +highly sensitive to evaluation design and strongly +task-dependent across occupancy, spread, retrieval, and +regression settings. + +\end{itemize} + +% \begin{figure*} +% \centering +% \includegraphics[width=\linewidth]{figures/overview_wildfire.pdf} +% \caption{Overview of \textbf{\ourfm} and \textbf{Evaluation Protocol} in this paper.} +% \label{fig:overview} +% \end{figure*} diff --git a/paper/sections/2_backbone.tex b/paper/sections/2_backbone.tex new file mode 100644 index 0000000000000000000000000000000000000000..c51fcea4200de4e19176997fc90258012039afe5 --- /dev/null +++ b/paper/sections/2_backbone.tex @@ -0,0 +1,39 @@ +\section{\ourfm Reference Backbone} +\label{Reference_backbone} + +\ourfm is a wildfire-specialized regional backbone trained on fire-relevant multimodal data for wildfire prediction. Existing general-purpose Earth FMs are pretrained for atmospheric and geophysical objectives~\cite{lam2023graphcast}, or for remote-sensing objectives~\cite{reed2023scalemae}, so wildfire-relevant information enters only indirectly through those objectives. In contrast, \ourfm is trained with weather, active-fire observations, topography, vegetation, and static environmental context, so its representation is learned from inputs tied directly to wildfire behavior. This design makes \ourfm a strong wildfire-specific backbone whose features are shaped by signals directly relevant to fire occurrence and spread. +It provides a task-aligned regional model trained directly for wildfire prediction. +It also serves as an empirical anchor for interpreting how transferred Earth FMs behave under matched evaluation contracts. This section describes the data resources and training strategy used to build \ourfm as an in-domain reference backbone. The fixed-contract protocol used to compare it with transferred Earth FMs is defined separately in Section~\ref{sec:eval}. + +\subsection{Data Resources} +We group the resources by their role in the study: dynamic weather inputs, occupancy supervision, static context, and event-level resources for supporting tasks. Source and terms-of-use notes for the external data and model assets used in this study are summarized in Appendix Table~\ref{tab:external_assets_licenses}. + + +\noindent\textbf{Dynamic weather inputs.} +The weather inputs come from a California regional dataset built from NOAA High-Resolution Rapid Refresh (HRRR) fields~\cite{noaa_hrrr_ncei,noaa_hrrr_emc}. The data are placed on a projected 5 km grid in EPSG:5070. Each time map uses weather fields every 6 hours and predicts wildfire occupancy at a 12-hour lead. The variables include near-surface temperature and dew point, wind, CAPE, surface pressure, boundary-layer height, visibility, precipitation rate, and accumulated precipitation. + +\noindent\textbf{Occupancy supervision.} +Wildfire supervision comes from NASA FIRMS active-fire detections~\cite{nasa_firms}. The detections are mapped to the same grid as the weather fields. \ourfm is trained on gridded occupancy labels derived from these detections. This defines the occupancy target used by the reference backbone throughout the primary experiments. + +\noindent\textbf{Static context.} +Static context describes landscape and exposure factors that do not change at the weather time step. These variables are LANDFIRE fire-behavior fuel model~\cite{landfire_fbfm40}, LANDFIRE canopy cover~\cite{landfire_canopy_cover}, Wildfire Risk to Communities housing-unit density~\cite{usfs_wrc_housing_density}, and LandScan population~\cite{ornl_landscan_2024}. Together with validity masks for the weather and static fields, the occupancy input has 16 channels: 10 weather fields, two validity masks, and four static layers for regional fire prediction. + +\noindent\textbf{Event-level resources.} +Event-level resources are used for supporting burned-area and analog tasks, not as occupancy labels for \ourfm. These resources include WFIGS incident and perimeter attributes~\cite{nifc_wfigs_perimeters} and MTBS burned-area and burn-severity records~\cite{mtbs_usgs_2025}. They provide event-scale outcomes and incident metadata for supporting tasks in the experiments and appendix analyses. + + + +\subsection{Training Strategy} + +\noindent\textbf{Model and data split.} +\ourfm uses a compact U-Net~\cite{ronneberger2015unet} that maps gridded weather and static inputs to wildfire predictions. +Its primary output is fire occupancy on the common spatial grid. +Data are split by time: June--August 2024 for training, September 2024 for validation, and October 2024 for testing. +This yields 368 training time maps, 120 validation time maps, and 120 test time maps. +Temporal splitting keeps later fire outcomes out of earlier training periods. + +\noindent\textbf{Fire-aware tile training.} +Training is performed on 32$\times$32 tiles sampled from the time maps. The tiles include fire-centered regions and non-fire context, so the model sees both sparse fire labels and surrounding background conditions. This sampling reduces the dominance of empty cells without removing non-fire examples from the training distribution. Class-weighted binary cross-entropy is used for the primary occupancy target to further balance sparse positives. + +\noindent\textbf{Spatial-support training objective.} +Wildfire labels can shift by a few grid cells because detections, weather fields, and static layers are aligned on a common grid. To reduce sensitivity to these small displacements during training, the occupancy target is dilated by two grid cells. An auxiliary spatial-support output is trained for the same neighborhood alongside the primary occupancy output. At test time, \ourfm is scored under the same task-specific evaluation contracts as the transferred Earth-FM backbones in Section~\ref{sec:eval}, ensuring matched comparison conditions. \ No newline at end of file diff --git a/paper/sections/3_prelim.tex b/paper/sections/3_prelim.tex new file mode 100644 index 0000000000000000000000000000000000000000..d1f057562abfd6869380dc5e03bf36c94f061863 --- /dev/null +++ b/paper/sections/3_prelim.tex @@ -0,0 +1,84 @@ +\section{Evaluation Design} +\label{sec:eval} + +% Section~\ref{Reference_backbone} establishes \ourfm\ as a wildfire-specialized backbone. +% This section formalizes an evaluation contract and introduces two controlled checks to isolate evaluation effects. + +\subsection{Wildfire Output Records and Fire Sets} + +\paragraph{Output record.} +A wildfire prediction model produces scores over spatial units and forecast times, which are compared against observed fire activity to compute performance. We formalize this comparison as a \emph{wildfire output record} $\mathcal{O} = (S, Y)$, where the score field $S = \{s_{i,t}\}$ contains model scores over spatial units $i$ and times $t$, and the label field $Y = \{y_{i,t}\}$ contains the corresponding observations. For occupancy tasks, $y_{i,t} \in \{0,1\}$ indicates whether fire is observed at $(i,t)$. +\vspace{-0.5em} +\paragraph{Predicted and observed fire sets.} +To evaluate $\mathcal{O}$, the score field is thresholded at $\tau$ to produce a predicted fire set $\hat{P}_\tau = \{(i,t) : s_{i,t} \geq \tau\}$, while the observed fire set is $P = \{(i,t) : y_{i,t} = 1\}$. The pair $(\hat{P}_\tau, P)$ is evaluated under a matching rule. Given a matching rule, true positives (TP), false positives (FP), and false negatives (FN) are computed from matched and unmatched elements, and the decision F1 score is $\text{F1} = 2\text{TP}/(2\text{TP} + \text{FP} + \text{FN})$. The same $(\hat{P}_\tau, P)$ can yield different TP, FP, and FN counts under different matching rules without changing model outputs, motivating the fixed-output check in Section~\ref{sec:checks}. +\vspace{-0.5em} +\paragraph{Matching rules.} +A matching rule specifies when a predicted unit-time pair in $\hat{P}_\tau$ is considered a match to an observed pair in $P$~\cite{ebert2009neighborhood, gilleland2009intercomparison}. Because wildfire applications tolerate different levels of spatial and temporal error, we define three matching rules for occupancy outputs. \textit{(1) Exact matching}: requires agreement in both spatial unit and forecast time. \textit{(2) Tolerated matching}: accepts predictions within a fixed spatial or temporal neighborhood defined by the evaluation contract $\mathcal{C}$. \textit{(3) Union matching}: accepts predictions satisfying either exact or tolerated matching. +% \begin{itemize} +% \item \textbf{Exact matching}: requires agreement in both spatial unit and forecast time. +% \item \textbf{Tolerated matching}: accepts predictions within a fixed spatial or temporal neighborhood defined by the evaluation contract $\mathcal{C}$. +% \item \textbf{Union matching}: accepts predictions satisfying either exact or tolerated matching. +% \end{itemize} +% \vspace{-0.5em} +% +Figure~\ref{fig:toy_occupancy_contract} illustrates these rules for a fixed output. Because the output record is held constant, any score difference is attributed solely to the matching rule. + +\begin{figure} + \centering + \vspace{-2mm} + \includegraphics[width=0.8\linewidth]{figures/matching.pdf} + \vspace{-2mm} + \caption{Matching rules for one fixed occupancy output. + (a) Exact matching counts only same-time, same-cell overlap. + (b) Tolerated matching accepts bounded spatial or temporal offsets. + (c) The union reading counts matches accepted by either rule.} + \vspace{-5mm} + \label{fig:toy_occupancy_contract} +\end{figure} + +\subsection{Evaluation Contract} + +A wildfire transfer score depends not only on the model, but also on the evaluation choices used to compute it~\cite{luth2023activelearning}. Changing the matching rule $\Lambda$, metric $M$, or evaluation scope $\Omega$ changes what the score measures even when model outputs are fixed. + +We define an \emph{evaluation contract} as the tuple +$\mathcal{C} = (\mathcal{T}, M, \Lambda, \Omega, \mathcal{A})$, +where $\mathcal{T}$ denotes the task, $M$ the metric, +$\Lambda$ the matching rule, $\Omega$ the evaluation scope, +and $\mathcal{A}$ the allowed lightweight-head family. +Two transfer scores are comparable only when all five +components are identical. The evaluation scope $\Omega$ is particularly important in wildfire settings. A global scope evaluates the full spatial domain, including many fire-inactive regions that can mask differences between models. A fire-prone scope restricts evaluation to regions with higher historical fire activity. We report both scopes separately rather than averaging across them. Fixed matching-rule, task-form, and scope parameters are reported in Appendix Tables~\ref{tab:app_matching_rule_params}, \ref{tab:app_contract_params_full}, and~\ref{tab:app_scope_params}. + + +\subsection{Task-Form Contracts} +\label{sec:taskforms} +Contract components depend on task form. We distinguish \emph{primary} and \emph{supporting} tasks based on whether they directly evaluate wildfire decisions. Occupancy and fire spread are primary tasks because they evaluate spatial fire outputs under matching or overlap rules. +Retrieval, burned-area regression, smoke PM$_{2.5}$, and extreme heat are supporting tasks because they use different prediction units and metric families. Their results provide complementary evidence rather than direct substitutes for occupancy and spread evaluation~\cite{schaeffer2023mirage}. + +For primary tasks, multiple metrics are reported for the same output under different contracts. For occupancy, exact F1 requires same-cell same-time agreement, tolerated F1 accepts predictions within a spatial or temporal neighborhood, and union F1 accepts predictions satisfying either rule. For fire spread, exact F1 evaluates raster-cell agreement, spatial F1 evaluates region overlap between $\hat{B}$ and $B$~\cite{gilleland2009intercomparison}, and AP summarizes ranking quality across thresholds. These metrics are reported separately rather than aggregated because they measure different aspects of the same prediction task. Figure~\ref{fig:task_contract_tiles} summarizes the contract map across all six task forms. + +\subsection{Controlled Checks} +\label{sec:checks} +\begin{wrapfigure}[19]{r}{0.52\textwidth} +\vspace{-2em} +\centering +\includegraphics[width=\linewidth]{figures/fig_task_contract_tiles.pdf} +\vspace{-1.5em} +\caption{ +Evaluation contract map for the six fixed-contract tasks. +Yellow boxes denote \textcolor[RGB]{255,193,7}{\textbf{primary}} decision tasks; purple boxes denote \textcolor[RGB]{148,103,189}{\textbf{supporting}} tasks. +} +\label{fig:task_contract_tiles} +\vspace{-0.8em} +\end{wrapfigure} +We isolate the two instability sources with two checks. +Each check fixes all contract components except one, so any difference is attributed solely to that component. + +\paragraph{Fixed-output check.} +The fixed-output check isolates matching-rule effects by holding the output record $\mathcal{O} = (S, Y)$ and all other contract components fixed while varying only $\Lambda$. For the same occupancy record, we compute F1 under exact, tolerated, and union matching. Any score difference is therefore attributed solely to the matching rule. If matching rules alone shift F1 by tens of percentage points on the same output, then comparing models under different $\Lambda$ conflates model quality with evaluation design. + +\paragraph{Fixed-feature check and selection regret.} +The fixed-feature check isolates head-selection effects by holding the frozen feature source, $\mathcal{T}$, $\Omega$, $\Lambda$, and candidate head family $\mathcal{H} \subseteq \mathcal{A}$ fixed while varying only the selection metric. Let $R(h)$ denote the ranking score of head $h$ and $D(h)$ its decision score. Ranking-based selection chooses $h_R = \arg\max_{h \in \mathcal{H}} R(h)$, while decision-based selection chooses $h_D = \arg\max_{h \in \mathcal{H}} D(h)$. We define \emph{selection regret} as the decision-score gap incurred by using a ranking metric as a proxy for a decision metric during head selection: $\delta = D(h_D) - D(h_R) \geq 0$ ~\cite{mcdermott2024aurocauprc, traub2024selectiveclassification}. When $\delta > 0$, the ranking metric selects a head with lower decision performance under the same frozen representation, indicating that the observed gap arises from metric misalignment rather than from representation quality.The head family used in fixed-feature comparisons is summarized in Appendix Table~\ref{tab:app_head_architectures}. + + +\paragraph{Fixed-contract transfer comparison.} +After the controlled checks establish that matching-rule and selection-metric effects are non-trivial, Earth-FM backbones are evaluated under a shared contract $\mathcal{C}$. Entries are compared only when they satisfy the same $(\mathcal{T}, M, \Lambda, \Omega, \mathcal{A})$ tuple. Supporting tasks test whether occupancy and spread patterns generalize across task forms and provide additional evidence when transfer orderings are preserved. diff --git a/paper/sections/4_experiments.tex b/paper/sections/4_experiments.tex new file mode 100644 index 0000000000000000000000000000000000000000..972f3e6e698b8a9c74512a77854a4500fc13e925 --- /dev/null +++ b/paper/sections/4_experiments.tex @@ -0,0 +1,435 @@ +\section{Experiments} +\label{sec:experiments} + +We address three research questions under the fixed-contract framework defined in Section~\ref{sec:eval}. \textbf{RQ1:} Under fixed outputs, does the matching rule determine whether a wildfire model appears usable? +\textbf{RQ2:} Under fixed features, does ranking-based head selection lose decision performance? +\textbf{RQ3:} Under fixed task contracts, do model comparisons remain consistent across task forms? +\vspace{-0.5em} +\subsection{Experimental Setup} +\paragraph{Task instances.} +We instantiate the six task-form contracts defined in Section~\ref{sec:taskforms}. +Occupancy and fire spread serve as primary tasks because they evaluate spatial fire outputs under matching or overlap rules and align with the decision structure of early warning systems~\cite{goldammer1999early, farahmand2020fdeo}. +The four supporting tasks, \textit{final burned area, analog retrieval, smoke PM$_{2.5}$, and extreme heat}, use different prediction units and metric families; their results bound rather than replace primary decision evidence. + +\paragraph{Compared backbones.} +The frozen Earth-FM comparator set includes Prithvi-WxC~\cite{schmude2024prithviwxc}, Aurora~\cite{bodnar2025aurora}, ClimaX~\cite{nguyen2023climax}, StormCast~\cite{pathak2024stormcast}, DLWP~\cite{weyn2020dlwp}, FCN~\cite{pathak2022fourcastnet}, FengWu~\cite{chen2023fengwu}, FuXi~\cite{chen2023fuxi}, Pangu-Weather~\cite{bi2023panguweather}, and AlphaEarth~\cite{brown2025alphaearth}. +\ourfm\ serves as the wildfire-specialized reference backbone. + +\paragraph{Protocol.} +For each comparison, the contract $\mathcal{C} = (\mathcal{T}, M, \Lambda, \Omega, \mathcal{A})$ is fixed before reporting test scores. +Thresholds and morphology parameters are selected on validation data and held fixed at test time. +Stochastic components are evaluated over five seeds and reported as mean $\pm$ standard deviation; deterministic fixed-output checks have zero seed variance by construction. +Entries outside a fixed contract are omitted from main tables and documented in the appendix. +For error metrics lower is better ($\downarrow$); for F1, AP, nDCG, and correlation metrics higher is better ($\uparrow$). +Appendix Table~\ref{tab:app_seed_robustness} summarizes the seed-level checks behind the reported mean-with-std convention. + +\subsection{Matching-Rule Sensitivity Under Fixed Output (RQ1)} +\label{sec:rq1} + +To answer RQ1, we conduct a fixed-output check on occupancy and fire spread tasks, holding the score field $S$, label field $Y$, threshold, and all other operating choices fixed while varying only the matching rule $\Lambda$ across exact, tolerated, and union settings. Occupancy results are reported in Figure~\ref{fig:fireprone_contract_progression} under both global and fire-prone scopes. The same progression is applied to fire spread outputs. Complete occupancy sweeps and predicted-positive rates are reported in Appendix Tables~\ref{tab:fireprone_contract_progression} and~\ref{tab:app_occupancy_ppr_scope}. + +\begin{wrapfigure}[21]{r}{0.50\textwidth} +\centering +\vspace{-3mm} +\includegraphics[width=\linewidth]{figures/fig_primary_rank_change_map.pdf} +\caption{\textbf{Primary-task rank changes (RQ1).} +Cells show rank before\(\rightarrow\)after. Green/red/gray mark moving up/down/no change; darker green or red marks a larger move. Following Section~\ref{sec:taskforms}, Ex/Tol/Un are occupancy exact, tolerated, and union matching; Sp is spread spatial-overlap $F_1$.} +\label{fig:primary_ranking} +\vspace{-0.8em} +\end{wrapfigure} +Because both tasks involve spatially sparse targets, fire-active cells for occupancy, burned raster patches for spread, the operational assumptions encoded in $\Lambda$ directly govern what the model is being asked to get right, making matching-rule choice a substantive experimental setting rather than a post hoc evaluation detail. +The fixed-output results reveal a pattern that goes beyond score differences: matching-rule choice determines whether a model appears viable for wildfire decision tasks at all. Under exact matching, which requires same-cell same-time agreement, the majority of frozen Earth-FM backbones produce F1 scores that are effectively near zero, rendering them indistinguishable from an uninformative baseline and suggesting they have no practical utility for the task. As the matching rule relaxes to tolerated and then union matching, both of which reflect operationally realistic assumptions for early warning systems, where a prediction displaced by a few grid cells still triggers the correct response, the same frozen representations recover substantial decision performance, with several backbones crossing from near-zero to practically meaningful F1 levels. This transition is not a marginal score improvement: it is a qualitative change in whether a model can be considered usable. The same pattern holds for fire spread under region-level matching relaxation, where strict raster-cell agreement again suppresses performance for most backbones while spatial tolerance restores it. The implications for prior wildfire transfer claims are significant: papers that report model performance under a single implicit matching rule, which is common practice given that sparse decision targets almost always require some form of tolerance~\cite{ebert2009neighborhood, gilleland2009intercomparison}, may be drawing viability conclusions that are entirely dependent on an undisclosed protocol choice. A model claimed to perform well under one tolerance assumption may be completely unusable under a stricter one, and vice versa. Matching rule cannot be treated as an evaluation detail; it is an experimental setting that must be fixed, reported, and justified as part of any wildfire transfer claim. Additional spread AP values under fixed scopes are reported in Appendix Table~\ref{tab:app_spread_ap_by_scope}. + + +\begin{table}[t] +\centering +\small +\setlength{\tabcolsep}{4pt} +\renewcommand{\arraystretch}{1.20} +\caption{% + \textbf{Primary fixed-contract transfer results (RQ1).} + Occupancy metrics: exact, tolerated, union $F_1$ (\%). + Fire spread metrics: exact $F_1$ and spatial $F_1$ (\%). + Each block fixes $\mathcal{T}$, $\Lambda$, $\Omega$, and $\mathcal{A}$. + Upward arrows indicate that larger values are better. + \textbf{Bold} marks the best value per metric. \textbf{Tol.} = Tolerated +} +\label{tab:primary_results} +\setlength{\arrayrulewidth}{0.4pt} +\resizebox{\textwidth}{!}{% +\begin{tabular}{lccccc} +\toprule +& \multicolumn{3}{c}{\textbf{Occupancy}} +& \multicolumn{2}{c}{\textbf{Fire spread}} \\ +\cmidrule(lr){2-4}\cmidrule(lr){5-6} +\textbf{Comparator} +& \textbf{Exact $F_1\uparrow$} & \textbf{Tol.\ $F_1\uparrow$} & \textbf{Union $F_1\uparrow$} +& \textbf{Exact $F_1\uparrow$} & \textbf{Spatial $F_1\uparrow$} \\ +\midrule +\ourfm\ +& \ms{0.4546}{0.1412} +& \ms{29.7484}{1.2868} +& \ms{59.0656}{2.7372} +& \ensuremath{\mathbf{37.6700}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.9800}}} +& \ensuremath{\mathbf{80.9700}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{2.0200}}} \\ +\midrule +Prithvi-WxC +& \ms{0.0552}{0.0039} & \ms{7.1649}{0.6557} & \ms{20.1853}{1.8299} +& \ms{22.3500}{3.4500} & \ms{65.2600}{1.0700} \\ +Aurora +& \ms{0.0656}{0.0094} & \ms{8.5009}{1.9594} & \ms{23.1037}{4.9418} +& \ms{30.8757}{0.1343} & \ms{71.7329}{0.0141} \\ +ClimaX +& \ms{0.3480}{0.0754} +& \ensuremath{\mathbf{29.7535}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{3.6073}}} +& \ensuremath{\mathbf{60.1506}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{7.5865}}} +& \ms{27.9853}{2.0532} & \ms{69.0634}{2.3832} \\ +StormCast +& \ms{0.0626}{0.0119} & \ms{8.1951}{2.1895} & \ms{22.3817}{5.4294} +& \ms{14.8387}{7.5791} & \ms{55.7568}{21.3003} \\ +DLWP +& \ms{0.1693}{0.0419} & \ms{14.9148}{3.2446} & \ms{28.1901}{6.9658} +& \ms{5.9335}{10.0712} & \ms{22.8587}{22.3750} \\ +FCN +& \ms{0.2829}{0.0839} & \ms{19.5061}{3.3412} & \ms{40.0604}{9.3701} +& \ms{3.1798}{2.6598} & \ms{15.6203}{12.4531} \\ +FengWu +& \ms{0.2613}{0.0757} & \ms{12.0050}{6.0239} & \ms{24.1022}{13.6293} +& \ms{5.5189}{9.0883} & \ms{18.4774}{22.4703} \\ +FuXi +& \ms{0.3774}{0.1212} & \ms{21.0323}{4.8211} & \ms{37.2888}{9.4470} +& \ms{19.9909}{2.1364} & \ms{56.1826}{3.0412} \\ +Pangu-Weather +& \ms{0.2755}{0.1089} & \ms{17.0909}{4.0477} & \ms{35.6386}{9.0327} +& \ms{11.2583}{11.0719} & \ms{32.5081}{25.4969} \\ +AlphaEarth +& \ensuremath{\mathbf{2.0606}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.4404}}} +& \ms{29.4476}{6.0064} & \ms{37.4286}{9.9458} +& \ms{11.0995}{3.6088} & \ms{32.8316}{7.4634} \\ +\bottomrule +\end{tabular} +} +\end{table} + + +% \begin{figure}[H] +% \centering +% \includegraphics[width=\textwidth]{figures/fig_fireprone_contract_progression_compact.pdf} +% \caption{ +% \textbf{Matching-rule sensitivity in fire-prone occupancy (RQ1).} +% Each row holds the score field \(S\), label field \(Y\), threshold, and \(\Omega\) fixed, and changes only \(\Lambda\). +% Legend: \textcolor[HTML]{17375E}{$\blacksquare$} strict \(F_1\), +% \textcolor[HTML]{4F8DCC}{$\blacksquare$} added \(F_1\) from spatial tolerance, +% \textcolor[HTML]{BFD7F0}{$\blacksquare$} added \(F_1\) from union matching, +% red outline \ourfm, and dashed line original weather FMs vs.\ added baselines. +% The horizontal axis is \(F_1\) in percent. +% } +% \label{fig:fireprone_contract_progression} +% \end{figure} +\begin{wrapfigure}[14]{r}{0.50\textwidth} + \centering + \vspace{-1em} \includegraphics[width=0.50\textwidth]{figures/fig_selection_regret_scatter.pdf} +\caption{\textbf{Head-selection regret under fixed features (RQ2).} +Each point is one backbone; selection regret \(\delta\) follows Section~\ref{sec:checks} under global-scope union-\(F_1\).} + + \label{fig:selection_regret_diagnostic} + \vspace{-1.2em} +\end{wrapfigure} + +% \begin{wrapfigure}[17]{r}{0.46\textwidth} +% \vspace{-0.4em} +% \centering +% \resizebox{\linewidth}{!}{% +% \begin{tikzpicture}[x=1cm,y=1cm] +% \footnotesize +% \draw[black!12, line width=0.35pt] (2.450,-0.350) -- (2.450,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (2.450,-0.410) {-20}; +% \draw[black!12, line width=0.35pt] (3.243,-0.350) -- (3.243,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (3.243,-0.410) {-10}; +% \draw[wfgray, line width=0.55pt] (4.036,-0.350) -- (4.036,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (4.036,-0.410) {0}; +% \draw[black!12, line width=0.35pt] (4.829,-0.350) -- (4.829,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (4.829,-0.410) {10}; +% \draw[black!12, line width=0.35pt] (5.621,-0.350) -- (5.621,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (5.621,-0.410) {20}; +% \draw[black!12, line width=0.35pt] (6.414,-0.350) -- (6.414,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (6.414,-0.410) {30}; +% \draw[black!12, line width=0.35pt] (7.207,-0.350) -- (7.207,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (7.207,-0.410) {40}; +% \draw[black!12, line width=0.35pt] (8.000,-0.350) -- (8.000,4.530); +% \node[anchor=north, font=\scriptsize, text=black!70] at (8.000,-0.410) {50}; +% \draw[black!45, line width=0.4pt] (2.450,-0.350) -- (8.000,-0.350); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,4.350) {\textcolor{wfblue}{\textbf{\ourfm}}}; +% \draw[wfslate, line width=0.72pt] (4.030,4.220) -- (5.212,4.220); +% \draw[wfslate, line width=0.72pt] (4.030,4.185) -- (4.030,4.255); +% \draw[wfslate, line width=0.72pt] (5.212,4.185) -- (5.212,4.255); +% \filldraw[wfslate] (4.621,4.220) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.051,4.480) -- (4.487,4.480); +% \draw[wforange, line width=0.72pt] (4.051,4.445) -- (4.051,4.515); +% \draw[wforange, line width=0.72pt] (4.487,4.445) -- (4.487,4.515); +% \filldraw[wforange] (4.224,4.435) rectangle (4.314,4.525); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,3.940) {Prithvi-WxC}; +% \draw[wfslate, line width=0.72pt] (4.036,3.810) -- (4.036,3.810); +% \draw[wfslate, line width=0.72pt] (4.036,3.775) -- (4.036,3.845); +% \draw[wfslate, line width=0.72pt] (4.036,3.775) -- (4.036,3.845); +% \filldraw[wfslate] (4.036,3.810) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.036,4.070) -- (4.036,4.070); +% \draw[wforange, line width=0.72pt] (4.036,4.035) -- (4.036,4.105); +% \draw[wforange, line width=0.72pt] (4.036,4.035) -- (4.036,4.105); +% \filldraw[wforange] (3.991,4.025) rectangle (4.081,4.115); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,3.530) {Aurora}; +% \draw[wfslate, line width=0.72pt] (3.580,3.400) -- (5.276,3.400); +% \draw[wfslate, line width=0.72pt] (3.580,3.365) -- (3.580,3.435); +% \draw[wfslate, line width=0.72pt] (5.276,3.365) -- (5.276,3.435); +% \filldraw[wfslate] (4.428,3.400) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (2.627,3.660) -- (7.723,3.660); +% \draw[wforange, line width=0.72pt] (2.627,3.625) -- (2.627,3.695); +% \draw[wforange, line width=0.72pt] (7.723,3.625) -- (7.723,3.695); +% \filldraw[wforange] (5.130,3.615) rectangle (5.220,3.705); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,3.120) {ClimaX}; +% \draw[wfslate, line width=0.72pt] (4.032,2.990) -- (4.060,2.990); +% \draw[wfslate, line width=0.72pt] (4.032,2.955) -- (4.032,3.025); +% \draw[wfslate, line width=0.72pt] (4.060,2.955) -- (4.060,3.025); +% \filldraw[wfslate] (4.046,2.990) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.036,3.250) -- (4.036,3.250); +% \draw[wforange, line width=0.72pt] (4.036,3.215) -- (4.036,3.285); +% \draw[wforange, line width=0.72pt] (4.036,3.215) -- (4.036,3.285); +% \filldraw[wforange] (3.991,3.205) rectangle (4.081,3.295); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,2.710) {StormCast}; +% \draw[wfslate, line width=0.72pt] (4.036,2.580) -- (4.036,2.580); +% \draw[wfslate, line width=0.72pt] (4.036,2.545) -- (4.036,2.615); +% \draw[wfslate, line width=0.72pt] (4.036,2.545) -- (4.036,2.615); +% \filldraw[wfslate] (4.036,2.580) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.036,2.840) -- (4.036,2.840); +% \draw[wforange, line width=0.72pt] (4.036,2.805) -- (4.036,2.875); +% \draw[wforange, line width=0.72pt] (4.036,2.805) -- (4.036,2.875); +% \filldraw[wforange] (3.991,2.795) rectangle (4.081,2.885); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,2.300) {DLWP}; +% \draw[wfslate, line width=0.72pt] (4.036,2.170) -- (4.036,2.170); +% \draw[wfslate, line width=0.72pt] (4.036,2.135) -- (4.036,2.205); +% \draw[wfslate, line width=0.72pt] (4.036,2.135) -- (4.036,2.205); +% \filldraw[wfslate] (4.036,2.170) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.044,2.430) -- (4.735,2.430); +% \draw[wforange, line width=0.72pt] (4.044,2.395) -- (4.044,2.465); +% \draw[wforange, line width=0.72pt] (4.735,2.395) -- (4.735,2.465); +% \filldraw[wforange] (4.345,2.385) rectangle (4.435,2.475); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,1.890) {FCN}; +% \draw[wfslate, line width=0.72pt] (4.036,1.760) -- (4.036,1.760); +% \draw[wfslate, line width=0.72pt] (4.036,1.725) -- (4.036,1.795); +% \draw[wfslate, line width=0.72pt] (4.036,1.725) -- (4.036,1.795); +% \filldraw[wfslate] (4.036,1.760) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (3.971,2.020) -- (4.286,2.020); +% \draw[wforange, line width=0.72pt] (3.971,1.985) -- (3.971,2.055); +% \draw[wforange, line width=0.72pt] (4.286,1.985) -- (4.286,2.055); +% \filldraw[wforange] (4.083,1.975) rectangle (4.173,2.065); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,1.480) {FengWu}; +% \draw[wfslate, line width=0.72pt] (4.036,1.350) -- (4.036,1.350); +% \draw[wfslate, line width=0.72pt] (4.036,1.315) -- (4.036,1.385); +% \draw[wfslate, line width=0.72pt] (4.036,1.315) -- (4.036,1.385); +% \filldraw[wfslate] (4.036,1.350) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.028,1.610) -- (4.127,1.610); +% \draw[wforange, line width=0.72pt] (4.028,1.575) -- (4.028,1.645); +% \draw[wforange, line width=0.72pt] (4.127,1.575) -- (4.127,1.645); +% \filldraw[wforange] (4.032,1.565) rectangle (4.122,1.655); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,1.070) {FuXi}; +% \draw[wfslate, line width=0.72pt] (4.036,0.940) -- (4.036,0.940); +% \draw[wfslate, line width=0.72pt] (4.036,0.905) -- (4.036,0.975); +% \draw[wfslate, line width=0.72pt] (4.036,0.905) -- (4.036,0.975); +% \filldraw[wfslate] (4.036,0.940) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.029,1.200) -- (4.087,1.200); +% \draw[wforange, line width=0.72pt] (4.029,1.165) -- (4.029,1.235); +% \draw[wforange, line width=0.72pt] (4.087,1.165) -- (4.087,1.235); +% \filldraw[wforange] (4.013,1.155) rectangle (4.103,1.245); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,0.660) {Pangu-Weather}; +% \draw[wfslate, line width=0.72pt] (4.036,0.530) -- (4.036,0.530); +% \draw[wfslate, line width=0.72pt] (4.036,0.495) -- (4.036,0.565); +% \draw[wfslate, line width=0.72pt] (4.036,0.495) -- (4.036,0.565); +% \filldraw[wfslate] (4.036,0.530) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (4.025,0.790) -- (4.076,0.790); +% \draw[wforange, line width=0.72pt] (4.025,0.755) -- (4.025,0.825); +% \draw[wforange, line width=0.72pt] (4.076,0.755) -- (4.076,0.825); +% \filldraw[wforange] (4.006,0.745) rectangle (4.096,0.835); +% \node[anchor=east, font=\scriptsize, text=black!82] at (2.320,0.250) {AlphaEarth}; +% \draw[wfslate, line width=0.72pt] (4.700,0.120) -- (6.103,0.120); +% \draw[wfslate, line width=0.72pt] (4.700,0.085) -- (4.700,0.155); +% \draw[wfslate, line width=0.72pt] (6.103,0.085) -- (6.103,0.155); +% \filldraw[wfslate] (5.401,0.120) circle[radius=0.045]; +% \draw[wforange, line width=0.72pt] (3.872,0.380) -- (4.815,0.380); +% \draw[wforange, line width=0.72pt] (3.872,0.345) -- (3.872,0.415); +% \draw[wforange, line width=0.72pt] (4.815,0.345) -- (4.815,0.415); +% \filldraw[wforange] (4.298,0.335) rectangle (4.388,0.425); +% \end{tikzpicture}% +% } +% \caption{\textbf{Fixed-feature selection-regret check (RQ2).} Fixed-feature selection regret \(\delta = D(h_D)-D(h_R)\) under union-\(F_1\). \textcolor{wfslate}{$\bullet$} uses the full grid; \textcolor{wforange}{$\blacksquare$} uses the top \(20\%\) fire-prone cells from training fire frequency. Horizontal intervals use the same colors and show mean \(\pm\) std over five seeds, in percentage points.} +% \vspace{+0.4em} +% \label{fig:selection_regret_diagnostic} +% \end{wrapfigure} + +\subsection{Head-Selection Sensitivity Under Fixed Features (RQ2)} +\label{sec:rq2} + +To answer RQ2, we conduct a fixed-feature check on occupancy and fire spread tasks, holding the frozen feature source, $\mathcal{T}$, $\Omega$, $\Lambda$, and candidate head family $\mathcal{H} \subseteq \mathcal{A}$ fixed while varying only the selection metric between PR-AUC-based and decision-F1-based selection. The resulting selection regret $\delta = D(h_D) - D(h_R)$ measures the decision-score loss induced by metric misalignment. Occupancy results are reported in Figure~\ref{fig:selection_regret_diagnostic} under both global and fire-prone scopes. Full per-seed and per-head details are reported in Appendix~\ref{sec:app_seeded_audits}, and the exact, tolerated, and union regret breakdown is provided in Appendix Table~\ref{tab:appendix_selection_regret_tolerance}. + +The fixed-feature results show that head-selection metrics introduce substantial backbone-dependent variation that is not explained by representation quality alone. Some backbones exhibit near-zero regret, indicating agreement between PR-AUC and decision-F1 selection, while others show large regret concentrated in specific scope-matching settings. Regret is generally larger under the global scope, where severe fire imbalance amplifies misalignment between ranking and decision metrics~\cite{mcdermott2024aurocauprc}. Restricting evaluation to fire-prone scopes typically reduces regret by concentrating evaluation on fire-relevant regions. A similar pattern appears for fire spread, where ranking and decision metrics can favor different heads under the same frozen representation. These results show that selection metrics must be aligned with the evaluation objective as part of the evaluation contract~\cite{traub2024selectiveclassification}. + +\subsection{Supporting Task Checks (RQ3)} +\label{sec:rq3} + +To answer RQ3, we evaluate all backbones across the four supporting task contracts, \textit{burned area, analog retrieval, smoke PM$_{2.5}$, }and\textit{ extreme heat}, and examine whether the reference-versus-frozen ordering established under primary tasks generalizes across task forms. A rank overview across all six contracts is provided in Figure~\ref{fig:task_comparator_normalized_map}, which maps backbone-by-task rank positions and makes cross-task ordering shifts directly visible. Native metric values are reported in Table~\ref{tab:supporting_results}. Additional supporting-task diagnostics are reported in Appendix Tables~\ref{tab:app_burned_area_median_acre}, \ref{tab:app_analog_rank_depth}, \ref{tab:app_smoke_high_event}, and~\ref{tab:app_heat_event_pr}. + + +The supporting task results produce three qualitatively distinct patterns relative to the primary findings. Burned area largely preserves the reference-versus-frozen ordering seen under occupancy and spread: \ourfm\ leads frozen entries on log-RMSE and Spearman $\rho$, suggesting that the representational advantage of wildfire-specific pretraining generalizes to event-scale regression under a different metric family, providing convergent evidence for the primary claim. Analog retrieval and smoke PM$_{2.5}$ show a different pattern, with AlphaEarth matching \ourfm\ closely on both tasks while atmospheric FMs show near-zero correlation on smoke PM$_{2.5}$, indicating that retrieval and air-quality signals are captured comparably by a general remote-sensing backbone, and that the primary occupancy advantage does not extend uniformly to these task forms. Extreme heat exhibits the largest variance across the comparator set, with atmospheric FMs ranging from near-reference performance to near-complete failure depending on backbone pretraining domain, while AlphaEarth again matches \ourfm\ closely. The scale of this variance is itself informative: aggregating scores across task forms without respecting contract boundaries would produce rankings dominated by scale artifacts in the extreme heat block rather than by transfer quality. Taken together, these results establish that supporting tasks bound rather than extend the primary claim, they provide useful evidence about where backbone families generalize and where they do not, but they cannot substitute for primary decision task evaluation, and their results must within their own task-form contracts. + +\begin{figure}[t] + \centering +\vspace{-5mm} + + \includegraphics[width=\textwidth]{figures/fig_rank_heatmap1.pdf} + \vspace{-2mm} + \caption{{\textbf{Rank map for supporting task comparison (RQ4).} Each row fixes one task contract $\mathcal{C}$ and ranks the eligible backbones within that contract. The figure shows rank changes across task forms; native metric values are reported in Table~\ref{tab:supporting_results}.}} +\vspace{-6mm} + \label{fig:task_comparator_normalized_map} +\end{figure} + +\begin{table}[t] +\centering +\small +\setlength{\tabcolsep}{3.5pt} +\renewcommand{\arraystretch}{1.18} +\caption{% + \textbf{Supporting task-metric matrix (RQ3).} + Top: final burned area and analog retrieval. + Bottom: smoke PM$_{2.5}$ and extreme heat. + Each block fixes $\mathcal{T}$, $\Lambda$, and $\Omega$; backbone + column is shared across paired tasks. \ourfm\ row is + separated by a rule as the empirical anchor. \textbf{Bold} marks + the best value per metric. For error metrics + lower is better ($\downarrow$); for $F_1$, nDCG, and $r$ higher + is better ($\uparrow$). +} +\label{tab:supporting_results} +\resizebox{\textwidth}{!}{% +\begin{tabular}{lcccccc} +\toprule +& \multicolumn{3}{c}{\textbf{Burned area}} +& \multicolumn{3}{c}{\textbf{Analog retrieval}} \\ +\cmidrule(lr){2-4}\cmidrule(lr){5-7} +\textbf{Backbone} +& \textbf{log-RMSE$\downarrow$} & \textbf{log-MAE$\downarrow$} +& \textbf{Spearman$\uparrow$} +& \textbf{nDCG@10$\uparrow$} & \textbf{log-RMSE$\downarrow$} +& \textbf{log-MAE$\downarrow$} \\ +\midrule +\ourfm\ +& \ensuremath{\mathbf{1.1657}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0126}}} +& \ensuremath{\mathbf{1.0423}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0081}}} +& \ensuremath{\mathbf{0.6298}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0338}}} +& \ensuremath{\mathbf{0.5099}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0336}}} +& \ensuremath{\mathbf{1.1977}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.1029}}} +& \ensuremath{\mathbf{1.0043}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0759}}} \\ +\midrule +Prithvi-WxC +& \ms{1.3630}{0.0681} & \ms{1.2435}{0.0668} & \ms{0.1799}{0.3002} +& \ms{0.3857}{0.0189} & \ms{1.3908}{0.0938} & \ms{1.2585}{0.0865} \\ +Aurora +& \ms{1.8658}{0.2009} & \ms{1.6717}{0.1245} & \ms{-0.1156}{0.2982} +& \ms{0.4046}{0.0144} & \ms{1.3659}{0.0792} & \ms{1.2596}{0.0968} \\ +ClimaX +& \ms{2.0300}{0.2103} & \ms{1.8443}{0.1528} & \ms{-0.2515}{0.2688} +& \ms{0.4143}{0.0191} & \ms{1.4526}{0.0926} & \ms{1.2441}{0.1446} \\ +StormCast +& \ms{1.6679}{0.1438} & \ms{1.4745}{0.1134} & \ms{0.1830}{0.1969} +& \ms{0.4076}{0.0094} & \ms{1.3663}{0.0781} & \ms{1.2371}{0.1078} \\ +DLWP +& \ms{1.3070}{0.0980} & \ms{1.1769}{0.0834} & \ms{0.4888}{0.1368} +& \ms{0.3972}{0.0146} & \ms{1.5351}{0.0802} & \ms{1.3196}{0.0781} \\ +FCN +& \ms{1.3693}{0.0885} & \ms{1.2599}{0.0723} & \ms{0.3484}{0.1662} +& \ms{0.4316}{0.0134} & \ms{1.4604}{0.1035} & \ms{1.2351}{0.0586} \\ +FengWu +& \ms{1.3715}{0.1011} & \ms{1.2604}{0.0820} & \ms{0.3221}{0.2004} +& \ms{0.4246}{0.0237} & \ms{1.4179}{0.0986} & \ms{1.2233}{0.0915} \\ +FuXi +& \ms{1.4068}{0.1011} & \ms{1.3023}{0.0789} & \ms{0.2663}{0.2561} +& \ms{0.4279}{0.0212} & \ms{1.4290}{0.0929} & \ms{1.2236}{0.0961} \\ +Pangu-Weather +& \ms{1.3280}{0.0735} & \ms{1.2081}{0.0607} & \ms{0.4141}{0.1573} +& \ms{0.4017}{0.0245} & \ms{1.4235}{0.0731} & \ms{1.2225}{0.0847} \\ +AlphaEarth +& \ms{2.4068}{0.2841} & \ms{2.0822}{0.2371} & \ms{-0.3428}{0.1716} +& \ms{0.5086}{0.0440} & \ms{1.2158}{0.1310} & \ms{1.0350}{0.1018} \\ +\bottomrule +\end{tabular} +} + +\vspace{4pt} + +\resizebox{\textwidth}{!}{% +\begin{tabular}{lcccccc} +\toprule +& \multicolumn{3}{c}{\textbf{Smoke PM$_{2.5}$}} +& \multicolumn{3}{c}{\textbf{Extreme heat}} \\ +\cmidrule(lr){2-4}\cmidrule(lr){5-7} +\textbf{Backbone} +& \textbf{RMSE$\downarrow$} & \textbf{MAE$\downarrow$} +& \textbf{Pearson $r\uparrow$} +& \textbf{RMSE-C$\downarrow$} & \textbf{MAE-C$\downarrow$} +& \textbf{Exceed.\ $F_1\uparrow$} \\ +\midrule +\ourfm\ +& \ms{4.4646}{0.0060} +& \ms{2.4108}{0.0016} +& \ensuremath{\mathbf{0.6368}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0013}}} +& \ensuremath{\mathbf{0.2179}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0043}}} +& \ensuremath{\mathbf{0.1787}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0018}}} +& \ms{0.9541}{0.0164} \\ +\midrule +Prithvi-WxC +& \ms{6.0382}{0.0828} & \ms{3.7301}{0.0055} & \ms{0.0243}{0.0045} +& \ms{4.6225}{0.0192} & \ms{2.6315}{0.0128} & \ms{0.8693}{0.0023} \\ +Aurora +& \ms{6.0384}{0.0828} & \ms{3.7265}{0.0055} & \ms{0.0193}{0.0043} +& \ms{18.0474}{0.0708} & \ms{15.3747}{0.0594} & \ms{0.0951}{0.0038} \\ +ClimaX +& \ms{6.0402}{0.0828} & \ms{3.7290}{0.0055} & \ms{0.0004}{0.0029} +& \ms{17.6492}{0.0347} & \ms{14.4938}{0.0319} & \ms{0.7684}{0.0068} \\ +StormCast +& \ms{6.1230}{0.0830} & \ms{3.8182}{0.0073} & \ms{0.0183}{0.0041} +& \ms{1.7671}{0.2145} & \ms{1.3507}{0.1576} & \ms{0.9073}{0.0189} \\ +DLWP +& \ms{5.9289}{0.1031} & \ms{3.7331}{0.0088} & \ms{0.0303}{0.0060} +& \ms{2.2662}{0.1106} & \ms{1.7153}{0.0748} & \ms{0.9156}{0.0112} \\ +FCN +& \ms{5.9277}{0.1033} & \ms{3.7345}{0.0088} & \ms{0.0312}{0.0062} +& \ms{2.1657}{0.1800} & \ms{1.6033}{0.1039} & \ms{0.9257}{0.0096} \\ +FengWu +& \ms{5.9297}{0.1032} & \ms{3.7395}{0.0088} & \ms{0.0304}{0.0063} +& \ms{2.1266}{0.1589} & \ms{1.5801}{0.1004} & \ms{0.0481}{0.0459} \\ +FuXi +& \ms{5.9319}{0.1029} & \ms{3.7398}{0.0088} & \ms{0.0299}{0.0061} +& \ms{2.1282}{0.0969} & \ms{1.5759}{0.0719} & \ms{0.2268}{0.0623} \\ +Pangu-Weather +& \ms{5.9270}{0.1036} & \ms{3.7320}{0.0088} & \ms{0.0301}{0.0060} +& \ms{2.2045}{0.1483} & \ms{1.6307}{0.0889} & \ms{0.0199}{0.0062} \\ +AlphaEarth +& \ensuremath{\mathbf{4.4403}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0488}}} +& \ensuremath{\mathbf{2.3992}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0056}}} +& \ms{0.6347}{0.0066} +& \ms{0.2194}{0.0039} +& \ms{0.1800}{0.0014} +& \ensuremath{\mathbf{0.9542}{\mkern1mu}_{\scriptscriptstyle \boldsymbol{\pm}\mathbf{0.0107}}} \\ +\bottomrule +\end{tabular} +} +\end{table} + + +\paragraph{Pattern 1: primary pattern preserved (burned area).} +\ourfm\ leads all frozen entries on log-RMSE and Spearman $\rho$. The ordering observed under occupancy and spread is preserved under burned-area regression despite the different prediction unit and metric family. + +\paragraph{Pattern 2: primary pattern bounded (analog retrieval and smoke PM$_{2.5}$).} +For analog retrieval, AlphaEarth matches \ourfm\ (nDCG@10 $= 0.51 \pm 0.04$ vs.\ $0.51 \pm 0.03$). For smoke PM$_{2.5}$, AlphaEarth also matches \ourfm\ on MAE and Pearson $r$, while atmospheric Earth FMs show near-zero correlation. These results show that the occupancy-and-spread ordering does not fully extend to all supporting tasks once AlphaEarth is included. + +\paragraph{Pattern 3: primary pattern bounded with large variance (extreme heat).} AlphaEarth matches \ourfm on RMSE-C and remains close on exceedance F1, while atmospheric FMs range from RMSE-C $= 1.77$ (StormCast) to $18.05$ (Aurora). This large spread indicates that aggregated scores across task forms would be dominated by scale artifacts rather than transfer quality, reinforcing the need for per-contract reporting established in Section~\ref{sec:eval}. + +\textit{Answer to RQ3:} Figure~\ref{fig:task_comparator_normalized_map} and Table~\ref{tab:supporting_results} show that burned area preserves the primary reference-versus-frozen pattern under a different metric family. Analog retrieval, smoke PM$_{2.5}$, and extreme heat bound this pattern: AlphaEarth matches or approaches \ourfm on these tasks, indicating that the primary occupancy and spread claims do not extend uniformly across all task forms. diff --git a/paper/sections/5_conclusion.tex b/paper/sections/5_conclusion.tex new file mode 100644 index 0000000000000000000000000000000000000000..8cbc0785fbf3f7901d0cb541617b609a7c4316cf --- /dev/null +++ b/paper/sections/5_conclusion.tex @@ -0,0 +1,31 @@ +\vspace{-4em} +\section{Conclusion} + +We introduced \ourfm, the first foundation model pretrained +specifically for wildfire prediction using fire-relevant +multimodal data. Our results show that wildfire forecasting +requires representations aligned with wildfire dynamics rather +than transfer alone from general atmospheric or geophysical +pretraining. +At the same time, our study shows that reliable wildfire +transfer evaluation is substantially more difficult than +standard benchmark settings suggest. Wildfire transfer +conclusions depend strongly on matching rules, head-selection +metrics, and task form, and scores computed under different +evaluation settings are not directly comparable. These effects +become particularly pronounced in sparse spatiotemporal +prediction settings such as wildfire forecasting. +We therefore introduced a fixed-contract evaluation framework +for wildfire Earth-FM transfer. By explicitly specifying the +task, metric, matching rule, evaluation scope, and head family +before comparison, fixed-contract evaluation enables more +controlled and interpretable comparison across wildfire tasks +and models. +We hope \ourfm\ and the fixed-contract framework provide a +foundation for future wildfire-specific Earth FMs, transfer +benchmarks, and decision-oriented evaluation protocols. +More broadly, our research provides a reliable system to guide real-world intervention and resource allocation at the intersection of AI for environmental decision-making. + +\paragraph{Limitations.} The conclusions apply to the task forms, scopes, evaluation rules, and comparator eligibility decisions used in this study. +The evaluation covers selected wildfire decision tasks and supporting retrieval and regression task forms. +They provide task-form evidence rather than a single score across all wildfire-related prediction tasks. diff --git a/paper/sections/appendix.tex b/paper/sections/appendix.tex new file mode 100644 index 0000000000000000000000000000000000000000..58e51f94adb65a9195e3dc4bf528dcf211ceb9e5 --- /dev/null +++ b/paper/sections/appendix.tex @@ -0,0 +1,733 @@ +% ============================================================ +% APPENDIX +% ============================================================ +\appendix + +% Copy-paste safety: these definitions are no-ops when main.tex already defines them. +\providecommand{\ms}[2]{\ensuremath{#1{\mkern1mu}_{\scriptscriptstyle \pm #2}}} + + +% ──────────────────────────────────────────────────────────── +% TABLE OF CONTENTS (appendix only) +% ──────────────────────────────────────────────────────────── +\section*{Appendix Contents} +\addcontentsline{toc}{section}{Appendix Contents} + + +\begin{center} +\begin{tabular}{@{}p{0.82\textwidth}r@{}} +\textbf{A\quad Evaluation Contract Specifications} & \pageref{sec:app_contract} \\ +\quad A.1\enspace Matching Rule Definitions & \pageref{sec:app_contract_matching} \\ +\quad A.2\enspace Task-Form Contract Parameters & \pageref{sec:app_contract_params} \\ +\quad A.3\enspace Evaluation Scope Definitions & \pageref{sec:app_contract_scope} \\[4pt] +\textbf{B\quad Controlled Check Details} & \pageref{sec:app_checks} \\ +\quad B.1\enspace Fixed-Output Check: Full Sweep & \pageref{sec:app_checks_output} \\ +\quad B.2\enspace Fixed-Feature Check: Selection Summary & \pageref{sec:app_checks_feature} \\ +\quad B.3\enspace Selection Regret Under Matching Rules & \pageref{sec:app_checks_regret} \\ +\quad B.4\enspace Additional Value Tables & \pageref{sec:app_checks_values} \\[4pt] +\textbf{C\quad Comparator Eligibility Notes} & \pageref{sec:comparator_audit} \\[4pt] +\textbf{D\quad Seeded Audits} & \pageref{sec:app_seeded_audits} \\ +\quad D.1\enspace Seed Robustness Summary & \pageref{sec:app_seed_robustness} \\[4pt] +\textbf{E\quad Lightweight Head and Adaptation Details} & \pageref{sec:app_heads} \\[4pt] +\textbf{F\quad Limitations} & \pageref{sec:limitations} \\[4pt] +\textbf{G\quad Reproducibility and Evaluation Artifacts} & \pageref{sec:repro_compute_impact} \\ +\end{tabular} +\end{center} + +\noindent\textit{Retention rule.} +Appendix tables are retained when they add contract parameters, controlled-check arithmetic, +task-specific non-main metrics, seed summaries, eligibility checks, or protocol details. +Full task matrices and reference-summary tables that repeat the main result tables are not repeated here. + +\clearpage + +% ============================================================ +% A EVALUATION CONTRACT SPECIFICATIONS +% ============================================================ +\section{Evaluation Contract Specifications} +\label{sec:app_contract} + +% ──────────────────────────────────────────────────────────── +\subsection{Matching Rule Definitions} +\label{sec:app_contract_matching} + +The three matching rules used across occupancy task forms are defined as follows. + +\noindent\textbf{Exact matching.} +A predicted unit-time pair $(i,t) \in \widehat{P}_\tau$ is counted as a true positive if and only if the same pair appears in the observed fire set $P = \{(i,t): y_{i,t}=1\}$. +This is the strictest rule and yields the lowest $F_1$ for any fixed output. + +\noindent\textbf{Tolerated matching.} +A predicted pair $(i,t)$ is counted as correct if there exists an observed pair $(i',t') \in P$ such that $\|i - i'\|_\infty \le k$ and $|t - t'| \le \Delta t$, where $k$ is the spatial tolerance in grid cells and $\Delta t$ is the temporal tolerance in forecast steps. +Both parameters are fixed as part of the evaluation contract $\mathcal{C}$ before scoring. + +\noindent\textbf{Union matching.} +A predicted pair is counted as a true positive if it satisfies either exact or tolerated matching. +The resulting union-$F_1$ provides an upper bound on decision performance under the chosen tolerance. + +\noindent\textbf{Fixed parameter values.} +For occupancy, the spatial tolerance is $k=8$ grid cells. +The temporal tolerance is $\Delta t=3$ forecast steps for union matching and $\Delta t=0$ for spatial-only tolerance. +The threshold $\tau$ is selected on validation strict-$F_1$ before test scoring. +For fire spread, the spatial tolerance is $k=4$ grid cells, $\Delta t=0$, and the threshold is selected on validation spatial $F_1$. + +\noindent Table~\ref{tab:app_matching_rule_params} records the fixed matching-rule parameters. + +\begin{table}[h] +\centering +\small +\setlength{\tabcolsep}{10pt} +\renewcommand{\arraystretch}{1.2} +\caption{Matching-rule values used in the evaluation contracts.} +\label{tab:app_matching_rule_params} +\begin{tabular}{lll} +\toprule +\textbf{Parameter} & \textbf{Occupancy} & \textbf{Fire spread} \\ +\midrule +\(k\) & 8 cells & 4 cells \\ +\(\Delta t\) & 3 for union; 0 spatial-only & 0 \\ +\(\tau\) & val. strict \(F_1\) & val. spatial \(F_1\) \\ +\bottomrule +\end{tabular} +\end{table} + +% ──────────────────────────────────────────────────────────── +\subsection{Task-Form Contract Parameters} +\label{sec:app_contract_params} + +Table~\ref{tab:app_contract_params_full} lists fixed scoring values not shown in the main contract map. + +\begin{table}[h] +\centering +\scriptsize +\setlength{\tabcolsep}{3.5pt} +\renewcommand{\arraystretch}{1.2} +\caption{Fixed scoring values used by each task-form contract.} +\label{tab:app_contract_params_full} +\begin{adjustbox}{max width=\textwidth} +\begin{tabular}{llll} +\toprule +\textbf{\(\mathcal{T}\)} & \textbf{Scoring} & \textbf{Validation} & \textbf{\(\Omega\)} \\ +\midrule +Occupancy & \(k=8,\Delta t=3\); exact/tol./union \(F_1\) & val. strict \(F_1\) & global; top-5/10/20\% fire-prone \\ +Fire spread & \(k=4,\Delta t=0\); exact/spatial \(F_1\), AP & val. spatial \(F_1\) & spread-region cells \\ +Final burned area & log-RMSE, log-MAE, Spearman \(\rho\) & val. log-RMSE & test events \\ +Analog retrieval & nDCG@10; retrieved-event log error & val. nDCG@10 & test events \\ +Smoke PM\(_{2.5}\) & RMSE, MAE, Pearson \(r\); exceedance 35 & val. RMSE & test stations \\ +Extreme heat & RMSE-C, MAE-C, exceedance \(F_1\) & val. threshold 27/30/33\(^{\circ}\)C & heat-region stations \\ +\bottomrule +\end{tabular} +\end{adjustbox} +\end{table} + +% ──────────────────────────────────────────────────────────── +\subsection{Evaluation Scope Definitions} +\label{sec:app_contract_scope} + +\noindent\textbf{Global scope.} +Evaluation covers all spatial units in the domain, including fire-inactive regions. +This scope can mask model differences on fire-relevant locations because inactive cells inflate true-negative counts. + +\noindent\textbf{Fire-prone scope.} +Evaluation is restricted to grid cells in the top-$k$\% of historical fire activity. +We report results for top-5\%, top-10\%, and top-20\% cutoffs. +The cutoff thresholds are derived from the training period and held fixed at test time. + +\noindent\textbf{Spread region scope.} +For fire spread tasks, evaluation is restricted to the predicted and observed burned raster patches. +Only cells within the union of $\widehat{B}$ and $B$ contribute to metric computation. + +\noindent\textbf{Fixed scope sizes.} +The global scope contains 8,085,000 test cells. +The fire-prone top-5\%, top-10\%, and top-20\% scopes contain 404,280, 808,560, and 1,617,000 test cells, respectively. +The spread-region scope is event-specific and uses the union of $\widehat{B}$ and $B$. + +\begin{table}[h] +\centering +\small +\setlength{\tabcolsep}{8pt} +\renewcommand{\arraystretch}{1.2} +\caption{Scope values used in the evaluation contracts.} +\label{tab:app_scope_params} +\begin{tabular}{lcc} +\toprule +\textbf{\(\Omega\)} & \textbf{Definition} & \textbf{Units} \\ +\midrule +Global & full domain & 8,085,000 test cells \\ +Fire-prone top-5\% & top 5\% by training-period fire frequency & 404,280 test cells \\ +Fire-prone top-10\% & top 10\% by training-period fire frequency & 808,560 test cells \\ +Fire-prone top-20\% & top 20\% by training-period fire frequency & 1,617,000 test cells \\ +Spread region & union of \(\widehat{B}\) and \(B\) & event-specific cells \\ +\bottomrule +\end{tabular} +\end{table} + +\clearpage + +% ============================================================ +% B CONTROLLED CHECK DETAILS +% ============================================================ +\section{Controlled Check Details} +\label{sec:app_checks} + + +\begin{figure}[t] + \centering + \includegraphics[width=\textwidth]{figures/fig_fireprone_contract_progression_compact.pdf} + \caption{ +\textbf{Matching-rule sensitivity in fire-prone occupancy (RQ1).} +Each row holds the score field \(S\), label field \(Y\), threshold, and \(\Omega\) fixed, and changes only \(\Lambda\). +Legend: \textcolor[HTML]{17375E}{$\blacksquare$} strict \(F_1\), +\textcolor[HTML]{4F8DCC}{$\blacksquare$} added \(F_1\) from spatial tolerance, +\textcolor[HTML]{BFD7F0}{$\blacksquare$} added \(F_1\) from union matching, +red outline \ourfm, and dashed line original weather FMs vs.\ added baselines. +The horizontal axis is \(F_1\) in percent. +} + \label{fig:fireprone_contract_progression} +\end{figure} + +% ──────────────────────────────────────────────────────────── +\subsection{Fixed-Output Check: Full Sweep} +\label{sec:app_checks_output} + +The fixed-output check holds the score field $S$ and label field $Y$ fixed and varies only $\Lambda$. +Table~\ref{tab:fireprone_contract_progression} reports the full global and fire-prone sweep for all retained backbones. +The same table is the numeric counterpart to Figure~\ref{fig:fireprone_contract_progression}. + +\begin{table*}[t] + \centering + \scriptsize + \setlength{\tabcolsep}{4pt} + \caption{Occupancy \(F_1\) scores across global and fire-prone scopes. Global uses the full validation/test domain; top-\(k\) rows use train-defined fire-prone masks from historical fire frequency. Values are percentages from the same validation-selected strict threshold. Tolerance is spatial-only; union adds temporal and spatial matching. \(\Delta\) is union minus strict. Cells report five-seed mean with std in small type.} + \label{tab:fireprone_contract_progression} + \begin{tabular}{@{}llcccc@{}} + \toprule + Backbone & \(\Omega\) & Strict \(F_1\uparrow\) & Tol.\ \(F_1\uparrow\) & Union \(F_1\uparrow\) & \(\Delta\) \(\uparrow\) \\ + \midrule + \ourfm & global & \ms{0.4546}{0.1412} & \ms{29.7484}{1.2868} & \ms{59.0656}{2.7372} & \ms{58.6109}{2.6945} \\ + & top 5\% & \ms{3.5604}{0.8809} & \ms{39.2617}{1.4011} & \ms{72.8280}{2.5784} & \ms{69.2676}{1.9960} \\ + & top 10\% & \ms{3.5575}{0.8799} & \ms{39.1665}{1.3906} & \ms{72.5204}{2.5670} & \ms{68.9629}{1.9888} \\ + & top 20\% & \ms{3.5300}{0.8700} & \ms{38.2849}{1.2952} & \ms{69.7228}{2.4664} & \ms{66.1928}{1.9273} \\ + \addlinespace[1pt] + Prithvi-WxC & global & \ms{0.0552}{0.0039} & \ms{7.1649}{0.6557} & \ms{20.1853}{1.8299} & \ms{20.1301}{1.8297} \\ + & top 5\% & \ms{1.4119}{1.1635} & \ms{19.2636}{4.5019} & \ms{42.5793}{4.5495} & \ms{41.1674}{3.4846} \\ + & top 10\% & \ms{1.2376}{1.3201} & \ms{14.8780}{8.4429} & \ms{32.6913}{13.2085} & \ms{31.4536}{11.9053} \\ + & top 20\% & \ms{1.1520}{1.3770} & \ms{13.1512}{9.4556} & \ms{28.1319}{15.2866} & \ms{26.9800}{13.9224} \\ + \addlinespace[1pt] + Aurora & global & \ms{0.0656}{0.0094} & \ms{8.5009}{1.9594} & \ms{23.1037}{4.9418} & \ms{23.0382}{4.9325} \\ + & top 5\% & \ms{0.9859}{0.9299} & \ms{15.1337}{6.0821} & \ms{35.4834}{11.0192} & \ms{34.4975}{10.3728} \\ + & top 10\% & \ms{0.7790}{1.0453} & \ms{12.7381}{6.5558} & \ms{30.5305}{10.8842} & \ms{29.7515}{9.8656} \\ + & top 20\% & \ms{0.6655}{1.1043} & \ms{10.5304}{7.4309} & \ms{24.9444}{12.5844} & \ms{24.2790}{11.4943} \\ + \addlinespace[1pt] + ClimaX & global & \ms{0.3480}{0.0754} & \ms{29.7535}{3.6073} & \ms{60.1506}{7.5865} & \ms{59.8026}{7.5454} \\ + & top 5\% & \ms{1.2937}{0.1086} & \ms{34.5791}{2.3772} & \ms{69.2186}{5.7215} & \ms{67.9249}{5.7263} \\ + & top 10\% & \ms{1.2522}{0.1602} & \ms{34.3341}{2.2852} & \ms{68.5713}{5.5377} & \ms{67.3191}{5.5538} \\ + & top 20\% & \ms{1.0287}{0.2686} & \ms{30.2140}{4.2857} & \ms{60.0650}{7.5674} & \ms{59.0363}{7.5891} \\ + \addlinespace[1pt] + StormCast & global & \ms{0.0626}{0.0119} & \ms{8.1951}{2.1895} & \ms{22.3817}{5.4294} & \ms{22.3191}{5.4178} \\ + & top 5\% & \ms{0.9573}{0.8011} & \ms{15.3219}{5.5337} & \ms{36.1857}{9.7331} & \ms{35.2284}{9.1816} \\ + & top 10\% & \ms{0.7284}{0.9280} & \ms{12.6669}{6.3290} & \ms{30.4748}{10.6527} & \ms{29.7464}{9.7494} \\ + & top 20\% & \ms{0.5795}{0.9104} & \ms{10.4157}{7.3437} & \ms{24.6598}{12.3973} & \ms{24.0803}{11.4988} \\ + \addlinespace[1pt] + DLWP & global & \ms{0.1693}{0.0419} & \ms{14.9148}{3.2446} & \ms{28.1901}{6.9658} & \ms{28.0208}{6.9257} \\ + & top 5\% & \ms{1.8054}{0.4835} & \ms{31.7231}{3.2923} & \ms{55.4596}{5.2920} & \ms{53.6542}{5.4752} \\ + & top 10\% & \ms{1.6110}{0.5999} & \ms{27.6581}{5.9216} & \ms{47.1269}{8.0111} & \ms{45.5158}{7.7927} \\ + & top 20\% & \ms{1.5248}{0.8987} & \ms{20.9403}{4.7971} & \ms{34.9301}{7.8471} & \ms{33.4054}{7.8760} \\ + \addlinespace[1pt] + FCN & global & \ms{0.2829}{0.0839} & \ms{19.5061}{3.3412} & \ms{40.0604}{9.3701} & \ms{39.7775}{9.3423} \\ + & top 5\% & \ms{1.6231}{0.5064} & \ms{29.3769}{2.7626} & \ms{54.3033}{7.4089} & \ms{52.6801}{7.4389} \\ + & top 10\% & \ms{1.1777}{0.5118} & \ms{22.4217}{3.9803} & \ms{43.4510}{9.2513} & \ms{42.2734}{9.0251} \\ + & top 20\% & \ms{0.9962}{0.4315} & \ms{16.9792}{3.9371} & \ms{34.0859}{8.2616} & \ms{33.0897}{7.9275} \\ + \addlinespace[1pt] + FengWu & global & \ms{0.2613}{0.0757} & \ms{12.0050}{6.0239} & \ms{24.1022}{13.6293} & \ms{23.8410}{13.5736} \\ + & top 5\% & \ms{1.5695}{0.3592} & \ms{16.2763}{3.7024} & \ms{30.1055}{5.0103} & \ms{28.5360}{4.7696} \\ + & top 10\% & \ms{1.2427}{0.5333} & \ms{12.9503}{5.6052} & \ms{24.1854}{8.6854} & \ms{22.9427}{8.1863} \\ + & top 20\% & \ms{1.1192}{0.5023} & \ms{11.9508}{5.0745} & \ms{22.7860}{7.9115} & \ms{21.6668}{7.4438} \\ + \addlinespace[1pt] + FuXi & global & \ms{0.3774}{0.1212} & \ms{21.0323}{4.8211} & \ms{37.2888}{9.4470} & \ms{36.9114}{9.4327} \\ + & top 5\% & \ms{2.0307}{0.6800} & \ms{31.8944}{4.7331} & \ms{53.9308}{8.3822} & \ms{51.9001}{8.6878} \\ + & top 10\% & \ms{1.6542}{0.7316} & \ms{24.0128}{5.7784} & \ms{40.2140}{9.9307} & \ms{38.5597}{9.7744} \\ + & top 20\% & \ms{1.3646}{0.6773} & \ms{21.9548}{5.8601} & \ms{36.7314}{10.0289} & \ms{35.3668}{9.9223} \\ + \addlinespace[1pt] + Pangu-Weather & global & \ms{0.2755}{0.1089} & \ms{17.0909}{4.0477} & \ms{35.6386}{9.0327} & \ms{35.3630}{9.0774} \\ + & top 5\% & \ms{1.3656}{0.3064} & \ms{22.2222}{6.8613} & \ms{43.4234}{13.2383} & \ms{42.0578}{13.0599} \\ + & top 10\% & \ms{1.0931}{0.3535} & \ms{18.9337}{5.9329} & \ms{38.5325}{11.7221} & \ms{37.4394}{11.5261} \\ + & top 20\% & \ms{0.8844}{0.3601} & \ms{17.0172}{5.4859} & \ms{34.5688}{10.2932} & \ms{33.6844}{10.1334} \\ + \addlinespace[1pt] + AlphaEarth & global & \ms{2.0606}{0.4404} & \ms{29.4476}{6.0064} & \ms{37.4286}{9.9458} & \ms{35.3679}{10.0271} \\ + & top 5\% & \ms{6.9133}{0.8450} & \ms{42.8790}{4.6087} & \ms{51.7449}{8.7321} & \ms{44.8315}{9.0763} \\ + & top 10\% & \ms{6.6366}{0.9901} & \ms{41.8981}{5.9454} & \ms{50.5712}{10.0057} & \ms{43.9346}{9.9156} \\ + & top 20\% & \ms{6.1908}{1.1330} & \ms{38.8325}{7.4966} & \ms{46.3833}{12.1697} & \ms{40.1925}{11.6788} \\ + \bottomrule + \end{tabular} +\end{table*} + +% ──────────────────────────────────────────────────────────── +\subsection{Fixed-Feature Check: Selection Summary} +\label{sec:app_checks_feature} + +The paper appendix keeps the fixed-feature result at the selection-summary level. +The full per-head rows are retained in the supplementary CSV files and are not repeated as a manuscript table because many degenerate heads produce identical zero decision scores. +The supplementary selection rows report the decision-score loss after changing only the head-selection metric. + +% ──────────────────────────────────────────────────────────── +\subsection{Selection Regret Under Matching Rules} +\label{sec:app_checks_regret} + +The fixed-feature check trains the same head family $\mathcal{H}$ on a fixed feature source and changes only the selection metric. +Table~\ref{tab:appendix_selection_regret_tolerance} reports the same selection comparison under exact, tolerated, and union matching. +Here, \(h_R\) is selected by PR-AUC and \(h_D\) is selected by the decision metric. +The reported regret is \(D(h_D)-D(h_R)\). +Exact zero entries mean the two selectors give the same decision score for all five seeds. + +\begin{table*}[!t] + \centering + \scriptsize + \setlength{\tabcolsep}{4pt} + \caption{Selection-regret values under exact, tolerated, and union matching. Values are percentage-point regret from selecting \(h_R\) by PR-AUC instead of \(h_D\) by the decision metric. Rows report mean with small std over five seeds; \(0.0000\) denotes exact zero regret.} + \label{tab:appendix_selection_regret_tolerance} + \begin{adjustbox}{max width=\textwidth} + \begin{tabular}{llccc} + \toprule + \textbf{Feature} & \textbf{\(\Omega\)} & \textbf{Exact regret} & \textbf{Tolerated regret} & \textbf{Union regret} \\ + \midrule + \ourfm & global & 0.0000 & \ms{8.7830}{9.6705} & \ms{8.7830}{9.6705} \\ + \ourfm & fire-prone & 0.0000 & \ms{3.4027}{3.2045} & \ms{3.4027}{3.2045} \\ + Prithvi-WxC & global & 0.0000 & 0.0000 & 0.0000 \\ + Prithvi-WxC & fire-prone & 0.0000 & 0.0000 & 0.0000 \\ + Aurora & global & \ms{0.0200}{0.0267} & \ms{9.8520}{12.9878} & \ms{9.8520}{12.9878} \\ + Aurora & fire-prone & \ms{0.8203}{1.8341} & \ms{14.3919}{32.1219} & \ms{14.3919}{32.1219} \\ + ClimaX & global & \ms{0.0003}{0.0004} & \ms{0.1296}{0.1775} & \ms{0.1296}{0.1775} \\ + ClimaX & fire-prone & 0.0000 & 0.0000 & 0.0000 \\ + StormCast & global & 0.0000 & 0.0000 & 0.0000 \\ + StormCast & fire-prone & 0.0000 & 0.0000 & 0.0000 \\ + DLWP & global & 0.0000 & 0.0000 & 0.0000 \\ + DLWP & fire-prone & \ms{0.0770}{0.1100} & \ms{4.3266}{4.3323} & \ms{4.3266}{4.3323} \\ + FCN & global & 0.0000 & 0.0000 & 0.0000 \\ + FCN & fire-prone & \ms{0.0006}{0.0013} & \ms{1.1680}{1.9872} & \ms{1.1680}{1.9872} \\ + FengWu & global & 0.0000 & 0.0000 & 0.0000 \\ + FengWu & fire-prone & \ms{0.0691}{0.1191} & \ms{0.5222}{0.6239} & \ms{0.5222}{0.6239} \\ + FuXi & global & 0.0000 & 0.0000 & 0.0000 \\ + FuXi & fire-prone & 0.0000 & \ms{0.1084}{0.1729} & \ms{0.1084}{0.1729} \\ + Pangu-Weather & global & 0.0000 & 0.0000 & 0.0000 \\ + Pangu-Weather & fire-prone & \ms{0.0728}{0.1179} & \ms{0.1849}{0.3263} & \ms{0.1849}{0.3263} \\ + AlphaEarth & global & 0.0000 & \ms{17.2217}{8.8492} & \ms{17.2217}{8.8492} \\ + AlphaEarth & fire-prone & 0.0000 & \ms{3.8804}{5.9483} & \ms{3.8804}{5.9483} \\ + \bottomrule + \end{tabular} + \end{adjustbox} +\end{table*} + + +% ──────────────────────────────────────────────────────────── +\subsection{Additional Value Tables} +\label{sec:app_checks_values} + +Table~\ref{tab:app_occupancy_ppr_scope} +reports the predicted-positive rate behind the occupancy \(F_1\) sweep. + +\begin{table*}[t] +\centering +\small +\setlength{\tabcolsep}{4pt} +\renewcommand{\arraystretch}{1.18} +\caption{For fixed occupancy \(\mathcal{T}\), this table reports predicted-positive rate. +Values are percentages under the same validation-selected strict threshold. +Scopes \(\Omega\) are fixed before test scoring; cells report five-seed mean with std in small type.} +\label{tab:app_occupancy_ppr_scope} +\begin{tabular}{lcccc} +\toprule +\textbf{Backbone} & \textbf{\(\Omega=\)global} & \textbf{\(\Omega=\)top 5\%} & \textbf{\(\Omega=\)top 10\%} & \textbf{\(\Omega=\)top 20\%} \\ +\midrule +\ourfm & \ms{1.6808}{0.3684} & \ms{3.0619}{1.0925} & \ms{1.5310}{0.5463} & \ms{0.7655}{0.2732} \\ +Prithvi-WxC & \ms{61.9711}{30.9101} & \ms{57.4117}{47.8987} & \ms{58.4565}{51.0897} & \ms{58.9788}{52.6991} \\ +Aurora & \ms{55.5849}{19.7524} & \ms{57.2238}{35.3400} & \ms{68.7942}{37.6958} & \ms{67.2891}{38.3991} \\ +ClimaX & \ms{5.6763}{3.9261} & \ms{24.0091}{9.2816} & \ms{11.8450}{4.5067} & \ms{5.7442}{4.1341} \\ +StormCast & \ms{60.6507}{17.4895} & \ms{57.6017}{35.2921} & \ms{68.0766}{37.3899} & \ms{67.8397}{39.2410} \\ +DLWP & \ms{4.3221}{1.5619} & \ms{9.4001}{5.0807} & \ms{4.9700}{3.6849} & \ms{1.9198}{1.4678} \\ +FCN & \ms{1.5202}{1.3446} & \ms{4.7856}{2.9409} & \ms{2.7257}{1.6353} & \ms{0.8368}{0.2358} \\ +FengWu & \ms{0.4277}{0.4830} & \ms{0.6004}{0.3041} & \ms{0.2609}{0.1935} & \ms{0.1501}{0.1206} \\ +FuXi & \ms{0.4505}{0.2773} & \ms{2.9315}{2.6392} & \ms{0.5197}{0.6074} & \ms{0.3621}{0.4346} \\ +Pangu-Weather & \ms{1.0801}{1.1308} & \ms{2.0549}{2.1893} & \ms{1.4029}{1.4739} & \ms{1.0103}{1.1084} \\ +AlphaEarth & \ms{0.0691}{0.0499} & \ms{0.2826}{0.1497} & \ms{0.1524}{0.0770} & \ms{0.0656}{0.0414} \\ +\bottomrule +\end{tabular} +\end{table*} + +Tables~\ref{tab:app_spread_ap_by_scope}--\ref{tab:app_heat_event_pr} +report additional values that are not repeated in the main tables. +Each table fixes the task \(\mathcal{T}\) and reports either a different \(\Omega\), metric, or event subset. + +\begin{table*}[t] +\centering +\scriptsize +\setlength{\tabcolsep}{3pt} +\caption{For fixed spread \(\mathcal{T}\) and strict \(\Lambda\), this table reports AP under three \(\Omega\) scopes: full test, top-5\% train-fire area, and top-10\% train-fire area. Values are percentages; cells report mean with small std.} +\label{tab:app_spread_ap_by_scope} +\begin{tabular}{lccc} +\toprule +Backbone & full \(\Omega\) AP & top-5\% \(\Omega\) AP & top-10\% \(\Omega\) AP \\ +\midrule +\ourfm & \ms{30.0197}{1.5651} & \ms{40.7452}{2.0542} & \ms{37.4096}{1.8731} \\ +Prithvi-WxC & \ms{4.8319}{0.1731} & \ms{12.6086}{0.4468} & \ms{8.7051}{0.1889} \\ +Aurora & \ms{17.7723}{0.4293} & \ms{30.3106}{0.9404} & \ms{26.4732}{0.6932} \\ +ClimaX & \ms{11.1726}{0.2337} & \ms{25.7871}{1.2896} & \ms{19.9977}{1.2217} \\ +StormCast & \ms{8.1147}{1.1569} & \ms{18.5461}{1.1727} & \ms{14.1286}{1.2956} \\ +DLWP & \ms{9.2142}{2.6587} & \ms{19.3346}{2.3922} & \ms{14.9788}{2.6696} \\ +FCN & \ms{6.6774}{1.3001} & \ms{16.7396}{3.2955} & \ms{11.9308}{2.3881} \\ +FengWu & \ms{11.0046}{2.7092} & \ms{21.1506}{1.2163} & \ms{17.0113}{1.5778} \\ +FuXi & \ms{13.5507}{0.3840} & \ms{22.5434}{0.4100} & \ms{19.1964}{0.3943} \\ +Pangu-Weather & \ms{10.6250}{1.4643} & \ms{19.8294}{1.3044} & \ms{15.8013}{1.1602} \\ +AlphaEarth & \ms{12.2847}{1.3562} & \ms{22.8692}{0.4915} & \ms{18.2992}{1.2110} \\ +\bottomrule +\end{tabular} +\end{table*} + +\begin{table*}[t] +\centering +\scriptsize +\setlength{\tabcolsep}{3pt} +\caption{For fixed final-area \(\mathcal{T}\) and \(\Omega\), this table reports median log error and acre-scale errors in addition to the main log-RMSE/log-MAE/Spearman metrics. Cells report mean with small std.} +\label{tab:app_burned_area_median_acre} +\begin{tabular}{lccc} +\toprule +Backbone & log median AE & acre median AE & acre MAPE \\ +\midrule +\ourfm & \ms{1.0235}{0.0982} & \ms{4504.0692}{459.0483} & \ms{1.4525}{0.0254} \\ +Prithvi-WxC & \ms{1.2184}{0.2107} & \ms{5375.8770}{788.7906} & \ms{1.9517}{0.2875} \\ +Aurora & \ms{1.4547}{0.0301} & \ms{9904.9483}{457.4260} & \ms{6.8728}{3.0026} \\ +ClimaX & \ms{1.6841}{0.1818} & \ms{18130.4820}{3248.3873} & \ms{8.2373}{2.8540} \\ +StormCast & \ms{1.4522}{0.1519} & \ms{11155.7881}{2020.8656} & \ms{4.6142}{1.1500} \\ +DLWP & \ms{1.0952}{0.1306} & \ms{4406.9315}{303.0944} & \ms{1.7357}{0.3625} \\ +FCN & \ms{1.1688}{0.1139} & \ms{5166.9993}{213.0333} & \ms{2.0800}{0.4004} \\ +FengWu & \ms{1.1589}{0.1772} & \ms{5137.2822}{628.7543} & \ms{2.0944}{0.4545} \\ +FuXi & \ms{1.1855}{0.0612} & \ms{5697.7117}{796.8785} & \ms{2.4411}{0.5567} \\ +Pangu-Weather & \ms{1.1221}{0.1470} & \ms{5092.3621}{483.8243} & \ms{1.9571}{0.3113} \\ +AlphaEarth & \ms{1.7459}{0.6057} & \ms{15110.7573}{7106.3417} & \ms{9.7398}{2.7425} \\ +\bottomrule +\end{tabular} +\end{table*} + +\begin{table*}[t] +\centering +\scriptsize +\setlength{\tabcolsep}{3pt} +\caption{For fixed retrieval \(\mathcal{T}\) and \(\Omega\), this table reports nDCG@5, best log gap, and rank \(\rho\) in addition to the main nDCG@10/log-error metrics. Cells report mean with small std.} +\label{tab:app_analog_rank_depth} +\begin{tabular}{lccc} +\toprule +Backbone & nDCG@5 & best log gap & rank $\rho$ \\ +\midrule +\ourfm & \ms{0.5175}{0.0445} & \ms{0.1868}{0.0285} & \ms{0.6019}{0.1460} \\ +Prithvi-WxC & \ms{0.3591}{0.0107} & \ms{0.2151}{0.0594} & \ms{0.1514}{0.1489} \\ +Aurora & \ms{0.4423}{0.0210} & \ms{0.1551}{0.0437} & \ms{0.2162}{0.1856} \\ +ClimaX & \ms{0.4151}{0.0293} & \ms{0.2129}{0.0653} & \ms{0.1587}{0.2831} \\ +StormCast & \ms{0.3960}{0.0240} & \ms{0.1714}{0.0310} & \ms{0.1258}{0.1625} \\ +DLWP & \ms{0.3795}{0.0274} & \ms{0.1944}{0.0807} & \ms{-0.3865}{0.2802} \\ +FCN & \ms{0.4250}{0.0112} & \ms{0.1856}{0.0846} & \ms{-0.1357}{0.2571} \\ +FengWu & \ms{0.4228}{0.0310} & \ms{0.1870}{0.0858} & \ms{-0.1926}{0.2194} \\ +FuXi & \ms{0.4544}{0.0356} & \ms{0.2171}{0.0806} & \ms{-0.1367}{0.2885} \\ +Pangu-Weather & \ms{0.3988}{0.0506} & \ms{0.1901}{0.0838} & \ms{-0.1970}{0.2216} \\ +AlphaEarth & \ms{0.5276}{0.0531} & \ms{0.1782}{0.0454} & \ms{0.4639}{0.2802} \\ +\bottomrule +\end{tabular} +\end{table*} + +\begin{table*}[t] +\centering +\scriptsize +\setlength{\tabcolsep}{3pt} +\caption{For fixed smoke \(\mathcal{T}\) and station \(\Omega\), this table reports RMSE, MAE, and 90th-percentile absolute error on test rows with observed PM$_{2.5}\ge35$; std uses a row bootstrap over those rows. Cells report mean with small std.} +\label{tab:app_smoke_high_event} +\begin{tabular}{lccc} +\toprule +Backbone & high-smoke RMSE & high-smoke MAE & high-smoke 90th AE \\ +\midrule +\ourfm & \ms{47.4870}{0.6346} & \ms{34.3954}{0.7654} & \ms{65.6213}{3.8778} \\ +Prithvi-WxC & \ms{57.2224}{1.7268} & \ms{47.3871}{0.3153} & \ms{74.9666}{3.2381} \\ +Aurora & \ms{57.2752}{1.7248} & \ms{47.4368}{0.3149} & \ms{75.0755}{3.1074} \\ +ClimaX & \ms{57.2828}{1.7239} & \ms{47.4407}{0.3140} & \ms{75.1012}{3.0777} \\ +StormCast & \ms{56.6512}{1.7517} & \ms{46.7914}{0.3281} & \ms{74.0794}{3.4707} \\ +DLWP & \ms{57.0075}{1.7359} & \ms{47.1971}{0.3198} & \ms{74.4936}{3.3826} \\ +FCN & \ms{57.0582}{1.7339} & \ms{47.2401}{0.3187} & \ms{74.6431}{3.1982} \\ +FengWu & \ms{57.0158}{1.7357} & \ms{47.1957}{0.3194} & \ms{74.5652}{3.2871} \\ +FuXi & \ms{56.9622}{1.7371} & \ms{47.1508}{0.3201} & \ms{74.3278}{3.4435} \\ +Pangu-Weather & \ms{57.1282}{1.7307} & \ms{47.3050}{0.3170} & \ms{74.6830}{3.2375} \\ +AlphaEarth & \ms{48.0665}{0.7904} & \ms{35.6088}{0.7341} & \ms{66.7613}{3.9235} \\ +\bottomrule +\end{tabular} +\end{table*} + +\begin{table*}[t] +\centering +\scriptsize +\setlength{\tabcolsep}{3pt} +\caption{For fixed heat \(\mathcal{T}\) and heat-region \(\Omega\), this table reports precision and recall for the exceedance label used by the main \(F_1\). Cells report mean with small std.} +\label{tab:app_heat_event_pr} +\begin{tabular}{lcc} +\toprule +Backbone & precision & recall \\ +\midrule +\ourfm & \ms{0.9767}{0.0117} & \ms{0.9330}{0.0299} \\ +Prithvi-WxC & \ms{0.8260}{0.0030} & \ms{0.9173}{0.0033} \\ +Aurora & \ms{0.5920}{0.0347} & \ms{0.0517}{0.0020} \\ +ClimaX & \ms{0.7397}{0.0099} & \ms{0.7994}{0.0051} \\ +StormCast & \ms{0.8840}{0.0237} & \ms{0.9320}{0.0165} \\ +DLWP & \ms{0.9429}{0.0085} & \ms{0.8899}{0.0167} \\ +FCN & \ms{0.9408}{0.0097} & \ms{0.9111}{0.0127} \\ +FengWu & \ms{0.3808}{0.2719} & \ms{0.0266}{0.0267} \\ +FuXi & \ms{0.3262}{0.1262} & \ms{0.1810}{0.0481} \\ +Pangu-Weather & \ms{0.1159}{0.0743} & \ms{0.0112}{0.0032} \\ +AlphaEarth & \ms{0.9824}{0.0040} & \ms{0.9278}{0.0178} \\ +\bottomrule +\end{tabular} +\end{table*} + +\clearpage + +% ============================================================ +% C COMPARATOR ELIGIBILITY NOTES +% ============================================================ +\section{Comparator Eligibility Notes} +\label{sec:comparator_audit} + +All numeric comparator rows in Tables~\ref{tab:primary_results} and~\ref{tab:supporting_results} +are included only after the task form, metric, matching rule, scope, and head family are fixed. +The appendix does not repeat those full matrices. +The key eligibility rule is simple: reported rows satisfy the same contract as the row block in which they appear, while excluded rows are excluded because their representation or output form does not satisfy that contract. + +\noindent\textbf{Reading rule.} +Exact-only, tolerated, union, ranking, retrieval, and regression scores answer different questions. +The fixed-contract reading is therefore to compare entries only within one row block and not to average across task forms. + +\clearpage + +% ============================================================ +% D SEEDED AUDITS +% ============================================================ +\section{Seeded Audits} +\label{sec:app_seeded_audits} + +\subsection{Seed Robustness Summary} +\label{sec:app_seed_robustness} + +Table~\ref{tab:app_seed_robustness} summarizes stochastic checks used to support the reported mean-with-std convention. +It is not a replacement for the main fixed-contract result tables. + +\begin{table}[h] +\centering +\small +\setlength{\tabcolsep}{5pt} +\renewcommand{\arraystretch}{1.2} +\caption{Seed summaries for stochastic checks. Values report mean with small std over completed seeds.} +\label{tab:app_seed_robustness} +\begin{adjustbox}{max width=\textwidth} +\begin{tabular}{p{0.28\textwidth}cllp{0.18\textwidth}} +\toprule +\textbf{\(\mathcal{T}\) check} & \textbf{Seeds} & \textbf{Primary value} & \textbf{Other value(s)} & \textbf{Reading} \\ +\midrule +Final burned area & +5 & log-RMSE \ms{1.1657}{0.0126} & +log-MAE \ms{1.0423}{0.0081}; Spear.\ \ms{0.6298}{0.0338} & +stable across seeds \\ +Smoke PM\(_{2.5}\) & +5 & RMSE \ms{4.4646}{0.0060} & +MAE \ms{2.4108}{0.0016}; \(r\) \ms{0.6368}{0.0013} & +stable at table precision \\ +Extreme heat & +5 & RMSE-C \ms{0.2179}{0.0043} & +MAE-C \ms{0.1787}{0.0018}; exceed.\ \(F_1\) \ms{0.9541}{0.0164} & +stable across seeds \\ +Fire spread & +5 & exact \(F_1\) \ms{37.6700}{0.9800} & +spatial \(F_1\) \ms{80.9700}{2.0200}; AP \ms{30.0900}{1.2500} & +stable across seeds \\ +Aurora paired-head check & +5 & fire-prone score diff.\ \ms{6.3500}{13.2800} & +PR-AUC and union choices differ in 2/5 seeds & +variable across seeds \\ +\bottomrule +\end{tabular} +\end{adjustbox} +\end{table} + +\clearpage + +% ============================================================ +% E LIGHTWEIGHT HEAD AND ADAPTATION DETAILS +% ============================================================ +\section{Lightweight Head and Adaptation Details} +\label{sec:app_heads} + +All frozen-transfer comparisons use the same five lightweight head architectures applied +on top of the frozen backbone representations. +Table~\ref{tab:app_head_architectures} summarises each head family, its architecture, +approximate parameter count, and the adaptation procedure used. + +\begin{table}[h] +\centering +\small +\setlength{\tabcolsep}{5pt} +\renewcommand{\arraystretch}{1.3} +\caption{Lightweight head architectures used in the fixed-contract transfer comparisons. +All heads are trained from random initialisation on the frozen backbone features. +Parameter counts are approximate and depend on the feature dimensionality of each backbone.} +\label{tab:app_head_architectures} +\begin{tabular}{p{0.15\textwidth}p{0.30\textwidth}p{0.12\textwidth}p{0.33\textwidth}} +\toprule +\textbf{$\mathcal{A}$ head} & \textbf{Architecture} & \textbf{Approx.\ params} & \textbf{Notes} \\ +\midrule +Constant prior & + Outputs a fixed bias vector, ignoring input features. & + Output dimension only & + Provides a degenerate baseline; selected when backbone features carry no useful signal. \\ +Linear probe & + Single linear layer mapping backbone features to output. No nonlinearity. & + $d\times c + c$ & + Standard frozen-representation baseline. \\ +Pixel MLP & + Two-layer MLP applied independently per spatial unit. & + $d\times h + h\times c$ & + Captures per-pixel nonlinearity; ignores spatial context. \\ +Shallow adapter & + Two-layer MLP with a spatial context window; uses $3\times3$ convolution before the linear output. & + $9dh + hc$ & + Balances local spatial context with parameter efficiency. \\ +Wide adapter & + Shallow adapter with wider hidden dimension. & + $9dH + Hc$ & + Higher capacity variant; can overfit on small fire-event sets. \\ +\bottomrule +\end{tabular} +\end{table} + +\noindent\textbf{Training protocol.} +Each occupancy head-control run uses seeds $\{1,7,42,99,123\}$, the five heads listed above, and the fixed variants identity, erode-r1, and close-r1. +The spread U-Net reference is trained for 4 epochs. +The threshold $\tau$ is selected on the validation split by maximising union-$F_1$ (for occupancy) or spatial $F_1$ (for spread) and held fixed at test time. +Morphology parameters (spatial tolerance $k$, temporal tolerance $\Delta t$) are fixed as part of the evaluation contract and are not tuned after validation. + +\noindent\textbf{Head selection procedure.} +For each (feature source, scope, seed) tuple, all five heads are trained independently. +The PR-AUC-based selector picks $h_R = \arg\max_{h \in \mathcal{H}} R(h)$ on the validation set; +the decision-based selector picks $h_D = \arg\max_{h \in \mathcal{H}} D(h)$ on the same set. +The selection regret $\delta = D(h_D) - D(h_R) \ge 0$ is computed on the held-out test set. + +\clearpage + +% ============================================================ +% F LIMITATIONS +% ============================================================ +\section{Limitations} +\label{sec:limitations} + +The conclusions apply to the task forms, scopes, evaluation rules, and comparator eligibility decisions used in this study. +The evaluation covers selected wildfire decision tasks and supporting retrieval and regression task forms. +Comparator eligibility is fixed before metric values are interpreted. +This eligibility rule keeps each comparison within one task-form contract. +It also leaves some model and task pairs outside the evaluated comparison set by design. + +The transfer comparison uses frozen backbones with lightweight heads. +The results therefore describe frozen-backbone transfer under the allowed head families in each contract. +Full fine-tuning, alternative adaptation procedures, and broader head families are outside the evaluated scope. +The task-specific reference baselines serve as empirical anchors for same-contract comparison. +\ourfm is a regional wildfire reference for the reported California fixed-contract experiments. + + +The supporting retrieval and regression checks bound the primary spatial decision claim. +They provide task-form evidence rather than a single score across all wildfire-related prediction tasks. +The analysis focuses on the reported metric families, matching rules, and fixed comparison choices. +Operational response rules, intervention costs, and deployment policies are part of wildfire early-warning use contexts~\cite{goldammer1999early,pickell2017early,farahmand2020fdeo}. +They are outside the scope of this evaluation study and are not inferred from the reported scores. + +\clearpage + +% ============================================================ +% G REPRODUCIBILITY AND EVALUATION ARTIFACTS +% ============================================================ +\section{Reproducibility and Evaluation Artifacts} +\label{sec:repro_compute_impact} +\subsection{External Assets and Terms of Use} +\label{sec:external_assets_terms} + +We use external datasets and model assets only for research evaluation. +Access to each asset follows the original provider's portal, license, or terms of use; this submission does not imply that all assets are openly redistributable. +We do not redistribute raw external datasets, provider-hosted embeddings, or third-party model weights. +Table~\ref{tab:external_assets_licenses} records the source and terms-of-use status used to interpret reproducibility. + +\begin{table}[h] +\centering +\small +\setlength{\tabcolsep}{4pt} +\renewcommand{\arraystretch}{1.18} +\caption{External assets used by the study and their source or terms-of-use status.} +\label{tab:external_assets_licenses} +\begin{tabular}{p{0.25\textwidth}p{0.34\textwidth}p{0.34\textwidth}} +\toprule +\textbf{Asset family} & \textbf{Use in this study} & \textbf{Source and terms-of-use note} \\ +\midrule +NOAA HRRR fields~\cite{noaa_hrrr_ncei,noaa_hrrr_emc} +& Dynamic weather inputs for \ourfm and transfer tasks. +& NOAA provider terms and citation requirements apply. \\ +NASA FIRMS~\cite{nasa_firms} +& Active-fire occupancy supervision. +& NASA Earthdata/FIRMS access terms and citation requirements apply. \\ +LANDFIRE and WRC layers~\cite{landfire_fbfm40,landfire_canopy_cover,usfs_wrc_housing_density} +& Static fuel, canopy, and exposure context. +& Original geospatial-product provider terms and citations apply. \\ +LandScan~\cite{ornl_landscan_2024} +& Static population context. +& ORNL/LandScan source-specific access terms apply; raw data are not redistributed. \\ +WFIGS and MTBS~\cite{nifc_wfigs_perimeters,mtbs_usgs_2025} +& Event-level resources for burned-area and analog tasks. +& Original incident/perimeter-product provider terms and citations apply. \\ +External Earth-FM baselines~\cite{schmude2024prithviwxc,bodnar2025aurora,nguyen2023climax,pathak2024stormcast,weyn2020dlwp,pathak2022fourcastnet,chen2023fengwu,chen2023fuxi,bi2023panguweather,brown2025alphaearth} +& Frozen comparator representations or task-model baselines. +& Original model-provider licenses and access terms apply; third-party weights are not redistributed. \\ +\bottomrule +\end{tabular} +\end{table} + +This note supports the NeurIPS checklist and identifies the files that support the reported claims. +This file statement does not imply full raw-data release. +The main claims can be checked from the manuscript contracts, metric +definitions, and per-head result files, even if full raw-data release is +delayed or limited. Sections~3 and~4 specify the contract components used by +the main claims: task definition, split logic, label space, tolerance +parameters, scope definitions, threshold or operating-point rules, and +lightweight-head set. + +The supplementary source includes the check scripts, per-head and per-seed +CSV result files, and \LaTeX{} result tables for the expanded check and matching-rule support. +These files expose exact \(F_1\), +tolerated \(F_1\), union-\(F_1\), PR-AUC, per-head selection, +top-1 agreement, and selection-regret arithmetic. The manuscript also includes +full figure and table reproduction values in result tables and appendix tables. +These files provide a runnable check of the +selection-regret arithmetic and the table-construction logic from fixed +per-head rows. The seeded occupancy check uses seeds +$\{1,7,42,99,123\}$, and the spread task-specific U-Net check uses repeated seeds; reported error bars are standard deviations over the completed +seeded runs. Full raw wildfire inputs and large feature arrays are not +released at submission because redistribution and storage constraints require a +separate review. + +For stochastic results, the paper reports mean with standard deviation over repeated seeds. +For fixed-output or fixed-feature controls, the table uses one fixed output or feature set; the changed item is the matching rule or selection metric. + +The reported experiments use two resource classes on a shared Slurm-managed +cluster. Tabular retrieval/regression checks and same-feature head controls run +on CPU workers with 4 to 8 cores, 24 to 64~GB host memory, and 2 to 4~hour wall-clock +limits. Spread U-Net training and threshold calibration run on single-GPU jobs +with one B200 GPU, 8 CPU cores, 96~GB host memory, and a 4~hour wall-clock +limit. The seed/check waves reported in the appendix correspond to roughly +78 CPU job-hours and 12 GPU job-hours of scheduled wall-clock budget; +exploratory runs are not included in the reported compute accounting. + +The raw-data limitation is separate from the selection-regret files. +The supplementary source is sufficient to inspect the selection-regret arithmetic and reproduce the reported tables. +Full end-to-end recomputation from raw wildfire inputs is not included at submission because redistribution review is still required. +The broader impact is evaluation-facing rather than operational. +Better reading of wildfire transfer evidence can reduce overconfident benchmark claims, while misread transfer results could still encourage inappropriate reliance on models with low decision scores. +For that reason, the paper keeps its claims wildfire-centered, decision-task +specific, and explicitly separate from any predictive deployment +recommendation. diff --git a/paper_outputs/figures/fig_primary_rank_change_map.pdf b/paper_outputs/figures/fig_primary_rank_change_map.pdf new file mode 100644 index 0000000000000000000000000000000000000000..87993317c7a501f04d6924f004d5d4ed83af6a95 --- /dev/null +++ b/paper_outputs/figures/fig_primary_rank_change_map.pdf @@ -0,0 +1,133 @@ +%PDF-1.4 +%âãÏÓ +1 0 obj +<< /Type /Catalog /Pages 2 0 R >> +endobj +2 0 obj +<< /Type /Pages /Kids [3 0 R] /Count 1 >> +endobj +3 0 obj +<< /Type /Page /Parent 2 0 R /MediaBox [0 0 205 150] /Resources << /Font << /F1 4 0 R /F2 5 0 R >> >> /Contents 6 0 R >> +endobj +4 0 obj +<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >> +endobj +5 0 obj +<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Bold >> +endobj +6 0 obj +<< /Length 7493 >> +stream +1.0000 1.0000 1.0000 rg 0.00 0.00 205.00 150.40 re f +BT /F2 5.45 Tf 0.0300 0.2000 0.3300 rg 1 0 0 1 22.18 113.30 Tm (WildFIRE-FM) Tj ET +0.9850 0.9860 0.9870 rg 0.00 99.60 205.00 10.40 re f +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 25.04 102.90 Tm (Prithvi-WxC) Tj ET +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 38.45 92.50 Tm (Aurora) Tj ET +0.9850 0.9860 0.9870 rg 0.00 78.80 205.00 10.40 re f +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 39.21 82.10 Tm (ClimaX) Tj ET +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 27.77 71.70 Tm (StormCast) Tj ET +0.9850 0.9860 0.9870 rg 0.00 58.00 205.00 10.40 re f +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 41.39 61.30 Tm (DLWP) Tj ET +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 45.86 50.90 Tm (FCN) Tj ET +0.9850 0.9860 0.9870 rg 0.00 37.20 205.00 10.40 re f +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 36.82 40.50 Tm (FengWu) Tj ET +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 45.05 30.10 Tm (FuXi) Tj ET +0.9850 0.9860 0.9870 rg 0.00 16.40 205.00 10.40 re f +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 16.98 19.70 Tm (Pangu-Weather) Tj ET +BT /F1 5.45 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 28.04 9.30 Tm (AlphaEarth) Tj ET +BT /F2 6.50 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 76.09 141.00 Tm (Occupancy top-5%) Tj ET +0.60 w 0.4200 0.4300 0.4400 RG 60.00 6.00 m 60.00 136.90 l S +0.60 w 0.4200 0.4300 0.4400 RG 148.00 6.00 m 148.00 136.90 l S +BT /F2 6.50 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 156.33 141.00 Tm (Spread region) Tj ET +0.60 w 0.4200 0.4300 0.4400 RG 156.00 6.00 m 156.00 136.90 l S +0.60 w 0.4200 0.4300 0.4400 RG 200.00 6.00 m 200.00 136.90 l S +BT /F2 5.00 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 72.63 128.90 Tm (Ex->Tol) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 60.00 110.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 113.30 Tm (#2->2) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 60.00 99.60 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 102.90 Tm (#7->8) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 60.00 89.20 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 72.25 92.50 Tm (#10->11) Tj ET +0.45 w 0.4240 0.7000 0.6560 rg 1.0000 1.0000 1.0000 RG 60.00 78.80 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 82.10 Tm (#9->3) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 60.00 68.40 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 72.25 71.70 Tm (#11->10) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 60.00 58.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 61.30 Tm (#4->5) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 60.00 47.60 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 50.90 Tm (#5->6) Tj ET +0.45 w 0.9020 0.7110 0.6660 rg 1.0000 1.0000 1.0000 RG 60.00 37.20 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 40.50 Tm (#6->9) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 60.00 26.80 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 30.10 Tm (#3->4) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 60.00 16.40 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 19.70 Tm (#8->7) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 60.00 6.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 75.04 9.30 Tm (#1->1) Tj ET +0.25 w 0.8000 0.8100 0.8200 RG 60.00 6.00 m 60.00 120.40 l S +BT /F2 5.00 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 116.63 128.90 Tm (Tol->Un) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 104.00 110.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 113.30 Tm (#2->1) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 104.00 99.60 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 102.90 Tm (#8->8) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 104.00 89.20 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 116.25 92.50 Tm (#11->10) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 104.00 78.80 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 82.10 Tm (#3->2) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 104.00 68.40 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 117.65 71.70 Tm (#10->9) Tj ET +0.45 w 0.7480 0.8800 0.8520 rg 1.0000 1.0000 1.0000 RG 104.00 58.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 61.30 Tm (#5->3) Tj ET +0.45 w 0.7480 0.8800 0.8520 rg 1.0000 1.0000 1.0000 RG 104.00 47.60 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 50.90 Tm (#6->4) Tj ET +0.45 w 0.9280 0.7840 0.7440 rg 1.0000 1.0000 1.0000 RG 104.00 37.20 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 117.65 40.50 Tm (#9->11) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 104.00 26.80 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 30.10 Tm (#4->5) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 104.00 16.40 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 19.70 Tm (#7->7) Tj ET +0.45 w 0.8500 0.5650 0.5100 rg 1.0000 1.0000 1.0000 RG 104.00 6.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 119.04 9.30 Tm (#1->6) Tj ET +0.25 w 0.8000 0.8100 0.8200 RG 104.00 6.00 m 104.00 120.40 l S +BT /F2 5.00 Tf 0.1200 0.1300 0.1400 rg 1 0 0 1 169.28 128.90 Tm (Ex->Sp) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 110.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 113.30 Tm (#1->1) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 99.60 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 102.90 Tm (#4->4) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 89.20 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 92.50 Tm (#2->2) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 78.80 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 82.10 Tm (#3->3) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 68.40 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 71.70 Tm (#6->6) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 58.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 61.30 Tm (#9->9) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 47.60 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 168.25 50.90 Tm (#11->11) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 37.20 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 168.25 40.50 Tm (#10->10) Tj ET +0.45 w 0.9400 0.9500 0.9500 rg 1.0000 1.0000 1.0000 RG 156.00 26.80 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 30.10 Tm (#5->5) Tj ET +0.45 w 0.9540 0.8570 0.8220 rg 1.0000 1.0000 1.0000 RG 156.00 16.40 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 19.70 Tm (#7->8) Tj ET +0.45 w 0.8290 0.9250 0.9010 rg 1.0000 1.0000 1.0000 RG 156.00 6.00 44.00 10.40 re B +BT /F2 5.10 Tf 0.0800 0.0900 0.1000 rg 1 0 0 1 171.04 9.30 Tm (#8->7) Tj ET +0.25 w 0.8000 0.8100 0.8200 RG 156.00 6.00 m 156.00 120.40 l S +0.55 w 0.4200 0.4300 0.4400 RG 0.00 120.40 m 205.00 120.40 l S +0.55 w 0.4200 0.4300 0.4400 RG 60.00 6.00 m 200.00 6.00 l S +endstream +endobj +xref +0 7 +0000000000 65535 f +0000000015 00000 n +0000000064 00000 n +0000000121 00000 n +0000000257 00000 n +0000000327 00000 n +0000000402 00000 n +trailer +<< /Size 7 /Root 1 0 R >> +startxref +7947 +%%EOF diff --git a/paper_outputs/figures/fig_rank_heatmap1.pdf b/paper_outputs/figures/fig_rank_heatmap1.pdf new file mode 100644 index 0000000000000000000000000000000000000000..59b1ad77c4c2a2cda7a1d8922a775e02cb7671df Binary files /dev/null and b/paper_outputs/figures/fig_rank_heatmap1.pdf differ diff --git a/paper_outputs/figures/fig_selection_regret_scatter.pdf b/paper_outputs/figures/fig_selection_regret_scatter.pdf new file mode 100644 index 0000000000000000000000000000000000000000..af0fb9796d7e9f770808204214d1b4d1d2281a43 --- /dev/null +++ b/paper_outputs/figures/fig_selection_regret_scatter.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fabb8b55aac901199cc03773741a26685becffd074f52568c93bee517c2c42c0 +size 164975 diff --git a/paper_outputs/figures/overview_wildfire.pdf b/paper_outputs/figures/overview_wildfire.pdf new file mode 100644 index 0000000000000000000000000000000000000000..69ea02cc264af86c9ce90253f4b2478c0cc26770 --- /dev/null +++ b/paper_outputs/figures/overview_wildfire.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7dca6ae4a9b179693802f47d24dd66734c0f332b372a7976832a0d429333b755 +size 1334869 diff --git a/paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex b/paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex index fba91a2110efd4f106cc89644798e25f84a76b3a..57c52f58f9e80d6485a18d8b31869cd25913c3e1 100644 --- a/paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex +++ b/paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex @@ -1,37 +1,2 @@ -\begin{table*}[!t] - \centering - \scriptsize - \setlength{\tabcolsep}{4pt} - \caption{Selection-regret values under exact, tolerated, and union matching. Values are percentage-point regret from selecting \(h_R\) by PR-AUC instead of \(h_D\) by the decision metric. Rows report mean with small std over five seeds; \(0.0000\) denotes exact zero regret.} - \label{tab:appendix_selection_regret_tolerance} - \begin{adjustbox}{max width=\textwidth} - \begin{tabular}{llccc} - \toprule - \textbf{Feature} & \textbf{\(\Omega\)} & \textbf{Exact regret} & \textbf{Tolerated regret} & \textbf{Union regret} \\ - \midrule - FireWx-FM ref. & global & 0.0000 & \ms{8.7830}{9.6705} & \ms{8.7830}{9.6705} \\ - FireWx-FM ref. & fire-prone & 0.0000 & \ms{3.4027}{3.2045} & \ms{3.4027}{3.2045} \\ - Prithvi-WxC & global & 0.0000 & 0.0000 & 0.0000 \\ - Prithvi-WxC & fire-prone & 0.0000 & 0.0000 & 0.0000 \\ - Aurora & global & \ms{0.0200}{0.0267} & \ms{9.8520}{12.9878} & \ms{9.8520}{12.9878} \\ - Aurora & fire-prone & \ms{0.8203}{1.8341} & \ms{14.3919}{32.1219} & \ms{14.3919}{32.1219} \\ - ClimaX & global & \ms{0.0003}{0.0004} & \ms{0.1296}{0.1775} & \ms{0.1296}{0.1775} \\ - ClimaX & fire-prone & 0.0000 & 0.0000 & 0.0000 \\ - StormCast & global & 0.0000 & 0.0000 & 0.0000 \\ - StormCast & fire-prone & 0.0000 & 0.0000 & 0.0000 \\ - DLWP & global & 0.0000 & 0.0000 & 0.0000 \\ - DLWP & fire-prone & \ms{0.0770}{0.1100} & \ms{4.3266}{4.3323} & \ms{4.3266}{4.3323} \\ - FCN & global & 0.0000 & 0.0000 & 0.0000 \\ - FCN & fire-prone & \ms{0.0006}{0.0013} & \ms{1.1680}{1.9872} & \ms{1.1680}{1.9872} \\ - FengWu & global & 0.0000 & 0.0000 & 0.0000 \\ - FengWu & fire-prone & \ms{0.0691}{0.1191} & \ms{0.5222}{0.6239} & \ms{0.5222}{0.6239} \\ - FuXi & global & 0.0000 & 0.0000 & 0.0000 \\ - FuXi & fire-prone & 0.0000 & \ms{0.1084}{0.1729} & \ms{0.1084}{0.1729} \\ - Pangu-Weather & global & 0.0000 & 0.0000 & 0.0000 \\ - Pangu-Weather & fire-prone & \ms{0.0728}{0.1179} & \ms{0.1849}{0.3263} & \ms{0.1849}{0.3263} \\ - AlphaEarth & global & 0.0000 & \ms{17.2217}{8.8492} & \ms{17.2217}{8.8492} \\ - AlphaEarth & fire-prone & 0.0000 & \ms{3.8804}{5.9483} & \ms{3.8804}{5.9483} \\ - \bottomrule - \end{tabular} - \end{adjustbox} -\end{table*} +% Replaced by the all-backbone value table in sections/appendix.tex +% (Table~\ref{tab:appendix_selection_regret_tolerance}). diff --git a/paper_outputs/tables/tab_fireprone_contract_progression.tex b/paper_outputs/tables/tab_fireprone_contract_progression.tex index 21e14d68fc8067d2995b15d2a77df55d7ba0158d..5616b6c2609fad6179e9e544b196c4e853f7c569 100644 --- a/paper_outputs/tables/tab_fireprone_contract_progression.tex +++ b/paper_outputs/tables/tab_fireprone_contract_progression.tex @@ -2,66 +2,68 @@ \centering \scriptsize \setlength{\tabcolsep}{4pt} - \caption{Occupancy \(F_1\) scores across global and fire-prone scopes. Global uses the full validation/test domain; top-\(k\) rows use train-defined fire-prone masks from historical fire frequency. Values are percentages from the same validation-selected strict threshold. Tolerance is spatial-only; union adds temporal and spatial matching. \(\Delta\) is union minus strict. Cells report five-seed mean with std in small type.} + \caption{Occupancy scores across global and fire-prone scopes. Global uses the full validation/test domain; top-\(k\) rows use train-defined fire-prone masks from historical fire frequency. Values are \(F_1\) percentages from the same validation-selected strict threshold. Tolerance is spatial-only; union adds temporal and spatial matching. Difference is union minus strict. Rows report five-seed mean with small std. Values use four decimals.} \label{tab:fireprone_contract_progression} + \begin{adjustbox}{max width=\textwidth} \begin{tabular}{@{}llcccc@{}} \toprule - Backbone & \(\Omega\) & Strict \(F_1\uparrow\) & Tol.\ \(F_1\uparrow\) & Union \(F_1\uparrow\) & \(\Delta\) \(\uparrow\) \\ + Backbone & Scope & Strict \(F_1\uparrow\) & Tolerance \(F_1\uparrow\) & Union \(F_1\uparrow\) & Difference \(\uparrow\) \\ \midrule - FireWx-FM ref. & global & \ms{0.4546}{0.1412} & \ms{29.7484}{1.2868} & \ms{59.0656}{2.7372} & \ms{58.6109}{2.6945} \\ - & top 5\% & \ms{3.5604}{0.8809} & \ms{39.2617}{1.4011} & \ms{72.8280}{2.5784} & \ms{69.2676}{1.9960} \\ - & top 10\% & \ms{3.5575}{0.8799} & \ms{39.1665}{1.3906} & \ms{72.5204}{2.5670} & \ms{68.9629}{1.9888} \\ - & top 20\% & \ms{3.5300}{0.8700} & \ms{38.2849}{1.2952} & \ms{69.7228}{2.4664} & \ms{66.1928}{1.9273} \\ + \textcolor{blue}{FireWx-FM ref.} & global & \ms{0.4550}{0.1410} & \ms{29.7480}{1.2870} & \ms{59.0660}{2.7370} & \ms{58.6110}{2.6950} \\ + & top 5\% & \ms{3.5600}{0.8810} & \ms{39.2620}{1.4010} & \ms{72.8280}{2.5780} & \ms{69.2680}{1.9960} \\ + & top 10\% & \ms{3.5580}{0.8800} & \ms{39.1660}{1.3910} & \ms{72.5200}{2.5670} & \ms{68.9630}{1.9890} \\ + & top 20\% & \ms{3.5300}{0.8700} & \ms{38.2850}{1.2950} & \ms{69.7230}{2.4660} & \ms{66.1930}{1.9270} \\ \addlinespace[1pt] - Prithvi-WxC & global & \ms{0.0552}{0.0039} & \ms{7.1649}{0.6557} & \ms{20.1853}{1.8299} & \ms{20.1301}{1.8297} \\ - & top 5\% & \ms{1.4119}{1.1635} & \ms{19.2636}{4.5019} & \ms{42.5793}{4.5495} & \ms{41.1674}{3.4846} \\ - & top 10\% & \ms{1.2376}{1.3201} & \ms{14.8780}{8.4429} & \ms{32.6913}{13.2085} & \ms{31.4536}{11.9053} \\ - & top 20\% & \ms{1.1520}{1.3770} & \ms{13.1512}{9.4556} & \ms{28.1319}{15.2866} & \ms{26.9800}{13.9224} \\ + Prithvi-WxC & global & \ms{0.0550}{0.0040} & \ms{7.1600}{0.6600} & \ms{20.1900}{1.8300} & \ms{20.1300}{1.8300} \\ + & top 5\% & \ms{1.4100}{1.1600} & \ms{19.2600}{4.5000} & \ms{42.5800}{4.5500} & \ms{41.1700}{3.4800} \\ + & top 10\% & \ms{1.2400}{1.3200} & \ms{14.8800}{8.4400} & \ms{32.6900}{13.2100} & \ms{31.4500}{11.9100} \\ + & top 20\% & \ms{1.1500}{1.3800} & \ms{13.1500}{9.4600} & \ms{28.1300}{15.2900} & \ms{26.9800}{13.9200} \\ \addlinespace[1pt] - Aurora & global & \ms{0.0656}{0.0094} & \ms{8.5009}{1.9594} & \ms{23.1037}{4.9418} & \ms{23.0382}{4.9325} \\ - & top 5\% & \ms{0.9859}{0.9299} & \ms{15.1337}{6.0821} & \ms{35.4834}{11.0192} & \ms{34.4975}{10.3728} \\ - & top 10\% & \ms{0.7790}{1.0453} & \ms{12.7381}{6.5558} & \ms{30.5305}{10.8842} & \ms{29.7515}{9.8656} \\ - & top 20\% & \ms{0.6655}{1.1043} & \ms{10.5304}{7.4309} & \ms{24.9444}{12.5844} & \ms{24.2790}{11.4943} \\ + Aurora & global & \ms{0.0700}{0.0100} & \ms{8.5000}{1.9600} & \ms{23.1000}{4.9400} & \ms{23.0400}{4.9300} \\ + & top 5\% & \ms{0.9900}{0.9300} & \ms{15.1300}{6.0800} & \ms{35.4800}{11.0200} & \ms{34.5000}{10.3700} \\ + & top 10\% & \ms{0.7800}{1.0500} & \ms{12.7400}{6.5600} & \ms{30.5300}{10.8800} & \ms{29.7500}{9.8700} \\ + & top 20\% & \ms{0.6700}{1.1000} & \ms{10.5300}{7.4300} & \ms{24.9400}{12.5800} & \ms{24.2800}{11.4900} \\ \addlinespace[1pt] - ClimaX & global & \ms{0.3480}{0.0754} & \ms{29.7535}{3.6073} & \ms{60.1506}{7.5865} & \ms{59.8026}{7.5454} \\ - & top 5\% & \ms{1.2937}{0.1086} & \ms{34.5791}{2.3772} & \ms{69.2186}{5.7215} & \ms{67.9249}{5.7263} \\ - & top 10\% & \ms{1.2522}{0.1602} & \ms{34.3341}{2.2852} & \ms{68.5713}{5.5377} & \ms{67.3191}{5.5538} \\ - & top 20\% & \ms{1.0287}{0.2686} & \ms{30.2140}{4.2857} & \ms{60.0650}{7.5674} & \ms{59.0363}{7.5891} \\ + ClimaX & global & \ms{0.3500}{0.0800} & \ms{29.7500}{3.6100} & \ms{60.1500}{7.5900} & \ms{59.8000}{7.5500} \\ + & top 5\% & \ms{1.2900}{0.1100} & \ms{34.5800}{2.3800} & \ms{69.2200}{5.7200} & \ms{67.9200}{5.7300} \\ + & top 10\% & \ms{1.2500}{0.1600} & \ms{34.3300}{2.2900} & \ms{68.5700}{5.5400} & \ms{67.3200}{5.5500} \\ + & top 20\% & \ms{1.0300}{0.2700} & \ms{30.2100}{4.2900} & \ms{60.0600}{7.5700} & \ms{59.0400}{7.5900} \\ \addlinespace[1pt] - StormCast & global & \ms{0.0626}{0.0119} & \ms{8.1951}{2.1895} & \ms{22.3817}{5.4294} & \ms{22.3191}{5.4178} \\ - & top 5\% & \ms{0.9573}{0.8011} & \ms{15.3219}{5.5337} & \ms{36.1857}{9.7331} & \ms{35.2284}{9.1816} \\ - & top 10\% & \ms{0.7284}{0.9280} & \ms{12.6669}{6.3290} & \ms{30.4748}{10.6527} & \ms{29.7464}{9.7494} \\ - & top 20\% & \ms{0.5795}{0.9104} & \ms{10.4157}{7.3437} & \ms{24.6598}{12.3973} & \ms{24.0803}{11.4988} \\ + StormCast & global & \ms{0.0560}{0.0110} & \ms{8.2000}{2.1900} & \ms{22.3800}{5.4300} & \ms{22.3200}{5.4200} \\ + & top 5\% & \ms{0.9600}{0.8000} & \ms{15.3200}{5.5300} & \ms{36.1900}{9.7300} & \ms{35.2300}{9.1800} \\ + & top 10\% & \ms{0.7300}{0.9300} & \ms{12.6700}{6.3300} & \ms{30.4700}{10.6500} & \ms{29.7500}{9.7500} \\ + & top 20\% & \ms{0.5800}{0.9100} & \ms{10.4200}{7.3400} & \ms{24.6600}{12.4000} & \ms{24.0800}{11.5000} \\ \addlinespace[1pt] - DLWP & global & \ms{0.1693}{0.0419} & \ms{14.9148}{3.2446} & \ms{28.1901}{6.9658} & \ms{28.0208}{6.9257} \\ - & top 5\% & \ms{1.8054}{0.4835} & \ms{31.7231}{3.2923} & \ms{55.4596}{5.2920} & \ms{53.6542}{5.4752} \\ - & top 10\% & \ms{1.6110}{0.5999} & \ms{27.6581}{5.9216} & \ms{47.1269}{8.0111} & \ms{45.5158}{7.7927} \\ - & top 20\% & \ms{1.5248}{0.8987} & \ms{20.9403}{4.7971} & \ms{34.9301}{7.8471} & \ms{33.4054}{7.8760} \\ + AlphaEarth & global & \ms{2.0600}{0.4400} & \ms{29.4500}{6.0100} & \ms{37.4300}{9.9500} & \ms{35.3700}{10.0300} \\ + & top 5\% & \ms{6.9100}{0.8500} & \ms{42.8800}{4.6100} & \ms{51.7400}{8.7300} & \ms{44.8300}{9.0800} \\ + & top 10\% & \ms{6.6400}{0.9900} & \ms{41.9000}{5.9500} & \ms{50.5700}{10.0100} & \ms{43.9300}{9.9200} \\ + & top 20\% & \ms{6.1900}{1.1300} & \ms{38.8300}{7.5000} & \ms{46.3800}{12.1700} & \ms{40.1900}{11.6800} \\ \addlinespace[1pt] - FCN & global & \ms{0.2829}{0.0839} & \ms{19.5061}{3.3412} & \ms{40.0604}{9.3701} & \ms{39.7775}{9.3423} \\ - & top 5\% & \ms{1.6231}{0.5064} & \ms{29.3769}{2.7626} & \ms{54.3033}{7.4089} & \ms{52.6801}{7.4389} \\ - & top 10\% & \ms{1.1777}{0.5118} & \ms{22.4217}{3.9803} & \ms{43.4510}{9.2513} & \ms{42.2734}{9.0251} \\ - & top 20\% & \ms{0.9962}{0.4315} & \ms{16.9792}{3.9371} & \ms{34.0859}{8.2616} & \ms{33.0897}{7.9275} \\ + DLWP & global & \ms{0.1700}{0.0400} & \ms{14.9100}{3.2400} & \ms{28.1900}{6.9700} & \ms{28.0200}{6.9300} \\ + & top 5\% & \ms{1.8100}{0.4800} & \ms{31.7200}{3.2900} & \ms{55.4600}{5.2900} & \ms{53.6500}{5.4800} \\ + & top 10\% & \ms{1.6100}{0.6000} & \ms{27.6600}{5.9200} & \ms{47.1300}{8.0100} & \ms{45.5200}{7.7900} \\ + & top 20\% & \ms{1.5200}{0.9000} & \ms{20.9400}{4.8000} & \ms{34.9300}{7.8500} & \ms{33.4100}{7.8800} \\ \addlinespace[1pt] - FengWu & global & \ms{0.2613}{0.0757} & \ms{12.0050}{6.0239} & \ms{24.1022}{13.6293} & \ms{23.8410}{13.5736} \\ - & top 5\% & \ms{1.5695}{0.3592} & \ms{16.2763}{3.7024} & \ms{30.1055}{5.0103} & \ms{28.5360}{4.7696} \\ - & top 10\% & \ms{1.2427}{0.5333} & \ms{12.9503}{5.6052} & \ms{24.1854}{8.6854} & \ms{22.9427}{8.1863} \\ - & top 20\% & \ms{1.1192}{0.5023} & \ms{11.9508}{5.0745} & \ms{22.7860}{7.9115} & \ms{21.6668}{7.4438} \\ + FCN & global & \ms{0.2800}{0.0800} & \ms{19.5100}{3.3400} & \ms{40.0600}{9.3700} & \ms{39.7800}{9.3400} \\ + & top 5\% & \ms{1.6200}{0.5100} & \ms{29.3800}{2.7600} & \ms{54.3000}{7.4100} & \ms{52.6800}{7.4400} \\ + & top 10\% & \ms{1.1800}{0.5100} & \ms{22.4200}{3.9800} & \ms{43.4500}{9.2500} & \ms{42.2700}{9.0300} \\ + & top 20\% & \ms{1.0000}{0.4300} & \ms{16.9800}{3.9400} & \ms{34.0900}{8.2600} & \ms{33.0900}{7.9300} \\ \addlinespace[1pt] - FuXi & global & \ms{0.3774}{0.1212} & \ms{21.0323}{4.8211} & \ms{37.2888}{9.4470} & \ms{36.9114}{9.4327} \\ - & top 5\% & \ms{2.0307}{0.6800} & \ms{31.8944}{4.7331} & \ms{53.9308}{8.3822} & \ms{51.9001}{8.6878} \\ - & top 10\% & \ms{1.6542}{0.7316} & \ms{24.0128}{5.7784} & \ms{40.2140}{9.9307} & \ms{38.5597}{9.7744} \\ - & top 20\% & \ms{1.3646}{0.6773} & \ms{21.9548}{5.8601} & \ms{36.7314}{10.0289} & \ms{35.3668}{9.9223} \\ + FengWu & global & \ms{0.2600}{0.0800} & \ms{12.0000}{6.0200} & \ms{24.1000}{13.6300} & \ms{23.8400}{13.5700} \\ + & top 5\% & \ms{1.5700}{0.3600} & \ms{16.2800}{3.7000} & \ms{30.1100}{5.0100} & \ms{28.5400}{4.7700} \\ + & top 10\% & \ms{1.2400}{0.5300} & \ms{12.9500}{5.6100} & \ms{24.1900}{8.6900} & \ms{22.9400}{8.1900} \\ + & top 20\% & \ms{1.1200}{0.5000} & \ms{11.9500}{5.0700} & \ms{22.7900}{7.9100} & \ms{21.6700}{7.4400} \\ \addlinespace[1pt] - Pangu-Weather & global & \ms{0.2755}{0.1089} & \ms{17.0909}{4.0477} & \ms{35.6386}{9.0327} & \ms{35.3630}{9.0774} \\ - & top 5\% & \ms{1.3656}{0.3064} & \ms{22.2222}{6.8613} & \ms{43.4234}{13.2383} & \ms{42.0578}{13.0599} \\ - & top 10\% & \ms{1.0931}{0.3535} & \ms{18.9337}{5.9329} & \ms{38.5325}{11.7221} & \ms{37.4394}{11.5261} \\ - & top 20\% & \ms{0.8844}{0.3601} & \ms{17.0172}{5.4859} & \ms{34.5688}{10.2932} & \ms{33.6844}{10.1334} \\ + FuXi & global & \ms{0.3800}{0.1200} & \ms{21.0300}{4.8200} & \ms{37.2900}{9.4500} & \ms{36.9100}{9.4300} \\ + & top 5\% & \ms{2.0300}{0.6800} & \ms{31.8900}{4.7300} & \ms{53.9300}{8.3800} & \ms{51.9000}{8.6900} \\ + & top 10\% & \ms{1.6500}{0.7300} & \ms{24.0100}{5.7800} & \ms{40.2100}{9.9300} & \ms{38.5600}{9.7700} \\ + & top 20\% & \ms{1.3600}{0.6800} & \ms{21.9500}{5.8600} & \ms{36.7300}{10.0300} & \ms{35.3700}{9.9200} \\ \addlinespace[1pt] - AlphaEarth & global & \ms{2.0606}{0.4404} & \ms{29.4476}{6.0064} & \ms{37.4286}{9.9458} & \ms{35.3679}{10.0271} \\ - & top 5\% & \ms{6.9133}{0.8450} & \ms{42.8790}{4.6087} & \ms{51.7449}{8.7321} & \ms{44.8315}{9.0763} \\ - & top 10\% & \ms{6.6366}{0.9901} & \ms{41.8981}{5.9454} & \ms{50.5712}{10.0057} & \ms{43.9346}{9.9156} \\ - & top 20\% & \ms{6.1908}{1.1330} & \ms{38.8325}{7.4966} & \ms{46.3833}{12.1697} & \ms{40.1925}{11.6788} \\ + Pangu-Weather & global & \ms{0.2800}{0.1100} & \ms{17.0900}{4.0500} & \ms{35.6400}{9.0300} & \ms{35.3600}{9.0800} \\ + & top 5\% & \ms{1.3700}{0.3100} & \ms{22.2200}{6.8600} & \ms{43.4200}{13.2400} & \ms{42.0600}{13.0600} \\ + & top 10\% & \ms{1.0900}{0.3500} & \ms{18.9300}{5.9300} & \ms{38.5300}{11.7200} & \ms{37.4400}{11.5300} \\ + & top 20\% & \ms{0.8800}{0.3600} & \ms{17.0200}{5.4900} & \ms{34.5700}{10.2900} & \ms{33.6800}{10.1300} \\ \bottomrule \end{tabular} + \end{adjustbox} \end{table*} diff --git a/paper_outputs/tables/tab_selection_regret_full_head.tex b/paper_outputs/tables/tab_selection_regret_full_head.tex new file mode 100644 index 0000000000000000000000000000000000000000..31f1a16d8792acc16ee2d4f27c119f27955008f4 --- /dev/null +++ b/paper_outputs/tables/tab_selection_regret_full_head.tex @@ -0,0 +1,2 @@ +% Full per-head rows are kept in the supplementary CSV files. +% The manuscript uses the all-backbone selection-regret summaries instead. diff --git a/paper_outputs/tables/tab_selection_regret_scope.tex b/paper_outputs/tables/tab_selection_regret_scope.tex new file mode 100644 index 0000000000000000000000000000000000000000..62e3549c0d2599148e0a5383f92988f050f2976c --- /dev/null +++ b/paper_outputs/tables/tab_selection_regret_scope.tex @@ -0,0 +1,24 @@ +\begin{table*}[!t] + \centering + \small + \setlength{\tabcolsep}{4pt} + \caption{Fixed-feature selection-regret check across evaluation scopes. Values are percentage-point regret \(\delta = D(h_D)-D(h_R)\) under union-\(F_1\), where \(h_R\) is selected by PR-AUC and \(h_D\) by the decision metric. Top-\(k\) columns use train-defined fire-prone scopes. Rows report mean with small std over five seeds; \(0.0000\) means the two selectors give the same decision score for all seeds.} + \label{tab:selection_regret_diagnostic} + \begin{tabular}{lcccc} + \toprule + \textbf{Feature source} & \textbf{\(\Omega=\)global} & \textbf{\(\Omega=\)top 5\%} & \textbf{\(\Omega=\)top 10\%} & \textbf{\(\Omega=\)top 20\%} \\ + \midrule + \textcolor{blue}{FireWx-FM ref.} & \ms{7.3831}{7.4536} & \ms{0.3664}{0.6812} & \ms{1.2275}{1.2665} & \ms{2.9385}{2.7513} \\ + Prithvi-WxC & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\ + Aurora & \ms{4.9455}{10.6974} & \ms{15.4283}{34.4987} & \ms{13.9934}{31.2903} & \ms{14.3706}{32.1337} \\ + ClimaX & \ms{0.1296}{0.1775} & 0.0000 & 0.0000 & 0.0000 \\ + StormCast & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\ + DLWP & 0.0000 & \ms{1.6716}{1.6079} & \ms{2.8465}{2.6938} & \ms{4.4634}{4.3561} \\ + FCN & 0.0000 & \ms{0.4510}{1.0071} & \ms{0.4200}{0.9390} & \ms{1.1680}{1.9872} \\ + FengWu & 0.0000 & \ms{0.8796}{0.5532} & \ms{0.4023}{0.5511} & \ms{0.5222}{0.6239} \\ + FuXi & 0.0000 & \ms{1.3545}{2.0970} & \ms{0.1656}{0.3703} & \ms{0.2833}{0.3681} \\ + Pangu-Weather & 0.0000 & \ms{0.7593}{0.8974} & \ms{0.3048}{0.5054} & \ms{0.1868}{0.3255} \\ + AlphaEarth & \ms{17.2217}{8.8492} & \ms{6.3846}{4.9653} & \ms{6.5738}{6.8970} & \ms{3.8804}{5.9483} \\ + \bottomrule + \end{tabular} +\end{table*} diff --git a/paper_outputs/tables/tab_selection_regret_scope_sweep.tex b/paper_outputs/tables/tab_selection_regret_scope_sweep.tex new file mode 100644 index 0000000000000000000000000000000000000000..397761ed00823310be4af0041178534708d7573d --- /dev/null +++ b/paper_outputs/tables/tab_selection_regret_scope_sweep.tex @@ -0,0 +1,24 @@ +\begin{table*}[!t] + \centering + \small + \setlength{\tabcolsep}{4pt} + \caption{Fixed-feature selection-regret sweep across evaluation scopes. Values are percentage-point regret \(\delta = D(h_D)-D(h_R)\) under union-\(F_1\). Top-\(k\) scopes are train-defined fire-prone masks. Rows report mean with small std over five seeds.} + \label{tab:selection_regret_scope_sweep} + \begin{tabular}{lcccc} + \toprule + \textbf{Feature source} & \textbf{\(\Omega=\)global} & \textbf{\(\Omega=\)top 5\%} & \textbf{\(\Omega=\)top 10\%} & \textbf{\(\Omega=\)top 20\%} \\ + \midrule + \textcolor{blue}{FireWx-FM ref.} & \ms{7.3831}{7.4536} & \ms{0.3664}{0.6812} & \ms{1.2275}{1.2665} & \ms{2.9385}{2.7513} \\ + Prithvi-WxC & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\ + Aurora & \ms{4.9455}{10.6974} & \ms{15.4283}{34.4987} & \ms{13.9934}{31.2903} & \ms{14.3706}{32.1337} \\ + ClimaX & \ms{0.1296}{0.1775} & 0.0000 & 0.0000 & 0.0000 \\ + StormCast & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\ + DLWP & 0.0000 & \ms{1.6716}{1.6079} & \ms{2.8465}{2.6938} & \ms{4.4634}{4.3561} \\ + FCN & 0.0000 & \ms{0.4510}{1.0071} & \ms{0.4200}{0.9390} & \ms{1.1680}{1.9872} \\ + FengWu & 0.0000 & \ms{0.8796}{0.5532} & \ms{0.4023}{0.5511} & \ms{0.5222}{0.6239} \\ + FuXi & 0.0000 & \ms{1.3545}{2.0970} & \ms{0.1656}{0.3703} & \ms{0.2833}{0.3681} \\ + Pangu-Weather & 0.0000 & \ms{0.7593}{0.8974} & \ms{0.3048}{0.5054} & \ms{0.1868}{0.3255} \\ + AlphaEarth & \ms{17.2217}{8.8492} & \ms{6.3846}{4.9653} & \ms{6.5738}{6.8970} & \ms{3.8804}{5.9483} \\ + \bottomrule + \end{tabular} +\end{table*} diff --git a/scripts/audit_release.py b/scripts/audit_release.py index 7a1011d1bea64c5267f3591514a70031f64d2e0a..1170985eda929c1bde9b34c6123014f89e4f82e5 100644 --- a/scripts/audit_release.py +++ b/scripts/audit_release.py @@ -1,8 +1,9 @@ #!/usr/bin/env python3 -"""Audit the public artifact release before upload.""" +"""Audit the public WildFIRE-FM release before upload.""" from __future__ import annotations +import json import re from pathlib import Path @@ -13,38 +14,31 @@ REQUIRED = [ "README.md", "LICENSE", "requirements.txt", - "artifacts/manifests/paper_outputs.yml", - "artifacts/manifests/paper_outputs.sha256", - "docs/artifact_map.md", "data_sources/DATA_SOURCES.md", + "models/wildfire_fm/README.md", + "models/wildfire_fm/modeling_unet.py", + "models/wildfire_fm/checkpoint_manifest.json", + "paper/manuscript_final.pdf", + "paper_outputs/figures/overview_wildfire.pdf", "paper_outputs/figures/matching.pdf", "paper_outputs/figures/fig_task_contract_tiles.pdf", - "paper_outputs/figures/fig_selection_regret_rq2.tikz", - "paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf", - "paper_outputs/figures/fig_task_rank_map.pdf", - "artifacts/results/fireprone_contract_progression_summary.json", - "artifacts/results/selection_regret_scope_sweep_20260505.csv", - "artifacts/results/selection_regret_rq2_figure_values.csv", + "paper_outputs/figures/fig_primary_rank_change_map.pdf", + "paper_outputs/figures/fig_selection_regret_scatter.pdf", + "paper_outputs/figures/fig_rank_heatmap1.pdf", + "assets/selection_regret_preview.png", + "assets/task_rank_map_preview.png", + "assets/primary_rank_change_preview.png", + "artifacts/manifests/paper_outputs.sha256", "scripts/check_paper_output_hashes.py", - "scripts/build_fireprone_contract_progression_figure.py", ] TABLE_LABELS = [ "tab_primary_results.tex", "tab_supporting_results.tex", - "tab_app_matching_rule_params.tex", - "tab_app_contract_params_full.tex", - "tab_app_scope_params.tex", "tab_fireprone_contract_progression.tex", + "tab_selection_regret_scope.tex", + "tab_selection_regret_scope_sweep.tex", "tab_appendix_selection_regret_tolerance.tex", - "tab_app_occupancy_ppr_scope.tex", - "tab_app_spread_ap_by_scope.tex", - "tab_app_burned_area_median_acre.tex", - "tab_app_analog_rank_depth.tex", - "tab_app_smoke_high_event.tex", - "tab_app_heat_event_pr.tex", - "tab_app_seed_robustness.tex", - "tab_app_head_architectures.tex", ] FORBIDDEN_TEXT = [ @@ -58,6 +52,7 @@ FORBIDDEN_TEXT = [ ] TEXT_SUFFIXES = {".md", ".py", ".sh", ".tex", ".csv", ".json", ".yml", ".yaml", ".txt"} +SKIP_FOR_FORBIDDEN = {"audit_release.py", "build_selection_regret_rq2_figure.py"} def iter_text_files() -> list[Path]: @@ -65,7 +60,7 @@ def iter_text_files() -> list[Path]: for path in ROOT.rglob("*"): if ".git" in path.parts or "__pycache__" in path.parts: continue - if path.name in {"audit_release.py", "build_selection_regret_rq2_figure.py"}: + if path.name in SKIP_FOR_FORBIDDEN: continue if path.is_file() and path.suffix in TEXT_SUFFIXES: out.append(path) @@ -88,13 +83,25 @@ def main() -> None: if token in text: issues.append(f"{path.relative_to(ROOT)} contains forbidden token {token!r}") - rq2 = ROOT / "paper_outputs/figures/fig_selection_regret_rq2.tikz" - if rq2.exists(): - text = rq2.read_text(errors="ignore") - if r"\begin{tikzpicture}" not in text: - issues.append("RQ2 TikZ output is missing tikzpicture") - if "Omega" in text or r"\Omega" in text: - issues.append("RQ2 TikZ still contains old legend text") + readme = (ROOT / "README.md").read_text(errors="ignore") + for phrase in ["WildFIRE-FM", "Model weights", "Data Used By The Study", "Loading A Checkpoint"]: + if phrase not in readme: + issues.append(f"README missing expected model-card phrase: {phrase}") + + manifest_path = ROOT / "models/wildfire_fm/checkpoint_manifest.json" + if manifest_path.exists(): + data = json.loads(manifest_path.read_text()) + checkpoints = data.get("checkpoints", []) + if len(checkpoints) != 5: + issues.append("checkpoint manifest should list five seeded checkpoints") + for item in checkpoints: + rel = item.get("filename", "") + if not rel.startswith("models/wildfire_fm/checkpoints/seed_"): + issues.append(f"unexpected checkpoint filename in manifest: {rel}") + if "source_path" in item: + issues.append("checkpoint manifest exposes source_path") + if not re.fullmatch(r"[0-9a-f]{64}", str(item.get("sha256", ""))): + issues.append(f"bad sha256 in checkpoint manifest: {item}") for path in (ROOT / "paper_outputs/tables").glob("*.tex"): text = path.read_text(errors="ignore") @@ -103,7 +110,7 @@ def main() -> None: checksum_manifest = ROOT / "artifacts/manifests/paper_outputs.sha256" if checksum_manifest.exists(): - listed = [] + listed: list[str] = [] for line in checksum_manifest.read_text(errors="ignore").splitlines(): if not line.strip(): continue @@ -115,7 +122,11 @@ def main() -> None: listed.append(rel) if not (ROOT / rel).exists(): issues.append(f"checksum manifest lists missing output: {rel}") - expected = sorted(str(p.relative_to(ROOT)) for p in (ROOT / "paper_outputs").rglob("*") if p.is_file()) + expected_paths = [] + for rel_root in ["paper_outputs", "assets"]: + expected_paths.extend(str(p.relative_to(ROOT)) for p in (ROOT / rel_root).rglob("*") if p.is_file()) + expected_paths.append("paper/manuscript_final.pdf") + expected = sorted(set(expected_paths)) if sorted(listed) != expected: missing = sorted(set(expected) - set(listed)) extra = sorted(set(listed) - set(expected)) diff --git a/scripts/reproduce_paper_outputs.py b/scripts/reproduce_paper_outputs.py index e50a17b3620c209743d36ef7f448835a89ae3522..870614b533a3057fa4bf25a2d105c8e47463eb55 100644 --- a/scripts/reproduce_paper_outputs.py +++ b/scripts/reproduce_paper_outputs.py @@ -1,9 +1,10 @@ #!/usr/bin/env python3 -"""Rebuild release paper outputs from bundled lightweight artifacts. +"""Verify the released WildFIRE-FM paper artifacts. -This release intentionally does not include raw HRRR, FIRMS, LANDFIRE, -WFIGS, MTBS, WRC, or LandScan data. The script rebuilds artifacts that can -be reproduced from the small summary files bundled in this repository. +The final paper figures/tables in this Hub release are copied from the current +manuscript bundle. Raw-data reruns are intentionally outside this lightweight +check because the public repository does not redistribute source data or local +feature caches. """ from __future__ import annotations @@ -22,12 +23,9 @@ def run(cmd: list[str]) -> None: def main() -> None: - run([sys.executable, "scripts/build_fireprone_contract_progression_figure.py"]) - run([sys.executable, "scripts/build_selection_regret_rq2_figure.py"]) - run([sys.executable, "scripts/build_task_rank_map.py"]) run([sys.executable, "scripts/check_paper_output_hashes.py"]) run([sys.executable, "scripts/audit_release.py"]) - print("Rebuilt reproducible outputs and passed release audit.") + print("Verified final paper outputs and release audit.") if __name__ == "__main__":