Spaces:

---

calibrationcomp

/

calibration_benchmark

like 0

Sleeping

The Primary purposes are:

• To enable a UQ leaderboard (on a new page)
• Adding some claude skills to help further iterations

The necessary work-pieces to achieve this:

• A new race-condition based on a score called the output coverage (replacing RMSE). Whereby we treat the output space as independent trials, and see what budgets/iterations will capture the truth accurately with a quantile. (e.g. 0.15 quantile caputres the truth 15% of the time), we require 3 quantiles (0.15,0.5,0.85) all be suitably approximated to achieve the race condition. a baseline target accuracy of the quantile is adapted to the quantile itself, and all are adjustable with a constant scaling to loosen the race condition.
• The plotted "metric" is the budget, # iterations (and %failures) on achieving a race condition for the different ensemble sizes and "C"
• We add the ces-eki-dmc data files
• We add some presentation defaults for ensemble sizes where all fail to reach the target.

Misc:

• We allow users to toggle methods, and assign a fixed color to each
• We add the race condition to each leaderboard in some latex.
• add some gitignore and a readme.
• added some figures to the home-page and improved the descriptions there