Spaces:

---

calibrationcomp

/

calibration_benchmark

Like 0

Running

In this PR we
Add ABC:

• add the method and runs into the UQ leaderboard
• Cap ABC to 10,000 experiments, and if we havent drawn a minimum number of posterior samples (dim+2) then the experiment is considered a failure. (main cause of fail in high dims)
• remove older ABC (And HM) methods from the optimization experiment (leave data for now)

Add score-based adam

• add the KGMM results (most promising from LM experiments) to the optimization leaderboard
• experiment adapted alpha scaled by the prior.
• We note there are many ways to play with hyperparameters of ADAM, for now we have done modest tuning - happy for these to be updated in future. Note that increasing/decreasing alpha improved some cases and made others worse so it is not as simple as global resclaes.

MISC

• full NaNs weren't displaying correctly, (due to a special case in the original way the EKP experiment data was handled (using NaN to mean not-run, not run-and-all-failed). Therefore we resolved this by splitting up the data files for these experiments.