| --- |
| title: Reproduction - Conditional Coverage Diagnostics for Conformal Prediction |
| emoji: 📐 |
| colorFrom: green |
| colorTo: gray |
| sdk: static |
| pinned: false |
| short_description: All 6 claims tested, arXiv 2512.11779 |
| tags: |
| - trackio |
| - open-reproductions |
| - icml2026-repro |
| - paper-vaApZm6MKM |
| --- |
| |
| # Reproduction bundle |
|
|
| Reproduction of **"Conditional Coverage Diagnostics for Conformal Prediction"** |
| (arXiv:2512.11779v1, OpenReview `vaApZm6MKM`). |
|
|
| The ERT family is implemented from Table 1 -- L1, L2 (Brier) and KL, each as |
| `R(1-alpha) - R(h)` for a cross-fitted classifier `h` of the coverage |
| indicator -- together with the asymmetric split of Section 3.3, the k-fold |
| estimator of Algorithm 1, and CovGap for comparison. |
|
|
| For the synthetic experiments the data-generating process is chosen so the |
| **true** conditional coverage p(x) is available in closed form, so every |
| estimate is scored against ground truth rather than against another estimate. |
| The oracle interval gives p(x) = 1-alpha exactly and is the negative control |
| that each metric must return ~0 on. |
|
|
| Claims 2 and 5 are audits of the authors' own released outputs |
| (`ElSacho/Conditional_Coverage_Estimation`), which are vendored under |
| `authors/`: 2,800 v1 rows, 5,600 v2 rows and 80 classification rows. |
|
|
| ```bash |
| python3 run_claims.py # all six claims |
| python3 build_pages.py # regenerate pages/ |
| ``` |
|
|
