File size: 1,881 Bytes
ebbad77 1b7a999 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | <!doctype html>
<meta charset="utf-8">
<title>Reproduction: Conditional Coverage Diagnostics for Conformal Prediction</title>
<style>
body{font:16px/1.6 system-ui,sans-serif;max-width:54rem;margin:3rem auto;padding:0 1.25rem;background:#0f1116;color:#e6e8ee}
a{color:#7aa2f7}code{background:#1a1d26;padding:.15em .4em;border-radius:4px}
th,td{border:1px solid #2a2f3a;padding:.4rem .6rem;text-align:left}
table{border-collapse:collapse;margin:1rem 0}h1{font-size:1.4rem}
.ok{color:#9ece6a}
</style>
<h1>Conditional Coverage Diagnostics for Conformal Prediction — reproduction</h1>
<p>arXiv:<a href="https://arxiv.org/abs/2512.11779">2512.11779</a>v1 · OpenReview <code>vaApZm6MKM</code></p>
<p>The synthetic design is chosen so the <em>true</em> conditional coverage
p(x) has a closed form, so every estimate is scored against ground truth
rather than against another estimate. The oracle interval gives p(x)=1−α
exactly and is the negative control each metric must return ≈0 on.</p>
<table>
<tr><th>Claim</th><th>Verdict</th><th>Headline</th></tr>
<tr><td>1 ERT family + constant-predictor floor</td><td class="ok">reproduced</td><td>oracle L1-ERT +0.0012; recovers 90% of true L1</td></tr>
<tr><td>2 LightGBM 68.4% vs PartitionWise 38.3%</td><td class="ok">reproduced</td><td>68.04% / 37.82% from released rows</td></tr>
<tr><td>3 ERT converges, CovGap does not</td><td class="ok">reproduced</td><td>CovGap error grows with n; ERT separates 14× better</td></tr>
<tr><td>4 Over/under-coverage decomposition</td><td class="ok">reproduced</td><td>60× ratio on a known-conservative predictor</td></tr>
<tr><td>5 Divergent KL± in classification</td><td class="ok">reproduced</td><td>residual 8.9e−8; 8/8 cells divergent</td></tr>
<tr><td>6 Algorithm 1 cross-fitting</td><td class="ok">reproduced</td><td>in-sample fabricates 0.177 on a valid predictor</td></tr>
</table>
|