File size: 2,754 Bytes
569ea0c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | {
"schema_version": "1.0",
"title": "Reproduction: Conditional Coverage Diagnostics for Conformal Prediction",
"space_id": "snaykey/repro-conditional-coverage-diagnostics",
"paper": {
"arxiv_id": "2512.11779",
"openreview_id": "vaApZm6MKM"
},
"tags": [
"icml2026-repro",
"paper-vaApZm6MKM"
],
"updated_at": "2026-07-29T11:00:00+00:00",
"root": {
"slug": "index",
"title": "Reproduction: Conditional Coverage Diagnostics for Conformal Prediction",
"children": [
{
"slug": "executive-summary",
"title": "Executive summary",
"children": []
},
{
"slug": "claim-1-ert-family",
"title": "The paper introduces a family of Excess Risk of the Target Coverage (ERT) metrics—L1-ERT, L2-ERT, and KL-ERT—for evaluating conditional coverage in conformal prediction, built on the principle that no classifier can outperform a constant 1-alpha predictor under perfect conditional coverage (Table 1, Section 3.1).",
"children": []
},
{
"slug": "claim-2-classifier-power",
"title": "Using LightGBM as the underlying classifier for L1-ERT achieves 68.4% relative statistical power (relative to maximum), compared to only 38.3% for the PartitionWise classifier underlying CovGap (Table 2, Section 4.1).",
"children": []
},
{
"slug": "claim-3-sample-efficiency",
"title": "In synthetic experiments, group-based metrics like CovGap remain unreliable and unaligned with their theoretical values even at 5,000 test points, whereas ERT metrics such as L1-ERT converge with far fewer samples (Figure 4, Section 4.2).",
"children": []
},
{
"slug": "claim-4-asymmetric-components",
"title": "The metrics decompose conditional coverage error into asymmetric components ell_plus-ERT and ell_minus-ERT, separating unnecessary conservatism (over-coverage) from excessive aggressiveness (under-coverage) (Section 3.3).",
"children": []
},
{
"slug": "claim-5-classification-decomposition",
"title": "Classification experiments report divergent KL_plus-ERT and KL_minus-ERT values across conformal prediction methods, demonstrating the over/under-coverage decomposition in practice (Table 4, Section 4.3.2).",
"children": []
},
{
"slug": "claim-6-cross-validation",
"title": "Algorithm 1 estimates the ERT metrics from finite samples using k-fold cross-validation to avoid overfitting the classifier used in the estimation (Algorithm 1).",
"children": []
},
{
"slug": "conclusion",
"title": "Conclusion",
"children": []
}
]
},
"revision": 1
}
|