paper.txt

 
 
 
 
 Conditional Coverage Diagnostics for Conformal Prediction 
 
 
 
 
 
 
 
 
 
 
 
 
 
 1 Introduction 
 
 Contributions. 
 Background on conformal prediction. 
 Notation. 
 
 
 
 2 Related Work on Evaluating Conditional Coverage 
 
 Group-based diagnostics. 
 Worst-case slab diagnostic. 
 
 
 
 3 Evaluating Conditional Coverage 
 
 
 3.1 Excess risk of the target coverage (ERT) 
 
 Interpretation and examples. 
 Estimation from finite samples. 
 
 
 3.2 Estimating general distances 
 3.3 Separating over-coverage and under-coverage 
 
 3.4 Extensions 
 
 A proxy for conditional coverage. 
 Evaluating conditional coverage rules. 
 
 
 
 
 
 4 Experiments 
 
 
 4.1 Comparing different classifiers 
 
 Results. 
 Differences between metrics. 
 
 
 
 4.2 Comparison with existing metrics 
 
 Results. 
 
 
 
 4.3 Real datasets 
 
 
 4.3.1 Regression 
 
 Multivariate results. 
 
 
 4.3.2 Classification 
 
 
 
 
 5 Conclusions 
 
 A Additional metrics 
 
 Group-based diagnostics. 
 Representation-based diagnostics (dependence on auxiliary variables). 
 
 
 
 B Estimating Bregman divergences with ERTs 
 
 
 C Link with gibbs2025conformal 
 
 
 D Extension to a conditional coverage rule 
 
 
 E Benchmarking strategies 
 
 
 F Strong classifiers 
 
 Tabular foundation models. 
 Gradient-boosted decision trees. 
 Bagging models. 
 PartitionWise 
 Trade-offs. 
 Discussion and other options. 
 
 Hardware. 
 
 
 G Additional information regarding the experiments 
 
 
 G.1 Details on the datasets 
 
 
 H Additional experiments 
 
 Synthetic dataset 
 Uni-variate regression. 
 Multivariate regression 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 1 1 affiliationtext: Sierra team, Inria Paris, France 
 {sacha.braun, francis.bach}@inria.fr 2 2 affiliationtext: Ecole Normale Supérieure, PSL Research University, Paris 3 3 affiliationtext: Soda team, Inria Paris-Saclay, France 
 david.holzmuller@inria.fr 4 4 affiliationtext: Departments of EECS and Statistics, UC Berkeley, USA 
 jordan@cs.berkeley.edu 
 Conditional Coverage Diagnostics for Conformal Prediction 
 
 
 Sacha Braun
 
 
 David Holzmüller
 
 
 Michael I. Jordan
 
 
 Francis Bach
 
 
 
 Abstract 
 Evaluating conditional coverage remains one of the most persistent challenges in assessing the reliability of predictive systems.
Although conformal methods can give guarantees on marginal coverage, no method can guarantee to produce sets with correct conditional coverage, leaving practitioners without a clear way to interpret local deviations.
To overcome sample-inefficiency and overfitting issues of existing metrics, we cast conditional coverage estimation as a classification problem. Conditional coverage is violated if and only if any classifier can achieve lower risk than the target coverage. Through the choice of a (proper) loss function, the resulting risk difference gives a conservative estimate of natural miscoverage measures such as L1 and L2 distance, and can even separate the effects of over- and under-coverage, as well as handle non-constant target coverages. We call the resulting family of metrics excess risk of the target coverage (ERT). We show experimentally that the use of modern classifiers provides much higher statistical power than simple classifiers underlying established metrics like CovGap.
Additionally, we use our metric to benchmark different conformal prediction methods.
Finally, we release an open-source package for ERT as well as previous conditional coverage metrics. Together, these contributions provide a new lens for understanding, diagnosing, and improving the conditional reliability of predictive systems. 
 
 
 
 1 Introduction 
 
 Uncertainty quantification is central to decision-making across science, engineering, and policy. In many applications, the goal is not a single point prediction but a set of plausible outcomes with a desired confidence level. This is formalized by a predictive set rule C ​ ( ⋅ ) C(\cdot) , which outputs a region expected to contain the true outcome with a desired probability. Such predictive sets capture data noise, model imperfections, and variability, supporting safer decisions and clearer communication of model confidence. 
 
 
 Conformal prediction (CP) offers a general framework for constructing prediction sets with finite-sample coverage guarantees ( vovk2005algorithmic ; shafer2008tutorial ) . Its only requirement is that the available data samples are exchangeable: a condition even weaker than the standard independent and identically distributed (i.i.d.) assumption.
In recent years, CP has rapidly emerged as a go-to tool for adding rigorous, model-agnostic uncertainty estimates to modern black-box predictors ( angelopoulos2021gentle ) . This makes it especially appealing for scientific and industrial applications where formal guarantees on model predictions are essential. 
 
 
 CP’s simplicity hides an important drawback: it only guarantees marginal coverage , which means that the constructed prediction set contains the true outcome with probability 1 − α 1-\alpha on average across the population. That is, the coverage is right on average, but not necessarily for each individual. In practice one often desires conditional coverage ; i.e., asking that the coverage guarantee holds not only on average but also for specific subpopulations or feature values. 
 
 
 Achieving exact conditional coverage is impossible in general without strong distributional assumptions ( vovk2012conditional ; lei2014distribution ; foygel2021limits ) , and even approximate versions are notoriously difficult to deploy. Improving conditional coverage in CP typically requires carefully designed nonconformity scores.
Common strategies rely on models that provide uncertainty estimates, such as quantile regression ( romano2019conformalized ) , predictive distributions ( izbicki2022cd ; braun2025multivariate ) , or local score adjustments ( guan2023localized ; messoudi2022ellipsoidal ; thurin2025optimal ) .
Recent work proposes post hoc corrections that directly model conditional quantiles of the nonconformity score ( plassier2025rectifying ) .
There is, however, a difficulty in assessing whether progress is being made in this literature, which is the lack of standard way to evaluate conditional coverage and thereby compare algorithms. The fundamental problem of the evaluation of conditional coverage is the focus of the current paper. 
 
 
 Evaluating conditional coverage is difficult.
Group-based diagnostics, such as fairness-style coverage gaps ( ding2023class ) , require large sample sizes per group and are highly sensitive to group definitions.
Geometric scans such as worst-case slab coverage (WSC, cauchois2021knowing ) offer a more adaptive view but suffer from severe sample complexity in high dimensions.
Dependence-based diagnostics ( feldman2021improving ) capture correlations between coverage and auxiliary variables but do not provide a standalone notion of conditional validity.
In short, there is still no robust, general-purpose metric for assessing conditional coverage in practice. 
 
 
 Contributions. 
 
 We address this gap by reframing conditional coverage evaluation as a supervised prediction task: given features X ∈ 𝒳 X\in\mathcal{X} , predict whether the label Y ∈ 𝒴 Y\in\mathcal{Y} falls inside the predictive set C α ​ ( X ) C_{\alpha}(X) , where ( X , Y ) ∼ ℙ X , Y (X,Y)\sim{\mathbb{P}}_{X,Y} . Under perfect conditional coverage, for any proper loss ℓ \ell the Bayes-optimal predictor, h : 𝒳 → 𝒴 h:\mathcal{X}\to\mathcal{Y} , which is defined as the minimizer of the risk 𝔼 [ ℓ ( h ( X ) , 𝟙 { Y ∈ C α ( X ) } ] {\mathbb{E}}[\ell(h(X),{\mathbbm{1}}\{Y\in C_{\alpha}(X)\}] 
is the constant 1 − α 1-\alpha . Consequently, any predictor that consistently outperforms this constant directly exposes a violation of conditional coverage.
Building on this insight, we introduce the excess risk of the target coverage ( ℓ ​ -ERT \ell\textrm{-ERT} ) metric to quantify deviations from conditional validity. Our metric provides an estimate of 𝔼 X ​ [ D φ ​ ( p ​ ( X ) ∥ 1 − α ) ] {\mathbb{E}}_{X}[D_{\varphi}(p(X)\|1-\alpha)] where D φ D_{\varphi} is the Bregman divergence of a convex function φ \varphi ( bregman1967relaxation ) . For instance, we can reliably estimate the quantity 𝔼 [ | 1 − α − ℙ ( Y ∈ C α ( X ) | X ) | ] \mathbb{E}[\,|1-\alpha-{\mathbb{P}}(Y\in C_{\alpha}(X)|X)|\,] . 
 
 
 To contextualize our contribution, we establish a formal connection between existing group-based diagnostics and our metrics, showing that our formulation generalizes partition-based estimators to arbitrary predictor classes. This unified perspective unlocks the full potential of functional estimation, integrating both parametric and nonparametric approaches to assess conditional coverage, moving beyond heuristic group evaluations toward principled, model-based inference. 
 
 
 Through experiments, we demonstrate that our proposed conditional coverage metrics are empirically more robust, that is, less prone to misleading diagnostics, than existing alternatives. Finally, we benchmark several conformal prediction methods on real-world regression and classification tasks to compare their conditional coverage performance. To support reproducibility and to catalyze further progress in conformal prediction, we release covmetrics 1 1 1 https://github.com/ElSacho/covmetrics , an open-source package for evaluating conditional coverage using our approach alongside established metrics. 
 
 
 Overall, this work casts conditional coverage evaluation as a key missing piece for practical conformal prediction, and offers new tools to fill the gap. 
 
 
 
 Background on conformal prediction. 
 
 We begin with a brief overview of split conformal prediction , a widely used and computationally simple approach for constructing prediction sets with marginal validity guarantees ( papadopoulos2002inductive ; lei2018distribution ; angelopoulos2021gentle ) . Suppose we observe data 𝒟 = { ( X i , Y i ) } i = 1 n \mathcal{D}=\{(X_{i},Y_{i})\}_{i=1}^{n} sampled i.i.d. from a joint distribution ℙ X , Y \mathbb{P}_{X,Y} , where X i ∈ 𝒳 X_{i}\in\mathcal{X} are feature vectors and Y i ∈ 𝒴 Y_{i}\in\mathcal{Y} are outcomes. For a new test pair, ( X test , Y test ) (X_{\text{test}},Y_{\text{test}}) , the aim is to form a predictive set C α ​ ( X test ) C_{\alpha}(X_{\text{test}}) such that 
 
 
 
 ℙ X , Y ​ ( Y test ∈ C α ​ ( X test ) ) = 1 − α . \displaystyle{\mathbb{P}}_{X,Y}\!\left(Y_{\text{test}}\in C_{\alpha}(X_{\text{test}})\right)=1-\alpha. 
 
 
 
 
 
 To achieve this, the dataset is randomly divided into two parts: a training set 𝒟 1 \mathcal{D}_{1} of size n 1 n_{1} and a calibration set 𝒟 2 \mathcal{D}_{2} of size n 2 n_{2} . A predictive model is fit on 𝒟 1 \mathcal{D}_{1} , while 𝒟 2 \mathcal{D}_{2} is reserved for calibrating the prediction sets. Central to the procedure is a nonconformity score S ​ ( X , Y ) ∈ ℝ S(X,Y)\in\mathbb{R} , which quantifies how atypical a candidate response Y Y is relative to the model. Using these scores, the predictive sets are then computed as: 
 
 
 
 C α ​ ( X test ) = { y : S ​ ( X test , y ) ≤ q ^ α } , C_{\alpha}(X_{\text{test}})=\left\{y:S(X_{\text{test}},y)\leq\hat{q}_{\alpha}\right\}, 
 
 (1) 
 
 
 with 
 
 
 
 q ^ α := Q 1 − α ​ ( 1 n 2 + 1 ​ ∑ k = 1 n 2 δ S ​ ( X k , Y k ) + δ ∞ n 2 + 1 ) , \hat{q}_{\alpha}:=Q_{1-\alpha}\!\left(\frac{1}{n_{2}+1}\sum_{k=1}^{n_{2}}\delta_{S(X_{k},Y_{k})}+\frac{\delta_{\infty}}{\,n_{2}+1}\right), 
 
 (2) 
 
 
 where Q 1 − α ​ ( ℙ ) Q_{1-\alpha}({\mathbb{P}}) returns the 1 − α 1-\alpha quantile of the distribution ℙ {\mathbb{P}} , and δ x \delta_{x} is the Dirac measure centered at x x .
By the exchangeability of the data, this construction guarantees the marginal coverage property: 
 
 
 
 ℙ X , Y ​ ( Y test ∈ C α ​ ( X test ) ∣ 𝒟 1 ) ∈ [ 1 − α , 1 − α + 1 n 2 + 1 ) . {\mathbb{P}}_{X,Y}\!\left(Y_{\text{test}}\in C_{\alpha}(X_{\text{test}})\mid\mathcal{D}_{1}\right)\in\Bigg[1-\alpha,1-\alpha+\cfrac{1}{n_{2}+1}\,\Bigg). 
 
 
 
 
 
 It is important to note that the guarantee in Eq. ( 1 ) is marginal, which means that coverage holds on average over the distribution of X test X_{\textrm{test}} and 𝒟 2 \mathcal{D}_{2} . A stronger requirement is conditional coverage , which demands 
 
 
 
 ℙ Y | X ( Y test ∈ C α ( X test ) | X test ) = 1 − α , \displaystyle{\mathbb{P}}_{Y|X}\!\left(Y_{\text{test}}\in C_{\alpha}(X_{\text{test}})\,\middle|\,X_{\text{test}}\right)=1-\alpha, 
 
 (3) 
 
 
 for almost every X test X_{\text{test}} , but achieving ( 3 ) is impossible in general without additional assumptions. For a given conformal prediction strategy that achieves marginal coverage ( 1 ), it is essential to be able to measure how close from a conditional guarantee ( 3 ) we are. 
 
 
 
 Notation. 
 
 We denote by 𝟙 x ∈ A {\mathbbm{1}}_{x\in A} the indicator function, equal to 1 if x ∈ A x\in A and to 0 if x ∉ A x\notin A , for
some set A A . We denote by sgn ⁡ ( x ) \operatorname{sgn}(x) the function that returns the sign of x ∈ ℝ x\in\mathbb{R} , where sgn ⁡ ( 0 ) = 0 \operatorname{sgn}(0)=0 . We write Δ d := { p ∈ ℝ d | p i ≥ 0 ​ ∀ i = 1 , … , d , ∑ i = 1 d p i = 1 } \Delta_{d}:=\Big\{p\in\mathbb{R}^{d}\;\Big|\;p_{i}\geq 0\ \forall i=1,\dots,d,\ \sum_{i=1}^{d}p_{i}=1\Big\} to denote the probability simplex. 
 
 
 
 
 
 2 Related Work on Evaluating Conditional Coverage 
 
 In the following, we assume that a predictive model has already been trained to produce a predictive set rule C α ​ ( ⋅ ) C_{\alpha}(\cdot) for the output variable Y ∈ 𝒴 Y\in\mathcal{Y} , given a feature vector X ∈ 𝒳 X\in\mathcal{X} .
We are given a test dataset, 𝒟 test = { ( X i , Y i ) } i = 1 m \mathcal{D}_{\text{test}}=\{(X_{i},Y_{i})\}_{i=1}^{m} , to evaluate the conditional coverage of this predictive strategy.
The test samples are assumed to be sampled i.i.d. from the distribution ℙ X , Y {\mathbb{P}}_{X,Y} and unseen during training. 
 
 
 Group-based diagnostics. 
 
 A common way to study conditional coverage for a predictive set C α ​ ( ⋅ ) C_{\alpha}(\cdot) is to evaluate coverage over subpopulations or other partitions of the data. To make this concrete, fix a finite set of groups 𝒢 \mathcal{G} and a mapping g : 𝒳 → 𝒢 g:\mathcal{X}\to\mathcal{G} that assigns each feature x ∈ 𝒳 x\in\mathcal{X} , to a group g ​ ( x ) ∈ 𝒢 g(x)\in\mathcal{G} . For a given group g ∈ 𝒢 \textbf{g}\in\mathcal{G} and a test sample with indices ℐ g = { i : g ​ ( X i ) = g } \mathcal{I}_{\textbf{g}}=\{i:g(X_{i})=\textbf{g}\} , the empirical coverage in group g is 
 
 
 
 C g = 1 | ℐ g | ​ ∑ i ∈ ℐ g 𝟙 ​ { Y i ∈ C α ​ ( X i ) } . C_{\textbf{g}}\;=\;\frac{1}{|\mathcal{I}_{\textbf{g}}|}\sum_{i\in\mathcal{I}_{\textbf{g}}}{\mathbbm{1}}\{Y_{i}\in C_{\alpha}(X_{i})\}. 
 
 
 
 We review several strategies for defining these groups in Appendix A . Unless stated otherwise, in the following, the groups are obtained by clustering the feature space using the k-means algorithm. Below we review strategies that use such groupings to diagnose conditional miscoverage. 
 
 
 
 
 • 
 
 Coverage gap (CovGap). This measure the average absolute deviation from the target coverage across groups, 
 
 
 
 CovGap = 1 | 𝒢 | ​ ∑ 𝐠 ∈ 𝒢 | C g − ( 1 − α ) | . \text{CovGap}\;=\;\frac{1}{|\mathcal{G}|}\sum_{\mathbf{g}\in\mathcal{G}}\big|C_{\textbf{g}}-(1-\alpha)\big|. 
 
 
 
 This metric is one of the most commonly used metrics in the literature (see, e.g, ding2023class ; kaur2025conformal ; zhu2025predicate ; fillioux2024foundation ; liu2025conformal ). A problem is that CovGap requires a large number of samples within each group to be consistent. 
 
 
 
 • 
 
 Weighted coverage gap (WCovGap). To connect CovGap to our main metrics, we introduce its corrected version that assigns a weight to each group’s coverage gap: 
 
 
 
 WCovGap = ∑ 𝐠 ∈ 𝒢 | ℐ 𝐠 | m ​ | C 𝐠 − ( 1 − α ) | . \textrm{WCovGap}\;=\;\sum_{\mathbf{g}\in\mathcal{G}}\frac{|\mathcal{I}_{\mathbf{g}}|}{m}\,\big|C_{\mathbf{g}}-(1-\alpha)\big|. 
 
 
 
 
 
 This formulation highlights that this metric can be interpreted as nonparametric estimators of the quantity, 
 
 
 
 L 1 -Miscoverage := 𝔼 X [ | ℙ Y | X ( Y ∈ C α ( X ) ∣ X ) − ( 1 − α ) | ] . \text{L}_{1}\text{-Miscoverage}:=\mathbb{E}_{X}\big[\,\big|\mathbb{P}_{Y|X}(Y\in C_{\alpha}(X)\mid X)-(1-\alpha)\big|\,\big]. 
 
 
 
 Indeed, CovGap ​ ( X ) := C g ​ ( X ) \mathrm{CovGap}(X):=C_{g(X)} is an estimate of ℙ ​ ( Y ∈ C α ​ ( X ) ∣ g ​ ( X ) = 𝐠 ) \mathbb{P}(Y\in C_{\alpha}(X)\mid g(X)=\mathbf{g}) , and | ℐ 𝐠 | m \frac{|\mathcal{I}_{\mathbf{g}}|}{m} an estimate of ℙ X ​ ( g ​ ( X ) = 𝐠 ) {\mathbb{P}}_{X}(g(X)=\mathbf{g}) . Under standard regularity assumptions, specifically, if the partition 𝒢 \mathcal{G} becomes increasingly fine (i.e., the number of groups tends to infinity while their diameters shrink to zero) and if h ∗ ​ ( X ) := 𝔼 ​ [ 1 Y ∈ C α ​ ( X ) ∣ X ] h^{*}(X):=\mathbb{E}[1_{Y\in C_{\alpha}(X)}\mid X] is Lipschitz-continuous, then the groupwise coverage C 𝐠 C_{\mathbf{g}} converges to the true conditional coverage ℙ Y | X ​ ( Y ∈ C α ​ ( X ) ∣ X ) \mathbb{P}_{Y|X}(Y\in C_{\alpha}(X)\mid X) (see, e.g, gyorfi2005distribution ; bach2024learning ). Consequently, the weighted CovGap (WCovGap) metric provides a nonparametric estimate of the stated quantity. If the groups are balanced in size, the metric CovGap admits the same type of probabilistic interpretation. 
 
 
 This observation is central to understanding the positioning of our work: previous strategies can be seen as partition-wise estimators of conditional coverage. In contrast, our approach leverages modern classifiers to obtain a more accurate estimation of conditional coverage. 
 
 
 
 
 
 
 Worst-case slab diagnostic. 
 
 Rather than pre-specified groups, some diagnostics scan geometric slices of the feature space, if 𝒳 ⊂ ℝ d \mathcal{X}\subset\mathbb{R}^{d} . This strategy is commonly refer to as the worst-case slab coverage (WSC, cauchois2021knowing ) . 
 
 
 
 
 • 
 
 Worst-case slab coverage (WSC). For a direction v ∈ ℝ d v\in\mathbb{R}^{d} and scalars a < b a<b , define the slab 
 
 
 
 S v , a , b := { x ∈ ℝ d : a ≤ v ⊤ ​ x ≤ b } . S_{v,a,b}:=\{x\in\mathbb{R}^{d}:a\leq v^{\top}x\leq b\}. 
 
 
 
 For a mass threshold δ ∈ ( 0 , 1 ] \delta\in(0,1] let ℐ v , a , b = { i : X i ∈ S v , a , b } \mathcal{I}_{v,a,b}=\{i:X_{i}\in S_{v,a,b}\} . The empirical WSC in direction v v is 
 
 
 
 WSC n ​ ( C α ​ ( ⋅ ) , v ) := inf a < b { 1 | ℐ v , a , b | ​ ∑ i ∈ ℐ v , a , b 𝟙 ​ { Y i ∈ C α ​ ( X i ) } | | ℐ v , a , b | n ≥ δ } . \text{WSC}_{n}\big(C_{\alpha}(\cdot),v\big)\;:=\;\inf_{a<b}\left\{\frac{1}{|\mathcal{I}_{v,a,b}|}\sum_{i\in\mathcal{I}_{v,a,b}}{\mathbbm{1}}\{Y_{i}\in C_{\alpha}(X_{i})\}\;\middle|\;\frac{|\mathcal{I}_{v,a,b}|}{n}\geq\delta\right\}. 
 
 
 
 In practice, the induced metric requires a finite set of directions V V and computes: 
 
 
 
 WSC = inf v ∈ V WSC n ​ ( C α ​ ( ⋅ ) , v ) . \displaystyle\text{WSC}=\inf_{v\in V}\text{WSC}_{n}\big(C_{\alpha}(\cdot),v\big). 
 
 
 
 This set is typically generated by sampling vectors at random from ℝ d \mathbb{R}^{d} . When evaluating over a finite set of directions V V , WSC uniformly approximates the population slab-coverage with high probability; the approximation error depends on the VC dimension ( vapnik2013nature ) of the class of slabs induced by V V . However, under conditional coverage, without sufficient test data WSC tends to provide a pessimist estimate of the conditional coverage violation by overfitting the test set by isolating miscovered points. Furthermore, this strategy does not adapt well to categorical data. 
 
 
 
 
 
 In Appendix  A we review additional metrics. One of them is feature-stratified coverage (FSC), a group-based metric that reports the group with the worst coverage. This metric often appears in fairness-related work (see e.g, angelopoulos2021gentle ; ding2023class ; jung2022batch ). Several grouping strategies besides categorical attributes or clustering have also been studied. For example, equal opportunity of coverage (EOC) ( wang2023equal ) forms groups based on the output, and size-stratified coverage (SSC) ( angelopoulos2020uncertainty ) groups examples by the size of the prediction set. Another approach is to avoid explicit grouping and instead measure statistical dependence between the coverage indicator
 Z := 𝟏 ​ { Y ∈ C α ​ ( X ) } Z:=\mathbf{1}\{Y\in C_{\alpha}(X)\} and the prediction-set size. feldman2021improving introduced two such dependence measures based on Pearson’s correlation and the Hilbert–Schmidt independence criterion (HSIC). 
 
 
 
 Each diagnostic has strengths and limitations. Group-based metrics (CovGap, FSC, EOC, SSC) are intuitive and directly tied to fairness-style guarantees, but their statistical power depends strongly on the choice of groups and on having enough data per group. Geometric scans like WSC explore slices of 𝒳 \mathcal{X} without pre-specified semantic groups but suffer from the complexity of the feature space. Representation-based measures (Pearson, HSIC) provide complementary, model-driven checks for dependence between coverage and auxiliary signals, but low dependence does not prove full conditional coverage. A central challenge in catalyzing research progress in conformal prediction is the lack of reliable ways to assess conditional coverage empirically.
Although many recent methods are designed to improve conditional coverage ( gibbs2025conformal ; ding2023class ; kaur2025conformal ; plassier2024probabilistic ) , existing guarantees are largely theoretical, and robust practical metrics remain elusive. 
 
 
 
 
 
 3 Evaluating Conditional Coverage 
 
 We would like the conditional coverage 
 
 
 
 p ​ ( x ) := ℙ ​ ( Y ∈ C α ​ ( X ) ∣ X = x ) , p(x):={\mathbb{P}}(Y\in C_{\alpha}(X)\mid X=x), 
 
 
 
 to be equal to 1 − α 1-\alpha ℙ X {\mathbb{P}}_{X} -almost surely. Introducing the binary random variable Z = 𝟙 ​ { Y ∈ C α ​ ( X ) } Z={\mathbbm{1}}\{Y\in C_{\alpha}(X)\} , we can rewrite 
 
 
 
 p ​ ( x ) = ℙ ​ ( Z = 1 | X = x ) . p(x)={\mathbb{P}}(Z=1|X=x). 
 
 
 
 Estimating ℙ ​ ( Z = 1 | X = x ) {\mathbb{P}}(Z=1|X=x) is a binary classification problem, as we have access to a dataset of pairs ( X i , Z i ) (X_{i},Z_{i}) . Some metrics such as CovGap or WSC implicitly learn classifiers based on histograms or slabs. However, these methods are rarely used for classification due to their poor practical performance. Explicitly reformulating conditional miscoverage estimation as a classification problem allows us to leverage strong and practically proven classifiers. Once a classifier h : 𝒳 → [ 0 , 1 ] h:\mathcal{X}\to[0,1] is trained, we still need to use it to assess conditional miscoverage. The key idea is that under conditional coverage, given a proper score ℓ \ell , no classifier can achieve a lower risk than the constant predictor 1 − α 1-\alpha . If we can learn a predictor that performs better, then conditional coverage does not hold. This leads to a metric with theoretical guarantees and a clear interpretation. In particular, our metric is a conservative estimate of 𝔼 ​ [ d ​ ( 1 − α , p ​ ( X ) ) ] \mathbb{E}[d(1-\alpha,p(X))] for any d : [ 0 , 1 ] × [ 0 , 1 ] → ℝ d:[0,1]\times[0,1]\to\mathbb{R} such that for all p ∈ [ 0 , 1 ] p\in[0,1] , d ​ ( p , ⋅ ) d(p,\cdot) is convex and minimized at p p . 
 
 
 
 3.1 Excess risk of the target coverage (ERT) 
 
 For a given classifier h h and loss function ℓ \ell , the associated risk is defined as 
 
 
 
 ℛ ℓ ​ ( h ) := 𝔼 X , Z ​ [ ℓ ​ ( h ​ ( X ) , Z ) ] . \mathcal{R}_{\ell}(h):=\mathbb{E}_{X,Z}[\ell(h(X),Z)]. 
 
 
 
 
 
 The Bayes predictor in this task is (see, e.g., devroye2013probabilistic ): 
 
 
 
 h ∗ ​ ( x ) ∈ argmin q ∈ [ 0 , 1 ] ​ 𝔼 ​ [ ℓ ​ ( q , Z ) ∣ X = x ] . h^{*}(x)\in\underset{q\in[0,1]}{\textrm{argmin}}\,{\mathbb{E}}[\ell(q,Z)\mid X=x]. 
 
 
 
 
 
 If the loss ℓ \ell is a proper loss (see, e.g, gneiting2007strictly ; brocker2009reliability ), then it is optimal to predict the true probability h ∗ ​ ( X ) = 𝔼 ​ [ Z | X ] = p ​ ( X ) ℙ X h^{*}(X)={\mathbb{E}}[Z|X]=p(X)\quad{\mathbb{P}}_{X} -almost surely. If conditional coverage holds, we get h ∗ ​ ( X ) = 𝔼 ​ [ Z | X ] = 1 − α ℙ X h^{*}(X)={\mathbb{E}}[Z|X]=1-\alpha\quad{\mathbb{P}}_{X} -almost surely, so no classifier can achieve lower risk than the constant 1 − α 1-\alpha prediction. Proper losses include the Brier score, ℓ ​ ( p , y ) = ( p − y ) 2 \ell(p,y)=(p-y)^{2} , and the log-loss score, ℓ ​ ( p , y ) = − y ​ log ⁡ p − ( 1 − y ) ​ log ⁡ ( 1 − p ) \ell(p,y)=-y\log p-(1-y)\log(1-p) , where p ∈ [ 0 , 1 ] p\in[0,1] denotes the predicted probability of the event occurring and y ∈ { 0 , 1 } y\in\{0,1\} denotes the observed outcome. 
 
 
 This motivates the excess risk of the target coverage ( ℓ ​ -ERT \ell\textrm{-ERT} ). For a general proper loss ℓ \ell , we define 
 
 
 
 ℓ ​ -ERT \displaystyle\ell\textrm{-ERT} 
 := ℛ ℓ ​ ( 1 − α ) − ℛ ℓ ​ ( p ) . \displaystyle:=\mathcal{R}_{\ell}(1-\alpha)-\mathcal{R}_{\ell}(p). 
 
 
 
 Larger values of ℓ ​ -ERT \ell\textrm{-ERT} correspond to greater violations of conditional coverage. 
 
 
 Interpretation and examples. 
 
 ERT has a probabilistic interpretation. Indeed, 
 
 
 
 ℓ ​ -ERT \displaystyle\ell\textrm{-ERT} 
 = ℛ ℓ ​ ( 1 − α ) − ℛ ℓ ​ ( p ) \displaystyle=\mathcal{R}_{\ell}(1-\alpha)-\mathcal{R}_{\ell}(p) 
 
 
 
 
 
 = 𝔼 X , Z ​ [ ℓ ​ ( 1 − α , Z ) − ℓ ​ ( p ​ ( X ) , Z ) ] \displaystyle=\mathbb{E}_{X,Z}[\ell(1-\alpha,Z)-\ell(p(X),Z)] 
 
 
 
 
 
 = 𝔼 X ​ [ 𝔼 Z ​ [ ℓ ​ ( 1 − α , Z ) − ℓ ​ ( p ​ ( X ) , Z ) | X ] ] \displaystyle=\mathbb{E}_{X}[{\mathbb{E}}_{Z}[\ell(1-\alpha,Z)-\ell(p(X),Z)|X]] 
 
 
 
 
 
 = 𝔼 X ​ [ d ℓ ​ ( 1 − α , p ​ ( X ) ) ] , \displaystyle={\mathbb{E}}_{X}[d_{\ell}(1-\alpha,p(X))], 
 
 
 
 where d ℓ ​ ( p , q ) := 𝔼 y ∼ q ​ [ ℓ ​ ( p , y ) − ℓ ​ ( q , y ) ] d_{\ell}(p,q):={\mathbb{E}}_{y\sim q}[\ell(p,y)-\ell(q,y)] is the divergence associated with the proper score ℓ \ell (see, e.g, brocker2009reliability ). We summarize the ℓ \ell -ERT scores for different proper scores in Table ˜ 1 . 
 
 
 
 
 Name 
 Proper score ℓ ​ ( p , y ) \ell(p,y) 
 
 ℓ \ell -ERT formula 
 
 
 
 
 L 1 L_{1} -ERT 
 sgn ⁡ ( p − ( 1 − α ) ) ​ ( 1 − α − y ) \operatorname{sgn}(p-(1-\alpha))(1-\alpha-y) 
 𝔼 X ​ [ | 1 − α − p ​ ( X ) | ] \mathbb{E}_{X}[|1-\alpha-p(X)|] 
 
 
 
 L 2 L_{2} -ERT 
 Brier score: ( y − p ) 2 (y-p)^{2} 
 
 𝔼 X ​ [ ( 1 − α − p ​ ( X ) ) 2 ] \mathbb{E}_{X}[(1-\alpha-p(X))^{2}] 
 
 
 KL-ERT 
 Log-loss: − log ⁡ p y -\log p_{y} 
 
 𝔼 X ​ [ D KL ​ ( p ​ ( X ) ∥ 1 − α ) ] \mathbb{E}_{X}[D_{\mathrm{KL}}(p(X)\|1-\alpha)] 
 
 
 Table 1: Examples of proper scoring rules and their associated ERT scores. 
 
 
 We will show in Section ˜ 3.2 that a general class of convex distances can be estimated via ERTs. 
 
 
 
 Estimation from finite samples. 
 
 Since p ​ ( X ) p(X) is unknown in practice, we define the functional 
 
 
 
 ℓ ​ -ERT ​ ( h ) := ℛ ℓ ​ ( 1 − α ) − ℛ ℓ ​ ( h ) , \displaystyle\ell\textrm{-ERT}(h):=\mathcal{R}_{\ell}(1-\alpha)-\mathcal{R}_{\ell}(h), 
 
 (4) 
 
 
 This metric quantifies how much better a predictor h h performs relative to the constant baseline  1 − α 1-\alpha . While we cannot guarantee that the learned predictor h h coincides with the Bayes-optimal predictor h ∗ h^{*} , our procedure always provides a lower bound on the true ℓ ​ -ERT \ell\textrm{-ERT} , that is for all measurable classifiers, 
 
 
 
 ℓ ​ -ERT ​ ( h ) ≤ ℓ ​ -ERT . \displaystyle\ell\textrm{-ERT}(h)\leq\ell\textrm{-ERT}. 
 
 
 
 
 
 Therefore, it suffices to find an h h that performs better than the constant 1 − α 1-\alpha to conclude that conditional coverage is not achieved, and use ℓ ​ -ERT ​ ( h ) \ell\textrm{-ERT}(h) to lower-bound 𝔼 ​ [ d ℓ ​ ( 1 − α , p ​ ( X ) ) ] {\mathbb{E}}[d_{\ell}(1-\alpha,p(X))] . 
 
 
 To estimate ℓ ​ -ERT ​ ( h ) \ell\textrm{-ERT}(h) , we can evaluate their empirical risks that is 
 
 
 
 ℓ ​ -ERT ^ ​ ( h ) = 1 m ​ ∑ i = 1 m [ ℓ ​ ( 1 − α , Z i ) − ℓ ​ ( h ​ ( X i ) , Z i ) ] . \widehat{\ell\textrm{-ERT}}(h)=\frac{1}{m}\sum_{i=1}^{m}\big[\ell(1-\alpha,Z_{i})-\ell(h(X_{i}),Z_{i})\big]. 
 
 
 
 
 
 To avoid overfitting and misleading diagnostics, we cannot train h h on the values X i X_{i} that it is evaluated on. In general, cross-validation can be used for this purpose, where multiple classifiers are trained on different subsets of the data, such that each data point can be evaluated using a classifier that was not trained on it. For random forest, we can use out-of-bag predictions. The resulting algorithm for evaluating conditional coverage using k k -fold cross-validation is summarized in Algorithm  1 . 
 
 
 Algorithm 1 Compute ℓ ​ -ERT ^ \widehat{\ell\textrm{-ERT}} . 
 
 
 1: Data { ( X i , Z i ) } i = 1 m \{(X_{i},Z_{i})\}_{i=1}^{m} , number of folds k ≥ 2 k\geq 2 , proper score ℓ \ell , level α \alpha , classification method.

 
 
 2: Partition the data: Randomly divide { 1 , … , m } \{1,\dots,m\} into k k approx. equal-sized folds
 { ℐ 1 , … , ℐ k } \{\mathcal{I}_{1},\dots,\mathcal{I}_{k}\} .

 
 
 3: for j = 1 j=1 to k k do 
 
 
 4:    Define folds: ℐ val ( j ) = ℐ j , ℐ tr ( j ) = { 1 , … , m } ∖ ℐ j . \displaystyle\mathcal{I}_{\text{val}}^{(j)}=\mathcal{I}_{j},\quad\mathcal{I}_{\text{tr}}^{(j)}=\{1,\dots,m\}\setminus\mathcal{I}_{j}. 
 
 
 5:    Train classifier: Fit a classifier h ( j ) h^{(j)} on { ( X i , Z i ) ∣ i ∈ ℐ tr ( j ) } \{(X_{i},Z_{i})\mid i\in\mathcal{I}_{\text{tr}}^{(j)}\} using the specified method.

 
 
 6:    Evaluate on validation fold: For each i ∈ ℐ val ( j ) i\in\mathcal{I}_{\text{val}}^{(j)} , compute

 
 
 
 ℓ ​ -ERT ^ ( j ) = 1 | ℐ val ( j ) | ​ ∑ i ∈ ℐ val ( j ) ℓ ​ ( 1 − α , Z i ) − ℓ ​ ( h ( j ) ​ ( X i ) , Z i ) . \widehat{\ell\textrm{-ERT}}^{(j)}=\cfrac{1}{|\mathcal{I}_{\text{val}}^{(j)}|}\sum_{i\in\mathcal{I}_{\text{val}}^{(j)}}\ell(1-\alpha,Z_{i})-\ell(h^{(j)}(X_{i}),Z_{i}). 
 
 
 
 
 
 7: end for 
 
 
 8: Aggregate across folds: ℓ ​ -ERT ^ = 1 k ​ ∑ j = 1 k ℓ ​ -ERT ^ ( j ) \displaystyle\widehat{\ell\textrm{-ERT}}=\cfrac{1}{k}\sum_{j=1}^{k}\widehat{\ell\textrm{-ERT}}^{(j)} .

 
 
 
 
 Figure  1 illustrates the usefulness of our metric by showing prediction sets produced under different conformal strategies together with their estimated conditional coverage. The function h h is estimated with a neural network that has two hidden layers of width 64. In the first strategy, which applies a non conditional conformal method, the prediction sets fail to reflect local variations in conditional miscoverage. This leads to large estimated ERT values, with L 1 ​ -ERT ​ ( h ) ≈ 0.0757 L_{1}\textrm{-ERT}(h)\approx 0.0757 and L 2 ​ -ERT ​ ( h ) ≈ 0.0073 L_{2}\textrm{-ERT}(h)\approx 0.0073 . In the second strategy, where prediction sets are closer to satisfying conditional coverage, the estimator h h identifies coverage levels near 1 − α 1-\alpha . The resulting ERT values are closer to zero, with L 1 ​ -ERT ​ ( h ) ≈ 0.0148 L_{1}\textrm{-ERT}(h)\approx 0.0148 and L 2 ​ -ERT ​ ( h ) ≈ − 0.00002 L_{2}\textrm{-ERT}(h)\approx-0.00002 , which signals improved conditional behavior. 
 
 
 We also compare our functional estimator h h to the partition based nonparametric estimator CovGap ​ ( X ) \mathrm{CovGap}(X) . In one dimension, the feature space is simple to cluster and the partition-based approach can approximate p p well. This advantage does not persist as the feature dimension grows, a point that will be demonstrated in Section  4 . 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 1: Illustration of conditional coverage estimation.
The top panel shows data generated from Y ∼ 𝒩 ​ ( f ​ ( X ) , σ ​ ( X ) ) Y\sim\mathcal{N}\large(f(X),\sigma(X)\large) , X ∼ 𝒰 ​ ( [ − 1 , 1 ] ) X\sim\mathcal{U}\large([-1,1]\large) with f ​ ( x ) = 3 ​ sin ⁡ ( x ) + e x f(x)=3\sin(x)+e^{x} and σ ​ ( x ) = 1 / 2 + | x | + x 2 \sigma(x)=1/2+|x|+x^{2} , and their predictive sets. The bottom panel shows the conditional coverage estimation h h used to estimate the L 2 ​ -ERT ​ ( h ) L_{2}\textrm{-ERT}(h) , conditional coverage estimation induced by a partition-wise estimator, true conditional coverage, and desired 1 − α 1-\alpha conditional coverage.
 Left: Conformal sets from the score S ​ ( X , Y ) = | Y − f ^ ​ ( X ) | S(X,Y)=|Y-\hat{f}(X)| .
 Right: Conformal sets by fitting quantiles α / 2 \alpha/2 and 1 − α / 2 1-\alpha/2 of ℙ Y | X {\mathbb{P}}_{Y|X} following the procedure in romano2019conformalized . 
 
 
 
 
 
 3.2 Estimating general distances 
 
 Previously, Table ˜ 1 illustrated that specific choices of the proper loss ℓ \ell can recover common distance functions. However, we can go much beyond that and estimate any convex distance function f ​ ( q ) = d ​ ( 1 − α , q ) f(q)=d(1-\alpha,q) using an ERT, as long as the proper score ℓ \ell is allowed to depend on f f and therefore the coverage 1 − α 1-\alpha itself. The following proposition formalizes this statement. 
 
 
 
 Proposition 3.1 (Representing convex losses as ERTs) . 
 
 
 Let f : [ 0 , 1 ] → ℝ ≥ 0 f:[0,1]\to\mathbb{R}_{\geq 0} be convex with f ​ ( 1 − α ) = 0 f(1-\alpha)=0 . Let f ′ f^{\prime} be a subderivative of f f satisfying f ′ ​ ( 1 − α ) = 0 f^{\prime}(1-\alpha)=0 . Then, the function 
 
 
 
 ℓ ​ ( p , y ) := ℓ f , f ′ ​ ( p , y ) := − f ​ ( p ) − ( y − p ) ​ f ′ ​ ( p ) ( p ∈ [ 0 , 1 ] , y ∈ { 0 , 1 } ) \ell(p,y):=\ell_{f,f^{\prime}}(p,y):=-f(p)-(y-p)f^{\prime}(p)\quad(p\in[0,1],y\in\{0,1\}) 
 
 
 
 is a proper score satisfying 
 
 
 
 ℓ ​ - ​ ERT = 𝔼 X ​ [ f ​ ( p ​ ( X ) ) ] . \displaystyle\ell\textrm{-}\mathrm{ERT}={\mathbb{E}}_{X}[f(p(X))]~. 
 
 
 
 
 
 
 Proof. 
 
 First, ℓ \ell is a proper score because for all p , q ∈ [ 0 , 1 ] p,q\in[0,1] , convexity of f f yields 
 
 
 
 𝔼 y ∼ q ​ [ ℓ ​ ( p , y ) ] = − f ​ ( p ) − ( q − p ) ​ f ′ ​ ( p ) ≥ − f ​ ( q ) = 𝔼 y ∼ q ​ [ ℓ ​ ( q , y ) ] . \mathbb{E}_{y\sim q}[\ell(p,y)]=-f(p)-(q-p)f^{\prime}(p)\geq-f(q)=\mathbb{E}_{y\sim q}[\ell(q,y)]~. 
 
 
 
 Since we assumed f ′ ​ ( 1 − α ) = 0 f^{\prime}(1-\alpha)=0 , we have ℓ ​ ( 1 − α , Z ) = − f ​ ( 1 − α ) = 0 \ell(1-\alpha,Z)=-f(1-\alpha)=0 . Hence, 
 
 
 
 𝔼 Z ∼ p ​ [ ℓ ​ ( 1 − α , Z ) − ℓ ​ ( p , Z ) ] = 𝔼 Z ∼ p ​ [ − ℓ ​ ( p , Z ) ] = f ​ ( p ) + ( p − p ) ​ f ′ ​ ( p ) = f ​ ( p ) . \mathbb{E}_{Z\sim p}[\ell(1-\alpha,Z)-\ell(p,Z)]=\mathbb{E}_{Z\sim p}[-\ell(p,Z)]=f(p)+(p-p)f^{\prime}(p)=f(p)~. 
 
 
 
 Taking the expectation over X X yields the claim.
∎ 
 
 
 
 A related formulation in Section ˜ B shows that ERT can estimate d d if it is a Bregman divergence of convex functions. 
 
 
 
 
 3.3 Separating over-coverage and under-coverage 
 
 Proposition ˜ 3.1 implies that we can estimate asymmetric distance measures to gain more insights on the nature of miscoverage. In particular, one can decompose the convex function f f from above as f = f + + f − f=f_{+}+f_{-} with an over-coverage part f + ​ ( p ) := f ​ ( max ⁡ { p , 1 − α } ) f_{+}(p):=f(\max\{p,1-\alpha\}) that only penalizes the case p > 1 − α p>1-\alpha and an under-coverage part f − ​ ( p ) = f ​ ( min ⁡ { p , 1 − α } ) f_{-}(p)=f(\min\{p,1-\alpha\}) that only penalizes p < 1 − α p<1-\alpha . Correspondingly, one can decompose the proper loss ℓ \ell and the ERT as 
 
 
 
 ℓ ​ ( p , y ) \displaystyle\ell(p,y) 
 = ℓ + ​ ( p , y ) + ℓ − ​ ( p , y ) − ℓ ​ ( 1 − α , y ) \displaystyle=\ell_{+}(p,y)+\ell_{-}(p,y)-\ell(1-\alpha,y) 
 
 
 
 
 ℓ ​ -ERT \displaystyle\ell\text{-ERT} 
 = ℓ + ​ -ERT + ℓ − ​ -ERT \displaystyle=\ell_{+}\text{-ERT}+\ell_{-}\text{-ERT} 
 
 
 
 
 ℓ + ​ ( p , y ) \displaystyle\ell_{+}(p,y) 
 := ℓ ​ ( max ⁡ { p , 1 − α } , y ) \displaystyle:=\ell(\max\{p,1-\alpha\},y) 
 
 
 
 
 ℓ − ​ ( p , y ) \displaystyle\ell_{-}(p,y) 
 := ℓ ​ ( min ⁡ { p , 1 − α } , y ) . \displaystyle:=\ell(\min\{p,1-\alpha\},y)~. 
 
 
 
 
 
 Together, ℓ + ​ -ERT \ell_{+}\text{-ERT} and ℓ − ​ -ERT \ell_{-}\text{-ERT} provide a decomposition of conditional coverage error into two complementary components. The first identifies unnecessary conservatism, while the second highlights locations where the procedure is too aggressive and exhibits under-coverage. This split view delivers more informative diagnostics and supports targeted improvements in the design of conformal prediction methods. 
 
 
 
 
 3.4 Extensions 
 
 A proxy for conditional coverage. 
 
 The learned predictor h h can also be used as a proxy for conditional coverage. For a given test point X test X_{\textrm{test}} , its conditional coverage can be estimated as 
 
 
 
 h ​ ( X test ) ≈ ℙ ​ ( Y test ∈ C α ​ ( X test ) ∣ X test ) . h(X_{\textrm{test}})\approx{\mathbb{P}}\big(Y_{\textrm{test}}\in C_{\alpha}(X_{\textrm{test}})\mid X_{\textrm{test}}\big). 
 
 
 
 This approximation provides a corrective proxy for conditional coverage: rather than modifying the predictive set C α ​ ( X test ) C_{\alpha}(X_{\textrm{test}}) , we adjust its predicted coverage level from 1 − α 1-\alpha to h ​ ( X test ) h(X_{\textrm{test}}) . 
 
 
 
 Evaluating conditional coverage rules. 
 
 We further extend our metric to settings where the target conditional coverage is not fixed at 1 − α 1-\alpha , but instead varies according to a specified decision rule.
This extension, detailed in Appendix  D , enables testing whether a given strategy satisfies conditional coverage with respect to adaptive or context-dependent coverage levels. 
 
 
 
 
 
 
 
 4 Experiments 
 
 Our package covmetrics 2 2 2 https://github.com/ElSacho/covmetrics and the code for all our experiments 3 3 3 https://github.com/ElSacho/Conditional_Coverage_Estimation is accessible and reproducible from GitHub. 
 
 
 
 4.1 Comparing different classifiers 
 
 The quality of the ERT estimation hinges on the choice of a good classifier, which depends on the data type of X X . We will restrict our experiments to the case where X X is a fixed-dimensional vector of numerical and/or categorical features, also known as tabular data . For other modalities like images or text, tabular classifiers could be used on top of embeddings of X X to keep the training fast. As we want our metric to be reasonably fast to compute, we are particularly interested in finding fast classifiers. For this reason, we do not tune the hyperparameters of the classifiers. Based on recent benchmarks ( erickson2025tabarena ; holzmuller2024better ) , we choose a subselection of classifiers that are promising in terms of their speed-accuracy trade-off. We provide more details on those classifiers in Appendix  F . We note that our results show performances of specific configurations, but other trade-offs can also be achieved. 
 
 
 We begin by examining how various classifiers perform when estimating the conditional coverage quantity. This is achieved by comparing how well different tabular classifiers estimate the conditional miscoverage. We review those classifiers in Appendix  F . To pursue this, we select the four largest regression datasets in TabArena ( erickson2025tabarena ) . Each dataset is divided into three parts; a training set (with 40 % 40\% of the data) used to learn a predictor f f that minimizes the empirical mean squared error. A calibration set (with 10 % 10\% of the data) used with the nonconformity score S ​ ( X , Y ) = | Y − f ​ ( X ) | S(X,Y)=|Y-f(X)| to construct the set rule C α ​ ( ⋅ ) C_{\alpha}(\cdot) with 1 − α = 0.9 1-\alpha=0.9 such that when the residual distribution is heteroskedastic, these sets are not expected to be conditional. A test set (with 50 % 50\% of the data) used to evaluate the L 1 L_{1} -ERT, L 2 L_{2} -ERT and KL-ERT metrics, performing 5 5 -fold cross-validation. 
 
 
 We compare the estimated metric values as a function of the number of test samples and average all results over ten runs. Since our estimator gives a lower bound on the true ℓ \ell -ERT value, a larger estimate indicates a stronger classifier. In Table ˜ 2 we report the average percentage improvement over the best strategy, averaged across all test sample sizes and all datasets, as well as the average time required to estimate our metrics per 1,000 samples. Because averaging can hide important effects, we also report results for each dataset as a function of the number of test samples in Figures  2 and  3 . 
 
 
 
 
 Classifier 
 Avg. % of max ERT 
 
 
 
 Avg. time per 
 
 
 1K samples [s] 
 
 
 Device 
 
 
 
 
 L 1 L_{1} -ERT 
 
 L 2 L_{2} -ERT 
 KL-ERT 
 
 
 
 
 TabICLv1.1 
 71.9 1.9 \textbf{71.9}_{1.9} 
 55.4 2.7 55.4_{2.7} 
 60.2 1.8 \textbf{60.2}_{1.8} 
 5.1 0.7 5.1_{0.7} 
 GPU 
 
 
 RealTabPFN-2.5 
 71.6 1.7 71.6_{1.7} 
 55.7 2.5 \textbf{55.7}_{2.5} 
 59.7 1.9 59.7_{1.9} 
 5.2 0.7 5.2_{0.7} 
 GPU 
 
 
 CatBoost 
 68.7 2.8 68.7_{2.8} 
 50.8 2.7 50.8_{2.7} 
 55.2 2.1 55.2_{2.1} 
 18.7 4.5 18.7_{4.5} 
 CPU 
 
 
 LightGBM (medium) 
 68.4 2.2 68.4_{2.2} 
 49.9 2.5 49.9_{2.5} 
 53.6 1.7 53.6_{1.7} 
 2.6 0.3 2.6_{0.3} 
 CPU 
 
 
 ExtraTrees 
 65.9 2.4 65.9_{2.4} 
 30.5 3.2 30.5_{3.2} 
 0.0 0.0 0.0_{0.0} 
 2.1 0.4 2.1_{0.4} 
 CPU 
 
 
 RandomForest 
 65.9 2.8 65.9_{2.8} 
 35.7 2.5 35.7_{2.5} 
 0.0 0.0 0.0_{0.0} 
 2.7 0.5 2.7_{0.5} 
 CPU 
 
 
 PartitionWise 
 38.3 1.9 38.3_{1.9} 
 14.1 1.1 14.1_{1.1} 
 1.7 0.9 1.7_{0.9} 
 0.2 0.0 \textbf{0.2}_{0.0} 
 CPU 
 
 
 Table 2: ERT recovered by different methods , relative to the highest value among all methods and number of samples, averaged over all number of test samples and datasets. Experiments are repeated 10 times, and the index number is the standard deviation across those 10 experiments. 
 
 
 Results. 
 
 As shown in Table ˜ 2 , the tabular foundation models TabICLv1.1 ( qu2025tabicl ) and RealTabPFN-2.5 ( grinsztajn2025tabpfn ) show excellent performance across different metrics. However, they can be very slow without a GPU, and they are generally only usable up to a certain size of datasets (around 100K samples). Gradient-boosted decision trees like CatBoost ( prokhorenkova2018catboost ) and LightGBM ( ke2017lightgbm ) are closely behind, but they do facilitate fast training on CPUs and are scalable to large datasets. In particular, LightGBM with the relatively cheap configuration from holzmuller2024better excels through its training speed while still recovering large ERT values. Therefore, we suggest LightGBM as the default classifier to use with ERT. Random Forest ( breiman2001random ) and ExtraTrees ( geurts2006extremely ) are also fast but exhibit worse results, particularly for the KL-ERT, which heavily penalizes overconfidence, and the L 2 L_{2} -ERT. PartitionWise, the strategy used for CovGap, is fastest but performs much worse than the other classifiers. Figures  2 and  3 show that on two out of four datasets, PartitionWise detects almost no miscoverage.
Overall, our results suggest that results of tabular classification benchmarks transfer approximately to the task of ERT estimation. 
 
 
 
 Differences between metrics. 
 
 Table ˜ 2 also shows that the L 1 L_{1} -ERT is considerably easier to estimate than the L 2 L_{2} -ERT and KL-ERT. Indeed, for the L 1 L_{1} -ERT, the classifiers only need to predict on the right side of 1 − α 1-\alpha to be optimal, whereas for the others they need to predict the exact probability. Hence, we recommend using the L 1 L_{1} -ERT as the default metric. 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 2: Illustration of the estimation of L 1 L_{1} -ERT for different classifiers as a number of sampled data available.
 Left: physiochemical_protein dataset.
 Right: Diamonds dataset. 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 3: Illustration of the estimation of L 1 L_{1} -ERT for different classifiers as a number of sampled data available. Legend shared with Figure ˜ 2 .
 Left: Food_Delivery_Time dataset.
 Right: Superconductivity dataset. 
 
 
 
 
 
 4.2 Comparison with existing metrics 
 
 We illustrate that existing methods can fail to accurately assess conditional coverage in scenarios where our approach succeeds.
To this end, we generate a synthetic dataset following 
 
 
 
 Y ∼ 𝒩 ​ ( 0 , σ ​ ( X 1 ) ) , with X ∼ 𝒰 ​ ( [ − 1 , 1 ] 8 ) , Y\sim\mathcal{N}\big(0,\sigma(X^{1})\big),\quad\text{with}\quad X\sim\mathcal{U}([-1,1]^{8}), 
 
 
 
 where X 1 X^{1} is the first component of the vector X X . 
 
 
 Prediction sets are constructed using two different strategies: 
 
 
 • 
 
 Standard CP: Using the nonconformity score S ​ ( X , Y ) = | Y | S(X,Y)=|Y| within the standard conformal prediction framework using 3 ​ , ​ 000 3\text{,}000 i.i.d. samples. 
 
 
 
 • 
 
 Oracle sets: Using the ground-truth oracle that provides the true conditional quantiles α / 2 \alpha/2 and 1 − α / 2 1-\alpha/2 of the underlying distribution. 
 
 
 
 
 
 The first strategy produces marginally valid but conditionally invalid prediction sets, while the second produces conditional sets by construction. 
 
 
 The first experiment evaluates how many test points are needed to obtain reliable estimates for commonly used metrics (CovGap, WSC) compared to our proposed metrics. We measure each metric as a function of the number of test points on a log scale, computing L 1 L_{1} -ERT and L 2 L_{2} -ERT respectively estimating 𝔼 X ​ [ | 1 − α − p ​ ( X ) | ] \mathbb{E}_{X}[|1-\alpha-p(X)|] and 𝔼 X ​ [ ( 1 − α − p ​ ( X ) ) 2 ] \mathbb{E}_{X}[(1-\alpha-p(X))^{2}] using 5 5 -fold cross-validation, and show the results in Figure  4 . 
 
 
 
 Results. 
 
 The results are striking: group-based metrics are extremely unaligned with their theoretical values and require large sample sizes to converge. Even with 5 , 000 5{,}000 points, they provide nearly identical diagnostics across these two very different scenarios, and WSC exhibits similar instability. By contrast, our metrics adapt rapidly. In particular, L 1 L_{1} -ERT stabilizes very quickly, providing reliable estimates of conditional coverage deviation in the naive scenario. L 2 L_{2} -ERT needs more samples to converge to its true value but already diagnoses conditional coverage failure with few samples. This is not surprising as, as explained earlier, the L 1 L_{1} version only requires that the sign of h ​ ( X ) − ( 1 − α ) h(X)-(1-\alpha) matches the sign of p ​ ( X ) − ( 1 − α ) p(X)-(1-\alpha) , whereas the L 2 L_{2} version instead depends on the closeness of h ​ ( X ) h(X) to the true conditional probability itself, which typically requires more data. 
 
 
 In the scenario with perfect conditional coverage, all of our proposed metrics converge rapidly to values indicating no failure, while WSC continues to struggle even with 50 , 000 50{,}000 samples. These results demonstrate that our methods not only provide more accurate diagnostics but also require far fewer samples to detect conditional coverage deviations reliably. 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 4: Estimated metrics as a number of test samples points. Top figure: WSC. Middle figure: CovGap & L 1 ​ -ERT L_{1}\textrm{-ERT} . Bottom axis: L 2 ​ -ERT L_{2}\textrm{-ERT} Left: Standard CP not conditional. Right: Oracle sets conditional. Theoretical values are estimated using the true value ℙ ​ ( Y ∈ C α ​ ( X ) | X ) {\mathbb{P}}(Y\in C_{\alpha}(X)|X) for 300 , 000 300,000 samples of ℙ X {\mathbb{P}}_{X} . 
 
 
 Figure  5 visualizes the data distribution and the induced prediction sets for both strategies with test dataset of size 1 , 500 1,500 . Precise metrics values are reported in Table ˜ 3 . Since only the first feature is informative, we plot ( X 1 , Y ) (X^{1},Y) while ignoring the remaining features, and we also show ( X 1 , h ​ ( X ) ) (X^{1},h(X)) to illustrate our estimator’s learned conditional coverage, as well as ( X 1 , C ​ o ​ v ​ G ​ a ​ p ​ ( X ) ) (X^{1},CovGap(X)) to illustrate the difference between a partition-wise estimator and our estimator. As expected, our approach clearly identifies regions of under- and over-coverage in the first scenario, and accurately recovers a near-constant predictor equal to 1 − α 1-\alpha in the oracle setting. 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 5: Illustration of conditional coverage estimation.
The top panel shows data, generated from Y ∼ 𝒩 ​ ( 0 , σ ​ ( X 1 ) ) Y\sim\mathcal{N}\large(0,\sigma(X^{1})\large) with σ ​ ( x ) = 0.5 + | x | + x 2 \sigma(x)=0.5+|x|+x^{2} , where X ∼ 𝒰 ​ ( [ − 1 , 1 ] 8 ) X\sim\mathcal{U}([-1,1]^{8}) , and X 1 X^{1} is the first component of such a vector X X . The bottom panel shows the conditional coverage estimation h h , conditional coverage estimation induced by a partition-wise estimator, true conditional coverage, and desired 1 − α 1-\alpha conditional coverage.
 Left: Conformal sets from the score S ​ ( X , Y ) = | Y | S(X,Y)=|Y| using 3 , 000 3,000 samples.
 Right: Oracle sets that achieve conditional coverage. 
 
 
 We evaluate the metrics on a test set of size 1 , 500 1,500 , as reported in Table ˜ 3 . In the first experiment, where the predictive sets are not conditional, certain metrics fail to detect deviations from conditional coverage. This is the case for SSC, HSIC, and Pearson correlation as they rely solely on prediction set sizes, which are uniform across samples. Similarly, metrics based on feature-space clustering (FSC, CovGap) also fail to detect the coverage violation, due to the difficulty of clustering the feature space. The only baseline that identifies a conditional coverage failure in this case is the WSC metric. 
 
 
 In contrast, under the second (oracle) strategy, all prediction sets satisfy conditional coverage by construction. The WSC metric still reports a conditional coverage failure, with values comparable to the first example. This occurs because, in high-dimensional feature spaces, WSC can overemphasize local fluctuations and misidentify regions with apparent over- or under-coverage. Similarly, EOC detects a false conditional coverage violation by grouping extreme outputs values together. 
 
 
 Our proposed L 1 L_{1} -ERT metric and L 2 L_{2} -ERT metrics, however, correctly distinguish between the two settings: they detect the conditional coverage failure in the first example and reports no such failure in the oracle case. 
 
 
 
 
 
 Not conditional 
 Conditional 
 
 
 FSC 
 
 0.868 0.014 0.868_{0.014} ✗ 
 
 0.881 0.012 0.881_{0.012} ✓ 
 
 
 CovGap 
 
 0.016 0.004 0.016_{0.004} ✗ 
 
 0.014 0.004 0.014_{0.004} ✓ 
 
 
 WCovGap 
 
 0.016 0.004 0.016_{0.004} ✗ 
 
 0.014 0.004 0.014_{0.004} ✓ 
 
 
 WSC 
 
 0.740 0.019 0.740_{0.019} ✓ 
 
 0.790 0.014 0.790_{0.014} ✗ 
 
 
 EOC 
 
 0.341 0.009 0.341_{0.009} ✓ 
 
 0.186 0.018 0.186_{0.018} ✗ 
 
 
 SSC 
 
 0.004 0.003 0.004_{0.003} ✗ 
 
 0.013 0.003 0.013_{0.003} ✓ 
 
 
 HSIC 
 
 0.000 0.000 0.000_{0.000} ✗ 
 
 0.000 0.000 0.000_{0.000} ✓ 
 
 
 Pearson 
 
 0.000 0.000 0.000_{0.000} ✗ 
 
 0.019 0.016 0.019_{0.016} ✓ 
 
 
 
 L 1 L_{1} -ERT (Ours) 
 
 
 0.091 0.007 0.091_{0.007} ✓ 
 
 − 0.005 0.009 -0.005_{0.009} ✓ 
 
 
 
 L 2 L_{2} -ERT (Ours) 
 
 
 0.009 0.001 0.009_{0.001} ✓ 
 
 − 0.000 0.000 -0.000_{0.000} ✓ 
 
 
 Table 3: Conditional metrics for both synthetic samples and whether they accurately diagnose conditional coverage.
✓: Accurate diagnostic. ✗: Failure. 
 
 
 
 
 
 4.3 Real datasets 
 
 
 4.3.1 Regression 
 
 We next compare conditional coverage metrics across the most widely used conformal prediction strategies. In particular, we build upon the benchmarking framework of dheur2025unified to evaluate multivariate regression conformal prediction methods. For each strategy, we first train the underlying predictive model and then conformalize its outputs using the standard split conformal prediction procedure. Detailed descriptions of all strategies are provided in Appendix  E . 
 
 
 Our evaluation is conducted on several datasets commonly used in regression studies. Dataset specifications and algorithmic details are reported in Appendix  G.1 . 
 
 
 All experiments are repeated ten times, and we report averaged estimates across runs. We distinguish between univariate regression ( Y ∈ ℝ Y\in\mathbb{R} ) and multivariate regression ( Y ∈ ℝ k Y\in\mathbb{R}^{k} , for k ≥ 2 k\geq 2 ) to highlight how conditional coverage behaves under increasing output dimensionality. The univariate results are deferred to Appendix  H . 
 
 
 Multivariate results. 
 
 We present aggregated results across six datasets, with dataset specific information provided in Appendix  H . Our first analysis compares the metrics L 1 L_{1} -ERT and WCovGap, which both aim to estimate 𝔼 X ​ [ | 1 − α − p ​ ( X ) | ] {\mathbb{E}}_{X}[|1-\alpha-p(X)|] , as shown in Figure  6 . Each metric is averaged over all datasets. Although both target the same quantity, our metric consistently yields a larger estimate of conditional miscoverage. Because it serves as a lower bound on the true expectation, this indicates that our approach offers a more precise view of conditional coverage than the partition based estimator used in prior methods. Figure  7 then compares the estimation of WSC with our estimator for L 2 L_{2} -ERT. 
 
 
 These results, however, must be interpreted with care. As shown in Figure  8 , improvements in conditional coverage often come hand-in-hand with larger prediction sets. For instance, the C-PCP ( dheur2025unified ) method achieves the best conditional coverage across all metrics, but also produces one of the largest prediction intervals. Conversely, methods such as MVCS ( braun2025minimum ) yield smaller predictive sets at the cost of poorer conditional coverage. This observation underscores a fundamental trade-off: strategies that aggressively minimize prediction volume while maintaining marginal coverage often sacrifice conditional coverage. Understanding and managing this trade-off is crucial for tailoring conformal prediction methods to specific applications. 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 6: Metric values averaged across all datasets for all methods in multivariate regression. Left: L 1 L_{1} -ERT (lower is better). Right: WCovGap (lower is better) 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 7: Metric values averaged across all datasets for all methods in multivariate regression. Left: L 2 L_{2} -ERT (lower is better). Right: WSC (closer to 0.9 0.9 is better). 
 
 
 Figure 8: Normalized set sizes averaged all datasets in multivariate regression, where the normalization is done by dividing each volume by the smallest volume across all methods (smaller is better). 
 
 
 
 
 
 4.3.2 Classification 
 
 In classification, most of the strategies are commonly tailored to specific problems such as long-tailed classification, so we choose to only compare the two most used conformal prediction strategies for classification, given a predictive model that returns probabilities estimates f ^ ​ ( X ) ∈ Δ d \hat{f}(X)\in\Delta_{d} . The first one is the negative likelihood prediction ( sadinle2019least ) and uses the score S ​ ( X , Y ) = − p ​ ( X ) Y S(X,Y)=-p(X)_{Y} . The second one ( romano2020classification ; angelopoulos2020uncertainty ) uses the cumulative likelihood scores. We first define the permutation
 π ​ ( x ) \pi(x) of { 1 , … , K } \{1,\ldots,K\} that sorts the probabilities in decreasing order, i.e., 
 
 
 
 f ^ π 1 ​ ( x ) ​ ( x ) ≥ f ^ π 2 ​ ( x ) ​ ( x ) ≥ ⋯ ≥ f ^ π K ​ ( x ) ​ ( x ) . \hat{f}_{\pi_{1}(x)}(x)\geq\hat{f}_{\pi_{2}(x)}(x)\geq\cdots\geq\hat{f}_{\pi_{K}(x)}(x). 
 
 
 
 Then, the score function is defined as : 
 
 
 
 S ​ ( X , Y ) = ∑ j = 1 π k ​ ( X ) f ^ π j ​ ( X ) ​ ( X ) , where  ​ Y = π k ​ ( X ) . S(X,Y)=\sum_{j=1}^{\pi_{k}(X)}\hat{f}_{\pi_{j}(X)}(X),\quad\text{where }Y=\pi_{k}(X). 
 
 
 
 
 
 For MNIST, FashionMNIST, and CIFAR, we trained a CNN composed of two convolutional layers followed by max pooling, then two fully connected layers with dropout and ReLU activations. The model was trained with cross entropy loss to learn f ^ \hat{f} . We used early stopping when the accuracy fell below 1 − α 1-\alpha , since otherwise both conformal strategies tend to produce many empty sets, which would make the results uninformative. 
 
 
 For the CIFAR100 experiment, we trained a ResNet model with cross-entropy loss to learn f ^ \hat{f} . To learn the classifier for ERT, we re-used this pretrained model, but replaced its final layer with a new one. This avoided the cost of learning a large feature space from scratch. 
 
 
 We report the ERT values in Table ˜ 4 . For the classification problem, both strategies remain far from conditional. In general, we believe that calibrating the predictors leads to better conditional coverage. Interestingly for CIFAR100, the L 1 L_{1} -ERT and the KL-ERT lead to two different conclusions: the former suggests that the likelihood strategy is more conditional than the cumulative one, while the latter suggests the opposite. We attribute this discrepancy to the larger number of empty predictive sets produced by the likelihood strategy, for which the conditional coverage equals zero, that are weighted differently by the KL than the L 1 L_{1} . This is supported by the analysis of under-coverage and over-coverage. This situation happens more frequently under the likelihood strategy. As a consequence, this strategy yields a larger value of KL − \textrm{KL}_{-} -ERT than KL + \textrm{KL}_{+} -ERT, since the KL divergence assigns more weight to such extreme situations. 
 
 
 
 
 Dataset 
 Method 
 
 L 1 L_{1} -ERT 
 KL-ERT 
 
 KL + \textrm{KL}_{+} -ERT 
 
 KL − \textrm{KL}_{-} -ERT 
 
 
 CIFAR10 
 cumulative 
 0.072 0.005 0.072_{0.005} 
 − 0.017 0.008 -0.017_{0.008} 
 − 0.030 0.005 -0.030_{0.005} 
 0.012 0.006 0.012_{0.006} 
 
 
 likelihood 
 0.016 0.002 0.016_{0.002} 
 0.028 0.006 0.028_{0.006} 
 0.007 0.001 0.007_{0.001} 
 0.022 0.007 0.022_{0.007} 
 
 
 CIFAR100 
 cumulative 
 0.041 0.005 0.041_{0.005} 
 0.191 0.024 0.191_{0.024} 
 0.016 0.008 0.016_{0.008} 
 0.175 0.026 0.175_{0.026} 
 
 
 likelihood 
 0.007 0.003 0.007_{0.003} 
 0.409 0.025 0.409_{0.025} 
 0.085 0.022 0.085_{0.022} 
 0.323 0.020 0.323_{0.020} 
 
 
 FashionMNIST 
 cumulative 
 0.165 0.004 0.165_{0.004} 
 − 0.260 0.014 -0.260_{0.014} 
 − 0.185 0.010 -0.185_{0.010} 
 − 0.075 0.004 -0.075_{0.004} 
 
 
 likelihood 
 0.098 0.005 0.098_{0.005} 
 − 0.068 0.008 -0.068_{0.008} 
 − 0.042 0.006 -0.042_{0.006} 
 − 0.026 0.005 -0.026_{0.005} 
 
 
 MNIST 
 cumulative 
 0.150 0.006 0.150_{0.006} 
 − 0.216 0.017 -0.216_{0.017} 
 − 0.159 0.012 -0.159_{0.012} 
 − 0.057 0.006 -0.057_{0.006} 
 
 
 likelihood 
 0.145 0.003 0.145_{0.003} 
 − 0.187 0.007 -0.187_{0.007} 
 − 0.128 0.005 -0.128_{0.005} 
 − 0.059 0.002 -0.059_{0.002} 
 
 
 Table 4: ERT scores obtained for the classification problems. 
 
 
 
 
 
 
 5 Conclusions 
 
 Reliable estimation of conditional coverage is a key challenge for catalyzing further research progress in conformal prediction. We have addressed this challenge by moving from partition-based estimators to a functional-based framework. This shift provides a new perspective on how conditional coverage can be understood, measured, and improved. By framing the problem as one of risk minimization, we introduce a family of interpretable and reliable metrics that leverage the full expressive power of modern predictive models to detect and quantify conditional coverage violations. 
 
 
 Our framework unifies and extends existing diagnostics, transforming what was previously a collection of local, nonparametric estimators into a coherent, model-based approach. This transition not only provides more accurate and stable estimates but also offers a deeper understanding of what conditional coverage represents in practice. While the reliability of our metrics naturally depends on the quality of the learned classifier, this dependency is also a strength. 
 
 
 To encourage reproducibility and practical adoption, we release an open-source package implementing all proposed metrics alongside existing ones. Empirical results confirm the benefits of our approach and reveal that improving conditional coverage often leads to larger prediction sets. Understanding and controlling this balance is a promising direction for future research. 
 
 
 
 Acknowledgements 
 
 Authors acknowledge funding from the European Union (ERC-2022-SYG-OCEAN-101071601). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. This publication is part of the Chair «Markets and Learning», supported by Air Liquide, BNP PARIBAS ASSET MANAGEMENT Europe, EDF, Orange and SNCF, sponsors of the Inria Foundation. This work has also received support from the French government, managed by the National Research Agency, under the France 2030 program with the reference «PR[AI]RIE-PSAI» (ANR-23-IACL-0008). 
 
 
 
 Finally, the authors would like to thank Eugène Berta for fruitful discussions regarding this work. 
 
 
 
 
 
 Appendix 
 
 
 A Additional metrics 
 
 Group-based diagnostics. 
 
 We start by reviewing additional group-based diagnostics. 
 
 
 
 
 • 
 
 Equalized / group-wise coverage. A fairness-style requirement is that every group attains the nominal coverage: 
 
 
 
 ℙ Y | g ​ ( X ) = 𝐠 ​ ( Y ∈ C α ​ ( X ) ∣ g ​ ( X ) = g ) = 1 − α ∀ g ∈ 𝒢 . {\mathbb{P}}_{Y|g(X)=\mathbf{g}}(Y\in C_{\alpha}(X)\mid g(X)=\textbf{g})=1-\alpha\qquad\forall\ \textbf{g}\in\mathcal{G}. 
 
 
 
 This notion appears in the conformal fairness literature (e.g, romano2019malice ; ding2025conformal ) and is evaluated in practice by returning the values C g C_{\textbf{g}} for all g . 
 
 
 
 • 
 
 Feature-stratified coverage (FSC). To focus on the worst-off group, FSC reports the minimal empirical coverage across groups: 
 
 
 
 FSC = min 𝐠 ∈ 𝒢 ⁡ C g . \text{FSC}\;=\;\min_{\mathbf{g}\in\mathcal{G}}C_{\textbf{g}}. 
 
 
 
 FSC highlights subgroups where coverage is lowest and has been used in several works (e.g, angelopoulos2021gentle ; ding2023class ; jung2022batch ). This metric is often viewed as a fairness measure, as it focuses on the group that exhibits the poorest coverage. 
 
 
 
 
 
 All of the above group-based diagnostics are sensitive to how the groups 𝒢 \mathcal{G} are chosen. Most of the time, they are created by partitioning the feature space, either with clustering methods, or by using categorical features. Much of the recent work focuses on finding or learning useful partitions that reveal conditional coverage violations, by applying the induced CovGap or FSC metric. In the following, unless otherwise specified, the groups used to evaluate FSC and CovGap are obtained by clustering the feature space. We will use the following notation to refer to alternative grouping strategies. 
 
 
 • 
 
 Equal opportunity of coverage (EOC). To account for differences in outcomes, EOC requires that coverage rates across protected groups are equal conditional on the true label; in regression this means that for each outcome value y y (or a discretization of y y ), the coverage within each protected subgroup should match. This idea was introduced by wang2023equal . However, defining groups based on the outcome can lead to misleading metrics. For instance, consider an interval predictor [ q α / 2 , q 1 − α / 2 ] [q_{\alpha/2},q_{1-\alpha/2}] for Y ∼ 𝒩 ​ ( 0 , 1 ) Y\sim\mathcal{N}(0,1) , where q α / 2 q_{\alpha/2} and q 1 − α / 2 q_{1-\alpha/2} are the α / 2 \alpha/2 and 1 − α / 2 1-\alpha/2 quantiles of the normal distribution respectively. If all extreme values of Y Y are grouped together, the resulting group may show a coverage of zero, even though the prediction interval is conditionally valid. 
 
 
 
 • 
 
 Size-stratified coverage (SSC). Instead of grouping by features, SSC groups examples by the size (e.g., volume or cardinality) of their prediction sets. It is model-agnostic and useful when X X is high-dimensional because it avoids requiring semantically meaningful groups. However, SSC can fail to detect conditional-coverage problems in some settings. Consider, for example regression with the nonconformity score S ​ ( X , Y ) := | Y − f ​ ( X ) | S(X,Y):=|Y-f(X)|   (cf. angelopoulos2020uncertainty ) . This score cannot capture heteroskedasticity ( vovk2012conditional ) , outputting prediction sets with the same sizes independently of the covariate. Thus, in this case, SSC will not reveal coverage failures. Furthermore, it requires access to the prediction set sizes which can be computationally costly for some strategies. 
 
 
 
 
 
 
 Representation-based diagnostics (dependence on auxiliary variables). 
 
 An alternative to grouping is to measure statistical dependence between the coverage indicator 
 
 
 
 Z := 𝟙 ​ { Y ∈ C α ​ ( X ) } Z:={\mathbbm{1}}\{Y\in C_{\alpha}(X)\} 
 
 
 
 and auxiliary variables V V (for example, prediction-set size, nonconformity score, residuals, or other model-derived quantities). If Z Z is independent of V V , this is evidence that coverage does not systematically vary with Z Z . feldman2021improving proposed two measures induced by this remark. 
 
 
 
 
 • 
 
 Pearson’s correlation. This is a simple measure of linear dependence, 
 
 
 
 R corr ​ ( Z , V ) = Cov ​ ( Z , V ) Var ​ ( Z ) ​ Var ​ ( V ) , R_{\mathrm{corr}}(Z,V)\;=\;\frac{\mathrm{Cov}(Z,V)}{\sqrt{\mathrm{Var}(Z)\,\mathrm{Var}(V)}}, 
 
 
 
 where V V is the prediction-set size. This is fast and interpretable but only captures linear relationships. 
 
 
 
 • 
 
 HSIC (Hilbert–Schmidt independence criterion). The HSIC is a nonparametric kernel-based dependence measure that can detect arbitrary nonlinear dependence. Given a suitable pair of kernels, HSIC estimates the maximum mean discrepancy (MMD) ( gretton2005measuring ) between the joint distribution ℙ Z , V \mathbb{P}_{Z,V} and the product of marginals ℙ Z ⊗ ℙ V \mathbb{P}_{Z}\otimes\mathbb{P}_{V} . feldman2021improving defined the metric based on HSIC: 
 
 
 
 R HSIC ​ ( Z , V ) = HSIC ​ ( Z , V ) , R_{\mathrm{HSIC}}(Z,V)\;=\;\sqrt{\mathrm{HSIC}(Z,V)}, 
 
 
 
 where the square root emphasizes small deviations from independence and gives a loss that is easier to interpret and optimize. Here, V V is again the prediction-set size. 
 
 
 
 
 
 
 
 
 B Estimating Bregman divergences with ERTs 
 
 The following proposition is related to Proposition ˜ 3.1 and studies the setting when there is a single proper score ℓ \ell that estimates a distance d ​ ( 1 − α , p ) d(1-\alpha,p) simultaneously for all α \alpha and p p . 
 
 
 
 Proposition 5.1 . 
 
 
 A function d ​ ( p , q ) d(p,q) arises as the divergence of some proper scoring rule if and only if there exists a convex function φ : [ 0 , 1 ] → ℝ \varphi:[0,1]\to\mathbb{R} such that 
 
 
 
 d ​ ( p , q ) = D φ ​ ( q ∥ p ) = φ ​ ( q ) − φ ​ ( p ) − ( q − p ) ​ s ​ ( p ) d(p,q)=D_{\varphi}(q\|p)=\varphi(q)-\varphi(p)-(q-p)\,s(p) 
 
 
 
 for any choice of subgradient s ​ ( p ) ∈ ∂ φ ​ ( p ) s(p)\in\partial\varphi(p) .
Furthermore, the associated proper score is defined as 
 
 
 
 ℓ φ ​ ( p , y ) := φ ​ ( p ) + ( y − p ) ​ s ​ ( p ) . \ell_{\varphi}(p,y):=\varphi(p)+(y-p)\,s(p). 
 
 
 
 and 
 
 
 
 ℓ φ ​ -ERT = 𝔼 X ​ [ D φ ​ ( p ​ ( X ) ∥ 1 − α ) ] . \ell_{\varphi}\textrm{-ERT}={\mathbb{E}}_{X}[D_{\varphi}(p(X)\|1-\alpha)]. 
 
 
 
 
 
 
 Proof. 
 
 Let ℓ \ell be a proper scoring rule for binary outcomes and write p , q ∈ [ 0 , 1 ] p,q\in[0,1] for probabilities of the event Y = 1 Y=1 . Define 
 
 
 
 φ ​ ( q ) := 𝔼 Y ∼ q ​ [ ℓ ​ ( q , Y ) ] = q ​ ℓ ​ ( q , 1 ) + ( 1 − q ) ​ ℓ ​ ( q , 0 ) . \varphi(q):=\mathbb{E}_{Y\sim q}\big[\ell(q,Y)\big]=q\,\ell(q,1)+(1-q)\,\ell(q,0). 
 
 
 
 Properness of ℓ \ell means that for every fixed p ∈ [ 0 , 1 ] p\in[0,1] and every q ∈ [ 0 , 1 ] q\in[0,1] , 
 
 
 
 φ ​ ( q ) = 𝔼 Y ∼ q ​ [ ℓ ​ ( q , Y ) ] ≤ 𝔼 Y ∼ q ​ [ ℓ ​ ( p , Y ) ] = q ​ ℓ ​ ( p , 1 ) + ( 1 − q ) ​ ℓ ​ ( p , 0 ) . \varphi(q)=\mathbb{E}_{Y\sim q}[\ell(q,Y)]\leq\mathbb{E}_{Y\sim q}[\ell(p,Y)]=q\,\ell(p,1)+(1-q)\,\ell(p,0). 
 
 
 
 Rearranging gives 
 
 
 
 φ ​ ( q ) ≤ φ ​ ( p ) + ( q − p ) ​ ( ℓ ​ ( p , 1 ) − ℓ ​ ( p , 0 ) ) \varphi(q)\leq\varphi(p)+(q-p)(\ell(p,1)-\ell(p,0)) 
 
 
 
 so φ \varphi is convex on [ 0 , 1 ] [0,1] and the function p ↦ ℓ ​ ( p , 1 ) − ℓ ​ ( p , 0 ) p\mapsto\ell(p,1)-\ell(p,0) defines a subgradient of φ \varphi at p p . Denote by s ​ ( p ) ∈ ∂ φ ​ ( p ) s(p)\in\partial\varphi(p) any such subgradient. Then for every p , q ∈ [ 0 , 1 ] p,q\in[0,1] , 
 
 
 
 𝔼 q ​ [ ℓ ​ ( p , Y ) ] = φ ​ ( p ) + ( q − p ) ​ s ​ ( p ) , \mathbb{E}_{q}[\ell(p,Y)]=\varphi(p)+(q-p)\,s(p), 
 
 
 
 and hence the divergence of ℓ \ell satisfies 
 
 
 
 d ℓ ​ ( p , q ) = 𝔼 q ​ [ ℓ ​ ( p , Y ) ] − 𝔼 q ​ [ ℓ ​ ( q , Y ) ] = φ ​ ( p ) + ( q − p ) ​ s ​ ( p ) − φ ​ ( q ) = φ ​ ( q ) − φ ​ ( p ) − ( q − p ) ​ s ​ ( p ) , d_{\ell}(p,q)=\mathbb{E}_{q}[\ell(p,Y)]-\mathbb{E}_{q}[\ell(q,Y)]=\varphi(p)+(q-p)\,s(p)-\varphi(q)=\varphi(q)-\varphi(p)-(q-p)\,s(p), 
 
 
 
 which is exactly the Bregman divergence D φ ​ ( q ∥ p ) D_{\varphi}(q\|p) generated by φ \varphi . 
 
 
 Conversely, let φ : [ 0 , 1 ] → ℝ \varphi:[0,1]\to\mathbb{R} be convex and for each p ∈ [ 0 , 1 ] p\in[0,1] choose a subgradient s ​ ( p ) ∈ ∂ φ ​ ( p ) s(p)\in\partial\varphi(p) . Define a score ℓ φ \ell_{\varphi} by 
 
 
 
 ℓ φ ​ ( p , y ) := φ ​ ( p ) + ( y − p ) ​ s ​ ( p ) . \ell_{\varphi}(p,y):=\varphi(p)+(y-p)\,s(p). 
 
 
 
 by construction, and convexity of φ \varphi implies for every p p and q q , 
 
 
 
 𝔼 q ​ [ ℓ φ ​ ( p , Y ) ] = φ ​ ( p ) + ( q − p ) ​ s ​ ( p ) ≥ φ ​ ( q ) . \mathbb{E}_{q}[\ell_{\varphi}(p,Y)]=\varphi(p)+(q-p)\,s(p)\geq\varphi(q). 
 
 
 
 Thus ℓ φ \ell_{\varphi} is proper, and its divergence equals 
 
 
 
 𝔼 q ​ [ ℓ φ ​ ( p , Y ) ] − 𝔼 q ​ [ ℓ φ ​ ( q , Y ) ] = φ ​ ( p ) + ( q − p ) ​ s ​ ( p ) − φ ​ ( q ) = D φ ​ ( q ∥ p ) . \mathbb{E}_{q}[\ell_{\varphi}(p,Y)]-\mathbb{E}_{q}[\ell_{\varphi}(q,Y)]=\varphi(p)+(q-p)\,s(p)-\varphi(q)=D_{\varphi}(q\|p). 
 
 
 
 
 
 Therefore a function d ​ ( p , q ) d(p,q) arises as the divergence of a proper scoring rule if and only if it is the Bregman divergence of some convex generator φ \varphi , as claimed. Note that the score ℓ φ \ell_{\varphi} is determined up to addition of an arbitrary function of the outcome whose expectation under every q q is zero, and that when φ \varphi is differentiable the subgradient s ​ ( p ) s(p) may be replaced by the derivative φ ′ ​ ( p ) \varphi^{\prime}(p) . 
 
 
 Using this proper score to evaluate the ℓ φ ​ -ERT \ell_{\varphi}\textrm{-ERT} we get 
 
 
 
 
 
 
                                                                                                      ℓ φ ​ -ERT \displaystyle\ell_{\varphi}\textrm{-ERT} 
 
 = \displaystyle{}={} 
 ℛ ℓ φ ​ ( 1 − α ) − ℛ ℓ φ ​ ( p ) \displaystyle\mathcal{R}_{\ell_{\varphi}}(1-\alpha)-\mathcal{R}_{\ell_{\varphi}}(p) 
 
 
 
 
 = \displaystyle{}={} 
 𝔼 X , Z ​ [ ℓ φ ​ ( 1 − α , Z ) − ℓ φ ​ ( p ​ ( X ) , Z ) ] \displaystyle\mathbb{E}_{X,Z}[\ell_{\varphi}(1-\alpha,Z)-\ell_{\varphi}(p(X),Z)] 
 
 
 
 
 = \displaystyle{}={} 
 𝔼 X ​ [ 𝔼 Z ​ [ ℓ φ ​ ( 1 − α , Z ) − ℓ φ ​ ( p ​ ( X ) , Z ) | X ] ] \displaystyle\mathbb{E}_{X}[{\mathbb{E}}_{Z}[\ell_{\varphi}(1-\alpha,Z)-\ell_{\varphi}(p(X),Z)|X]] 
 
 
 
 
 = \displaystyle{}={} 
 𝔼 X ​ [ 𝔼 Z ∼ p ​ ( X ) ​ [ ℓ φ ​ ( 1 − α , Z ) − ℓ φ ​ ( p ​ ( X ) , Z ) ] ] \displaystyle\mathbb{E}_{X}[{\mathbb{E}}_{Z\sim p(X)}[\ell_{\varphi}(1-\alpha,Z)-\ell_{\varphi}(p(X),Z)]] 
 
 
 
 
 = \displaystyle{}={} 
 𝔼 X ​ [ d ℓ φ ​ ( 1 − α , p ​ ( X ) ) ] \displaystyle{\mathbb{E}}_{X}[d_{\ell_{\varphi}}(1-\alpha,p(X))] 
 
 
 
 
 = \displaystyle{}={} 
 𝔼 X ​ [ D φ ​ ( p ​ ( X ) ∥ 1 − α ) ] . \displaystyle{\mathbb{E}}_{X}[D_{\varphi}(p(X)\|1-\alpha)]. 
                                                                                                     ∎ 
 
 
 
 
 
 C Link with gibbs2025conformal 
 
 
 To get predictive sets with conditional guarantees, gibbs2025conformal used the fact that conditional coverage holds if for all measurable function φ \varphi , 
 
 
 
 𝔼 X , Z ​ [ φ ​ ( X ) ​ ( 1 − α − Z ) ] = 0 . \displaystyle{\mathbb{E}}_{X,Z}[\varphi(X)(1-\alpha-Z)]=0. 
 
 
 
 
 To obtain a more operational interpretation, suppose we restrict attention to predictors of the form h ​ ( x ) = θ T ​ ϕ ​ ( x ) h(x)=\theta^{T}\phi(x) , where ϕ ​ ( x ) \phi(x) is a feature map. The associated mean squared risk is 
 
 
 
 F ​ ( θ ) = 𝔼 X , Z ​ [ ( θ T ​ ϕ ​ ( X ) − Z ) 2 ] F(\theta)={\mathbb{E}}_{X,Z}[(\theta^{T}\phi(X)-Z)^{2}] 
 
 
 and has gradient 
 
 
 
 ∇ θ F ​ ( θ ) = 2 ​ 𝔼 X , Z ​ [ ( θ T ​ ϕ ​ ( X ) − Z ) ​ ϕ ​ ( X ) ] . \nabla_{\theta}F(\theta)=2\,{\mathbb{E}}_{X,Z}[(\theta^{T}\phi(X)-Z)\phi(X)]. 
 
 
 Write θ α \theta_{\alpha} such that h θ α ​ ( X ) = 1 − α , h_{\theta_{\alpha}}(X)=1-\alpha, (that exists when ϕ ​ ( X ) \phi(X) has a non-zero constant component). 
 
 
 For this class of models, our metric L 2 L_{2} -ERT compares any θ \theta to this target via 
 
 
 
 F ​ ( θ α ) − inf θ F ​ ( θ ) . F(\theta_{\alpha})-\inf_{\theta}F(\theta). 
 
 
 When conditional coverage holds, both the gradient and the L 2 L_{2} -ERT are equal to zero.
In this sense, the quantity F ​ ( θ α ) − F ​ ( θ ) F(\theta_{\alpha})-F(\theta) offers more interpretability than the gradient alone. While the gradient describes only a local direction in the parameter space of improvement, the risk difference has a clear interpretation that has already been discussed. Indeed, when F ​ ( θ α ) − inf θ F ​ ( θ ) = 0 F(\theta_{\alpha})-\inf_{\theta}F(\theta)=0 we have 𝔼 X , Z ​ [ ( 1 − α − Z ) ​ ϕ ​ ( X ) ] = 0 {\mathbb{E}}_{X,Z}[(1-\alpha-Z)\phi(X)]=0 but it can be that F ​ ( θ α ) − inf θ F ​ ( θ ) F(\theta_{\alpha})-\inf_{\theta}F(\theta) is very small but that ‖ 𝔼 X , Z ​ [ ( 1 − α − Z ) ​ ϕ ​ ( X ) ] ‖ 2 \|{\mathbb{E}}_{X,Z}[(1-\alpha-Z)\phi(X)]\|_{2} is very large. That is why 𝔼 X , Z ​ [ φ ​ ( X ) ​ ( 1 − α − Z ) ] {\mathbb{E}}_{X,Z}[\varphi(X)(1-\alpha-Z)] cannot be used as an interpretable quantity to quantify conditional miscoverage deviation, but only to assess if there is conditional coverage or not. 
 
 
 
 D Extension to a conditional coverage rule 
 
 While conditional coverage is often defined with a fixed target level, 
 
 
 
 ℙ ​ ( Y ∈ C α ​ ( X test ) ∣ X test ) = 1 − α ℙ X ​ -almost surely , {\mathbb{P}}\big(Y\in C_{\alpha}(X_{\text{test}})\mid X_{\text{test}}\big)=1-\alpha\quad{\mathbb{P}}_{X}\textrm{-almost surely}, 
 
 
 ensuring that the prediction set C α ​ ( X ) C_{\alpha}(X) covers the true label with probability 1 − α 1-\alpha for every possible input X X ,
a uniform coverage level may be neither necessary nor desirable in practice.
In many settings, one may wish to adapt the coverage level to reflect varying uncertainty, heteroskedastic noise, or task-specific risk preferences.
Recent conformal prediction methods allow adapting α \alpha dynamically to optimize other objectives ( gauthier2025backward ; gauthier2025adaptive ) . 
 
 
 To formalize this flexibility, we introduce a conditional miscoverage rule 
 α : 𝒳 → [ 0 , 1 ] \alpha:\mathcal{X}\to[0,1] , which prescribes a desired miscoverage level α ​ ( X ) \alpha(X) that may vary with the input features.
Under this generalized framework, the target conditional coverage condition becomes 
 
 
 
 𝔼 ​ [ 𝟙 ​ { Y ∈ C α ​ ( X test ) } ∣ X test ] = 1 − α ​ ( X test ) ℙ X ​ -almost surely . {\mathbb{E}}\big[{\mathbbm{1}}\{Y\in C_{\alpha}(X_{\text{test}})\}\mid X_{\text{test}}\big]=1-\alpha(X_{\text{test}})\quad{\mathbb{P}}_{X}\textrm{-almost surely}. 
 
 
 This formulation recovers the standard conformal setting in the special case where α ​ ( X ) \alpha(X) is constant,
while enabling the analysis of predictors designed to achieve non-uniform, data-dependent coverage guarantees.
It thus provides a principled way to study how prediction methods align with arbitrary, application-driven notions of conditional reliability. This could be useful for example in classification, where the discrete nature of the target may not allow to achieve constant coverage. 
 
 
 We next adapt our framework for the ℓ ​ -ERT \ell\textrm{-ERT} metric under this setting by writing 
 
 
 
 ℓ ​ -ERT \displaystyle\ell\textrm{-ERT} 
 = ℛ ℓ ( 1 − α ) − ℛ ℓ ( p ) = 𝔼 [ ( d ℓ ( 1 − α ( X ) , p ( X ) ) ] . \displaystyle=\mathcal{R}_{\ell}(1-\alpha)-\mathcal{R}_{\ell}(p)={\mathbb{E}}\Big[\big(d_{\ell}(1-\alpha(X),p(X)\big)\Big]. 
 
 
 and its variational form 
 
 
 
 ℓ ​ -ERT ​ ( h ) \displaystyle\ell\textrm{-ERT}(h) 
 = ℛ ℓ ​ ( 1 − α ) − ℛ ℓ ​ ( h ) = 𝔼 X , Y ​ [ ℓ ​ ( 1 − α ​ ( X ) , Y ) − ℓ ​ ( h ​ ( X ) , Y ) ] . \displaystyle=\mathcal{R}_{\ell}(1-\alpha)-\mathcal{R}_{\ell}(h)={\mathbb{E}}_{X,Y}\Big[\ell(1-\alpha(X),Y\big)-\ell(h(X),Y\big)\Big]. 
 
 
 The formulas above work for the L 2 L_{2} -ERT and the KL-ERT, where the proper score ℓ \ell does not depend on 1 − α 1-\alpha . If general distance metrics should be estimated as in Proposition ˜ 3.1 , such as for the L 1 L_{1} -ERT, the proper scoring rule ℓ \ell needs to depend on α ​ ( X ) \alpha(X) , and we obtain 
 
 
 
 ℓ -ERT = 𝔼 [ ( d ℓ α ​ ( X ) ( 1 − α ( X ) , p ( X ) ) ] . \displaystyle\ell\textrm{-ERT}={\mathbb{E}}\Big[\big(d_{\ell_{\alpha(X)}}(1-\alpha(X),p(X)\big)\Big]. 
 
 
 and its variational form 
 
 
 
 ℓ ​ -ERT ​ ( h ) = 𝔼 X , Y ​ [ ℓ α ​ ( X ) ​ ( 1 − α ​ ( X ) , Y ) − ℓ α ​ ( X ) ​ ( h ​ ( X ) , Y ) ] . \displaystyle\ell\textrm{-ERT}(h)={\mathbb{E}}_{X,Y}\Big[\ell_{\alpha(X)}(1-\alpha(X),Y\big)-\ell_{\alpha(X)}(h(X),Y\big)\Big]. 
 
 
 
 
 The remaining components of the procedure remain unchanged.
For convenience, we refer to the resulting metrics using the same terminology as in the fixed 1 − α 1-\alpha case,
since that setting corresponds to a particular instance of this more general framework. 
 
 
 
 E Benchmarking strategies 
 
 The strategies we compare can be differentiated in four main groups—ones that uses density estimation, latent spaces, hyper-rectangles, or minimizing a quantity while ensuring marginal coverage. We build upon the work of dheur2025unified that already explained most of those strategies, but we recall their specificities here for completeness. 
 
 
 Among the density-based methods, given a predictive density p ^ ​ ( y | x ) \hat{p}(y|x) , the benchmarked strategies are: 
 
 
 
 • 
 
 DR-CP ( sadinle2019least ) : Defines the conformity score as S DR-CP ​ ( X , Y ) = − p ^ ​ ( Y | X ) S_{\text{DR-CP}}(X,Y)=-\hat{p}(Y|X) , leading to prediction regions that are density superlevel sets, C DR-CP ​ ( X ) = { y : p ^ ​ ( y | X ) ≥ − q ^ } C_{\text{DR-CP}}(X)=\{y:\hat{p}(y|X)\geq-\hat{q}\} . 
 
 • 
 
 C-HDR ( izbicki2022cd ) : Conformalize the highest predictive density (HPD) by using the nonconformity score S HDP ​ ( X , Y ) = ℙ y ∼ p ^ ( ⋅ | X ) ​ ( p ^ ​ ( y | X ) ≥ p ^ ​ ( Y | X ) ) S_{\text{HDP}}(X,Y)={\mathbb{P}}_{y\sim\hat{p}(\cdot|X)}\left(\;\hat{p}(y|X)\geq\hat{p}(Y|X)\;\right) . It then produces regions C C-HDR ​ ( X ) = { y : f ^ ​ ( y | X ) ≥ t ^ q } C_{\text{C-HDR}}(X)=\{y:\hat{f}(y|X)\geq\hat{t}_{q}\} , where t ^ q \hat{t}_{q} defines the highest density region (HDR) at level q ^ \hat{q} . 
 
 • 
 
 PCP ( wang2022probabilistic ) : Draws L L samples Y ~ ( l ) ∼ p ^ Y | x \tilde{Y}^{(l)}\sim\hat{p}_{Y|x} with p ^ Y | x \hat{p}_{Y|x} the estimated conditional distribution, and defines conformity as the distance to the nearest sample, S PCP ​ ( X , Y ) = min l ∈ [ L ] ⁡ ‖ Y − Y ~ ( l ) ‖ 2 S_{\text{PCP}}(X,Y)=\min_{l\in[L]}\|Y-\tilde{Y}^{(l)}\|_{2} ; the corresponding region is a union of L L balls centered at the sampled points. 
 
 • 
 
 HD-PCP ( wang2022probabilistic ) : Extends PCP by retaining only the top ⌊ ( 1 − α ) ​ L ⌋ \lfloor(1-\alpha)L\rfloor samples with highest density, concentrating the prediction region on high-density areas. 
 
 • 
 
 C-PCP ( dheur2025unified ) : Estimates the conditional CDF of the conformity score S ​ ( X , Y ) S(X,Y) , 
 
 
 
 S CDF ​ ( x , y ) = ℙ ​ ( S W ​ ( X , Y ) ≤ S W ​ ( x , y ) ∣ X = x ) , S_{\text{CDF}}(x,y)={\mathbb{P}}(S_{W}(X,Y)\leq S_{W}(x,y)\mid X=x), 
 
 
 using a Monte Carlo approximation with K K samples 
 
 
 
 S ECDF ​ ( x , y ) = 1 K ​ ∑ k = 1 K 𝟙 ​ [ S W ​ ( x , Y ^ ( k ) ) ≤ S W ​ ( x , y ) ] , Y ^ ( k ) ∼ F ^ Y | x . S_{\text{ECDF}}(x,y)=\frac{1}{K}\sum_{k=1}^{K}{\mathbbm{1}}[S_{W}(x,\hat{Y}^{(k)})\leq S_{W}(x,y)],\quad\hat{Y}^{(k)}\sim\hat{F}_{Y|x}. 
 
 
 When S ​ ( x , y ) = S PCP ​ ( x , y ) S(x,y)=S_{\text{PCP}}(x,y) , this yields 
 
 
 
 S C-PCP ​ ( x , y ) = 1 K ​ ∑ k ∈ [ K ] 1 ​ { min l ∈ [ L ] ⁡ ‖ Y ^ ( k ) − Y ~ ( l ) ‖ 2 ≤ min l ∈ [ L ] ⁡ ‖ y − Y ~ ( l ) ‖ 2 } . S_{\text{C-PCP}}(x,y)=\frac{1}{K}\sum_{k\in[K]}1\!\left\{\min_{l\in[L]}\|\hat{Y}^{(k)}-\tilde{Y}^{(l)}\|_{2}\leq\min_{l\in[L]}\|y-\tilde{Y}^{(l)}\|_{2}\right\}. 
 
 
 
 • 
 
 CP2-PCP ( plassier2024conditionally ) : Builds predictive sets by using samples from an implicit conditional generative model. For each calibration point it uses two independent draws from the conditional generator to define a conformity score and an inflation parameter τ \tau that accounts for the conditional mass around likely outputs. At prediction time it forms a union of balls around new generated samples, with their size chosen to guarantee marginal validity while improving approximate conditional adaptivity. 
 
 • 
 
 MSE : Naïve multivariate generalization of the univariate score S ​ ( X , Y ) = | Y − f ​ ( X ) | S(X,Y)=|Y-f(X)| , by using the multivariate score S ​ ( X , Y ) = ‖ Y − f ​ ( X ) ‖ 2 S(X,Y)=\|Y-f(X)\|_{2} . 
 
 
 
 
 Among the latent space-based methods, the benchmarked strategies are: 
 
 
 
 • 
 
 STDQR ( feldman2023calibrated ) : Constructs multivariate prediction regions in a latent space 𝒵 \mathcal{Z} to overcome limitations of standard multivariate prediction methods. Instead of using directional quantile regression as originally introduced, we follow dheur2025unified procedure where the region R 𝒵 R_{\mathcal{Z}} with coverage 1 − α 1-\alpha is constructed by selecting the 1 − α 1-\alpha proportion of latent samples closest to the origin, ensuring correct coverage directly in the latent space. These latent regions are then mapped to the output space 𝒴 \mathcal{Y} via a conditional generative model (originally a CVAE, here replaced with a normalizing flow). A conformalization step refines coverage by creating a grid of latent samples, mapping them to 𝒴 \mathcal{Y} , and forming small balls around each mapped point. 
 
 • 
 
 L-CP ( dheur2025unified ) : Defines conformity in a latent space using an invertible conditional generative model Q ^ : 𝒵 × 𝒳 → 𝒴 \hat{Q}:\mathcal{Z}\times\mathcal{X}\to\mathcal{Y} . A latent variable Z ∼ 𝒩 ​ ( 0 , I d ) Z\sim\mathcal{N}(0,I_{d}) is mapped to the output space via Q ^ \hat{Q} , and the conformity score is measured in latent space as 
 
 
 
 S L-CP ​ ( X , Y ) = ‖ Q ^ − 1 ​ ( Y ; X ) ‖ . S_{\text{L-CP}}(X,Y)=\|\hat{Q}^{-1}(Y;X)\|. 
 
 
 The prediction region is obtained by taking a ball of radius q ^ \hat{q} around the origin in latent space and mapping it back to the output space. This method avoids grid-based directional quantile regression, improving scalability and computational efficiency, and generalizes distributional conformal prediction to multivariate outputs. 
 
 
 
 
 Among the hyper-rectangle-based methods, the benchmarked strategies are: 
 
 
 
 • 
 
 CopulaCPTS ( sun2022copula ) : This method models the joint dependence between marginal conformity scores via a copula. The calibration data are split into two sets: 𝒟 cal-1 \mathcal{D}_{\text{cal-1}} to estimate empirical CDFs F ^ i \hat{F}_{i} of conformity scores for each output dimension i ∈ [ d ] i\in[d] , and 𝒟 cal-2 \mathcal{D}_{\text{cal-2}} to calibrate the copula parameters. The optimal thresholds s 1 ∗ , … , s d ∗ s_{1}^{*},\dots,s_{d}^{*} are obtained by minimizing a coverage-based loss, ensuring marginal validity while reducing region size. The final prediction region is 
 
 
 
 C CopulaCPTS ​ ( X ) = { y ∈ 𝒴 : S i ​ ( X , y i ) < s i ∗ , ∀ i ∈ [ d ] } . C_{\text{CopulaCPTS}}(X)=\{y\in\mathcal{Y}:S_{i}(X,y_{i})<s_{i}^{*},\ \forall i\in[d]\}. 
 
 
 Other copula’s based strategies include messoudi2021copula ; mukama2025copula . 
 
 • 
 
 HR ( romano2019conformalized ; zhou2024conformalized ) : Constructs axis-aligned (hyper-rectangular) prediction regions by fitting univariate quantiles q ~ α / 2 ​ ( x ) i \tilde{q}_{\alpha/2}(x)_{i} and q ~ 1 − α / 2 ​ ( x ) i \tilde{q}_{1-\alpha/2}(x)_{i} folowing romano2019conformalized for each output dimension i ∈ [ k ] i\in[k] , with α ~ = 2 ​ ( 1 − ( 1 − α ) 1 / k ) \tilde{\alpha}=2(1-(1-\alpha)^{1/k}) . The conformity score is defined as (inspired from zhou2024conformalized which extends uni-variate scorings to multi-variate ones) 
 
 
 
 S HR ​ ( X , Y ) = max i ∈ [ k ] ⁡ { q ~ α / 2 ​ ( X ) i − Y i , Y i − q ~ 1 − α / 2 ​ ( X ) i } , S_{\text{HR}}(X,Y)=\max_{i\in[k]}\left\{\tilde{q}_{\alpha/2}(X)_{i}-Y_{i},\;Y_{i}-\tilde{q}_{1-\alpha/2}(X)_{i}\right\}, 
 
 
 
 
 yielding rectangular prediction regions aligned with coordinate axes. 
 
 
 
 
 Finally, among the strategies which minimizes the size of the prediction sets, we use: 
 
 • 
 
 MVCS ( braun2025minimum ) : Minimizes the volume of the sets { y ∈ ℝ k , ‖ M ​ ( X ) ​ ( y − f ​ ( X ) ) ‖ p ≤ 1 } \{y\in\mathbb{R}^{k},\|M(X)(y-f(X))\|_{p}\leq 1\} where M ​ ( X ) M(X) is positive definite, f ​ ( X ) ∈ ℝ k f(X)\in\mathbb{R}^{k} , and p > 0 p>0 defines a p p -norm, while ensuring valid marginal coverage. The conformalization set is done with the score S ( X , Y ) = ∥ M ( X ) ( Y − f ( X ) ∥ p S(X,Y)=\|M(X)(Y-f(X)\|_{p} . 
 
 
 
 
 
 F Strong classifiers 
 
 Here, we provide more details on the classifiers used in Section ˜ 4 . In the following, we will refer to training and test data for the data that the classifier h h is trained and evaluated on, not to be confused with the data that the original prediction set method C α C_{\alpha} is trained on. We employ the following classifiers in our evaluation: 
 
 
 Tabular foundation models. 
 
 We evaluate the recent models RealTabPFN-2.5 ( grinsztajn2025tabpfn ) and TabICLv1.1 ( qu2025tabicl ) . These models can predict h ​ ( x 1 test ) , … , h ​ ( x n test ) h(x^{\mathrm{test}}_{1}),\ldots,h(x^{\mathrm{test}}_{n}) with a single forward pass through a neural network that takes both the test input and the entire training set into account. They have been found to perform very well already without hyperparameter tuning ( erickson2025tabarena ) . 
 
 
 
 Gradient-boosted decision trees. 
 
 We choose two representatives: CatBoost ( prokhorenkova2018catboost ) is known for its strong default performance. We adopt its hyperparameters from AutoGluon ( erickson2020autogluon ) and TabArena ( erickson2025tabarena ) , using 300 early stopping rounds instead of AutoGluon’s custom early stopping logic. To obtain a faster model, we use LightGBM ( ke2017lightgbm ) with cheaper hyperparameters adapted from the tuned defaults of holzmuller2024better , reducing the number of early stopping rounds to 100 and using cross-entropy loss for early stopping (as for CatBoost). Both CatBoost and LightGBM are fitted in parallel for eight inner cross-validation folds for each outer cross-validation fold, following TabArena. The inner cross-validation folds are used for early stopping, and the final validation predictions are concatenated and used to fit a quadratic scaling post-hoc calibrator ( berta2025structured ) . Test set predictions are made based on the post-hoc calibrator applied to the average of the eight models’ predictions. 
 
 
 
 Bagging models. 
 
 Random Forest ( breiman2001random ) is a popular baseline for tabular ML. Extremely randomized trees (ExtraTrees/XT, geurts2006extremely ) are a more randomized variant that performs similarly while being faster ( erickson2025tabarena ) . We fit both models with 300 estimators and otherwise use the default hyperparameters from scikit-learn ( pedregosa2011scikit ) . 
 
 
 
 PartitionWise 
 
 Partition-wise estimation first groups samples in the feature space and then predicts by using the average label within each group. The predictor relies on KMeans to form the partitions, and the number of clusters is selected through a fourth root rule based on the test set size, which keeps the model flexible while avoiding clusters that are too small. At inference time, each new sample is assigned to its nearest cluster and the model predicts the mean stored for that cluster. 
 
 
 
 Trade-offs. 
 
 We chose to only fit boosted trees with inner cross-validation, both because they need validation sets for early stopping and because they are still reasonably fast with parallelization. The use of cross-validation also allows for the application of post-hoc calibration. Other methods might also benefit from post-hoc calibration, especially for non-L1 metrics, at the cost of higher runtime due to cross-validation. 
 
 
 
 Discussion and other options. 
 
 We omit from-scratch trained tabular neural networks from our comparison as they are relatively slow, especially on CPUs, and their un-tuned performance is suboptimal ( erickson2025tabarena ) . If runtime is less of a concern, for ideal sample-efficiency, automated machine learning methods such as AutoGluon ( erickson2020autogluon ) can be employed that combine multiple models and hyperparameter setting, ensembling, and post-hoc calibration. However, when these methods are used with time limits, the result may not be reproducible. 
 
 
 
 Hardware. 
 
 We ran tabular foundation models on GPUs (NVIDIA RTX8000, V100 and RTX6000) and the other models on CPUs (Cascade Lake Intel Xeon 5217 with 8 cores and AMD EPYC 7302 with 16 cores). 
 
 
 
 G Additional information regarding the experiments 
 
 
 G.1 Details on the datasets 
 
 See Table ˜ 5 for details on the datasets used for the classifiers comparison, Table ˜ 6 for details on the uni-variate datasets and Table ˜ 7 for the multivariate ones. 
 
 
 
 
 Dataset 
 
 
 Number of 
 samples 
 
 
 
 Number 
 of test samples 
 
 
 
 Number 
 of features 
 
 
 physiochemical_protein 
 45730 
 22865 
 9 
 
 Food_Delivery_Time 
 45593 
 22797 
 10 
 
 diamonds 
 53940 
 26970 
 9 
 
 superconductivity 
 21263 
 10632 
 81 
 
 Table 5: Description of the univariate datasets. 
 
 
 
 
 Dataset 
 
 
 Number of 
 samples 
 
 
 
 Number 
 of test samples 
 
 
 
 Number 
 of features 
 
 
 ailerons 
 13750 
 1375 
 40 
 
 bank8FM 
 8192 
 820 
 8 
 
 cpu-act 
 8192 
 820 
 21 
 
 house-8L 
 22784 
 2280 
 8 
 
 miami 
 13932 
 1394 
 16 
 
 sulfur 
 10081 
 1009 
 6 
 
 Table 6: Description of the univariate datasets. 
 
 
 
 
 Dataset 
 
 
 Number of 
 samples 
 
 
 
 Number 
 of test samples 
 
 
 
 Number 
 of features 
 
 
 
 Dimension 
 of targets 
 
 
 Bias 
 7752 
 776 
 22 
 2 
 
 CASP 
 45730 
 4574 
 8 
 2 
 
 House 
 21613 
 2162 
 17 
 2 
 
 rf1 
 9125 
 914 
 64 
 8 
 
 rf2 
 9125 
 914 
 576 
 8 
 
 Taxi 
 61286 
 6130 
 6 
 2 
 
 Table 7: Description of the multivariate datasets. 
 
 
 
 H Additional experiments 
 
 Synthetic dataset 
 
 See Figures 9 & 10 & 11 & 12 . 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 9: Illustration of the estimation of L 2 L_{2} -ERT for different classifiers as a number of sampled data available.
 Left: physiochemical_protein dataset
 Right: Diamonds dataset 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 10: Illustration of the estimation of L 2 L_{2} -ERT for different classifiers as a number of sampled data available.
 Left: Food_Delivery_Time dataset
 Right: Superconductivity dataset 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 11: Illustration of the estimation of KL-ERT for different classifiers as a number of sampled data available.
 Left: physiochemical_protein dataset
 Right: Diamonds dataset 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 12: Illustration of the estimation of KL-ERT for different classifiers as a number of sampled data available.
 Left: Food_Delivery_Time dataset
 Right: Superconductivity dataset 
 
 
 
 Uni-variate regression. 
 
 See Figures  13 & 14 & 15 . 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 13: Metric values averaged across all datasets for all methods in univariate regression. Left: L 1 L_{1} -ERT (lower is better). Right: WCovGap (lower is better) 
 
 
 
 
 
 
 
 
 
 
 
 
 Figure 14: Metric values averaged across all datasets for all methods in univariate regression. Left: L 2 L_{2} -ERT (lower is better). Right: WSC (closer to 0.9 0.9 is better). 
 
 
 Figure 15: Normalized set sizes averaged all datasets in univariate regression, where the normalization is done by dividing each volume by the smallest volume across all methods (smaller is better). 
 
 
 
 
 
 Dataset 
 Method 
 L 1 L_{1} -ERT 
 L 2 L_{2} -ERT 
 WSC 
 WCovGap 
 Size 
 
 
 
 Ailerons 
 
 MSE 
 0.0287 0.0058 0.0287_{0.0058} 
 0.0010 0.0004 0.0010_{0.0004} 
 0.777 0.015 0.777_{0.015} 
 0.025 0.004 0.025_{0.004} 
 1.42 0.03 \mathbf{1.42_{0.03}} 
 
 MVCS 
 0.0249 0.0042 0.0249_{0.0042} 
 0.0007 0.0004 0.0007_{0.0004} 
 0.768 0.019 0.768_{0.019} 
 0.023 0.003 0.023_{0.003} 
 1.43 0.02 1.43_{0.02} 
 
 HR 
 0.0126 0.0000 0.0126_{0.0000} 
 − 0.0004 0.0000 -0.0004_{0.0000} 
 0.804 0.000 ¯ \underline{0.804_{0.000}} 
 0.017 0.000 0.017_{0.000} 
 1.46 0.00 1.46_{0.00} 
 
 DR-CP 
 0.0024 0.0000 0.0024_{0.0000} 
 − 0.0007 0.0000 ¯ \underline{-0.0007_{0.0000}} 
 0.783 0.000 0.783_{0.000} 
 0.009 0.000 \mathbf{0.009_{0.000}} 
 1.46 0.00 1.46_{0.00} 
 
 C-HDR 
 0.0293 0.0000 0.0293_{0.0000} 
 0.0013 0.0000 0.0013_{0.0000} 
 0.783 0.000 0.783_{0.000} 
 0.026 0.000 0.026_{0.000} 
 1.64 0.00 1.64_{0.00} 
 
 PCP 
 0.0072 0.0000 0.0072_{0.0000} 
 − 0.0000 0.0000 -0.0000_{0.0000} 
 0.779 0.000 0.779_{0.000} 
 0.020 0.000 0.020_{0.000} 
 1.58 0.00 1.58_{0.00} 
 
 HD-PCP 
 0.0004 0.0000 0.0004_{0.0000} 
 − 0.0005 0.0000 -0.0005_{0.0000} 
 0.797 0.000 0.797_{0.000} 
 0.027 0.000 0.027_{0.000} 
 1.44 0.00 1.44_{0.00} 
 
 C-PCP 
 0.0393 0.0000 0.0393_{0.0000} 
 0.0020 0.0000 0.0020_{0.0000} 
 0.804 0.000 \mathbf{0.804_{0.000}} 
 0.026 0.000 0.026_{0.000} 
 1.80 0.00 1.80_{0.00} 
 
 CP2-PCP 
 0.0052 0.0000 0.0052_{0.0000} 
 − 0.0003 0.0000 -0.0003_{0.0000} 
 0.799 0.000 0.799_{0.000} 
 0.022 0.000 0.022_{0.000} 
 1.69 0.00 1.69_{0.00} 
 
 L-CP 
 0.0132 0.0000 0.0132_{0.0000} 
 − 0.0001 0.0000 -0.0001_{0.0000} 
 0.779 0.000 0.779_{0.000} 
 0.020 0.000 0.020_{0.000} 
 1.55 0.00 1.55_{0.00} 
 
 STDQR 
 − 0.0018 0.0000 \mathbf{-0.0018_{0.0000}} 
 − 0.0005 0.0000 -0.0005_{0.0000} 
 0.790 0.000 0.790_{0.000} 
 0.015 0.000 ¯ \underline{0.015_{0.000}} 
 1.44 0.00 1.44_{0.00} 
 
 CopulaCPTS 
 − 0.0004 0.0000 ¯ \underline{-0.0004_{0.0000}} 
 − 0.0009 0.0000 \mathbf{-0.0009_{0.0000}} 
 0.794 0.000 0.794_{0.000} 
 0.022 0.000 0.022_{0.000} 
 1.44 0.00 1.44_{0.00} 
 
 
 
 MiamiHousing2016 
 
 MSE 
 0.0736 0.0050 0.0736_{0.0050} 
 0.0088 0.0015 0.0088_{0.0015} 
 0.647 0.014 0.647_{0.014} 
 0.056 0.012 0.056_{0.012} 
 0.91 0.02 0.91_{0.02} 
 
 MVCS 
 0.0509 0.0078 0.0509_{0.0078} 
 0.0047 0.0013 0.0047_{0.0013} 
 0.713 0.023 0.713_{0.023} 
 0.040 0.006 0.040_{0.006} 
 0.82 0.02 \mathbf{0.82_{0.02}} 
 
 HR 
 0.0274 0.0001 0.0274_{0.0001} 
 0.0005 0.0002 0.0005_{0.0002} 
 0.771 0.001 0.771_{0.001} 
 0.028 0.001 0.028_{0.001} 
 0.87 0.00 0.87_{0.00} 
 
 DR-CP 
 0.0308 0.0003 0.0308_{0.0003} 
 0.0010 0.0002 0.0010_{0.0002} 
 0.746 0.000 0.746_{0.000} 
 0.033 0.001 0.033_{0.001} 
 0.85 0.00 ¯ \underline{0.85_{0.00}} 
 
 C-HDR 
 0.0157 0.0003 0.0157_{0.0003} 
 0.0002 0.0003 0.0002_{0.0003} 
 0.800 0.003 ¯ \underline{0.800_{0.003}} 
 0.018 0.000 0.018_{0.000} 
 0.95 0.00 0.95_{0.00} 
 
 PCP 
 0.0277 0.0005 0.0277_{0.0005} 
 0.0011 0.0005 0.0011_{0.0005} 
 0.766 0.003 0.766_{0.003} 
 0.022 0.000 0.022_{0.000} 
 0.96 0.01 0.96_{0.01} 
 
 HD-PCP 
 0.0324 0.0069 0.0324_{0.0069} 
 0.0016 0.0004 0.0016_{0.0004} 
 0.757 0.000 0.757_{0.000} 
 0.021 0.001 0.021_{0.001} 
 0.86 0.00 0.86_{0.00} 
 
 C-PCP 
 0.0184 0.0026 0.0184_{0.0026} 
 0.0002 0.0003 ¯ \underline{0.0002_{0.0003}} 
 0.802 0.002 \mathbf{0.802_{0.002}} 
 0.023 0.002 0.023_{0.002} 
 1.06 0.00 1.06_{0.00} 
 
 CP2-PCP 
 0.0294 0.0040 0.0294_{0.0040} 
 0.0007 0.0003 0.0007_{0.0003} 
 0.777 0.001 0.777_{0.001} 
 0.017 0.001 ¯ \underline{0.017_{0.001}} 
 1.01 0.00 1.01_{0.00} 
 
 L-CP 
 0.0018 0.0027 \mathbf{0.0018_{0.0027}} 
 − 0.0003 0.0000 \mathbf{-0.0003_{0.0000}} 
 0.782 0.000 0.782_{0.000} 
 0.015 0.001 \mathbf{0.015_{0.001}} 
 0.93 0.00 0.93_{0.00} 
 
 STDQR 
 0.0171 0.0012 0.0171_{0.0012} 
 0.0003 0.0001 0.0003_{0.0001} 
 0.764 0.000 0.764_{0.000} 
 0.018 0.000 0.018_{0.000} 
 0.85 0.00 0.85_{0.00} 
 
 CopulaCPTS 
 0.0128 0.0001 ¯ \underline{0.0128_{0.0001}} 
 0.0002 0.0000 0.0002_{0.0000} 
 0.792 0.000 0.792_{0.000} 
 0.021 0.000 0.021_{0.000} 
 0.89 0.00 0.89_{0.00} 
 
 
 
 bank8FM 
 
 MSE 
 0.0976 0.0082 0.0976_{0.0082} 
 0.0249 0.0026 0.0249_{0.0026} 
 0.535 0.029 0.535_{0.029} 
 0.043 0.010 0.043_{0.010} 
 0.83 0.04 0.83_{0.04} 
 
 MVCS 
 0.0605 0.0089 0.0605_{0.0089} 
 0.0091 0.0025 0.0091_{0.0025} 
 0.639 0.038 0.639_{0.038} 
 0.021 0.008 0.021_{0.008} 
 0.71 0.01 \mathbf{0.71_{0.01}} 
 
 HR 
 0.0244 0.0000 0.0244_{0.0000} 
 0.0007 0.0000 0.0007_{0.0000} 
 0.744 0.000 0.744_{0.000} 
 0.029 0.000 0.029_{0.000} 
 0.88 0.00 0.88_{0.00} 
 
 DR-CP 
 0.0588 0.0007 0.0588_{0.0007} 
 0.0045 0.0001 0.0045_{0.0001} 
 0.744 0.000 0.744_{0.000} 
 0.034 0.000 0.034_{0.000} 
 0.71 0.00 ¯ \underline{0.71_{0.00}} 
 
 C-HDR 
 0.0330 0.0001 0.0330_{0.0001} 
 0.0007 0.0001 0.0007_{0.0001} 
 0.801 0.003 \mathbf{0.801_{0.003}} 
 0.032 0.000 0.032_{0.000} 
 0.97 0.00 0.97_{0.00} 
 
 PCP 
 0.0357 0.0003 0.0357_{0.0003} 
 0.0023 0.0001 0.0023_{0.0001} 
 0.718 0.001 0.718_{0.001} 
 0.032 0.001 0.032_{0.001} 
 0.83 0.00 0.83_{0.00} 
 
 HD-PCP 
 0.0322 0.0021 0.0322_{0.0021} 
 0.0017 0.0003 0.0017_{0.0003} 
 0.768 0.001 0.768_{0.001} 
 0.025 0.001 0.025_{0.001} 
 0.80 0.00 0.80_{0.00} 
 
 C-PCP 
 0.0215 0.0035 0.0215_{0.0035} 
 0.0003 0.0000 0.0003_{0.0000} 
 0.780 0.000 ¯ \underline{0.780_{0.000}} 
 0.021 0.000 0.021_{0.000} 
 1.07 0.00 1.07_{0.00} 
 
 CP2-PCP 
 0.0136 0.0021 ¯ \underline{0.0136_{0.0021}} 
 − 0.0001 0.0000 \mathbf{-0.0001_{0.0000}} 
 0.755 0.003 0.755_{0.003} 
 0.020 0.000 0.020_{0.000} 
 1.01 0.00 1.01_{0.00} 
 
 L-CP 
 0.0231 0.0003 0.0231_{0.0003} 
 0.0002 0.0001 ¯ \underline{0.0002_{0.0001}} 
 0.745 0.004 0.745_{0.004} 
 0.033 0.000 0.033_{0.000} 
 0.94 0.00 0.94_{0.00} 
 
 STDQR 
 0.0016 0.0018 \mathbf{0.0016_{0.0018}} 
 0.0003 0.0002 0.0003_{0.0002} 
 0.740 0.001 0.740_{0.001} 
 0.019 0.000 0.019_{0.000} 
 0.82 0.00 0.82_{0.00} 
 
 CopulaCPTS 
 0.0237 0.0000 0.0237_{0.0000} 
 0.0011 0.0000 0.0011_{0.0000} 
 0.761 0.000 0.761_{0.000} 
 0.015 0.000 \mathbf{0.015_{0.000}} 
 0.87 0.00 0.87_{0.00} 
 
 
 
 
 
 Dataset 
 Method 
 L 1 L_{1} -ERT 
 L 2 L_{2} -ERT 
 WSC 
 WCovGap 
 Size 
 
 
 
 cpu-act 
 
 MSE 
 0.0752 0.0108 0.0752_{0.0108} 
 0.0142 0.0034 0.0142_{0.0034} 
 0.612 0.034 0.612_{0.034} 
 0.049 0.005 0.049_{0.005} 
 1.08 0.02 1.08_{0.02} 
 
 MVCS 
 0.0764 0.0036 0.0764_{0.0036} 
 0.0238 0.0022 0.0238_{0.0022} 
 0.671 0.012 0.671_{0.012} 
 0.027 0.003 0.027_{0.003} 
 1.05 0.04 1.05_{0.04} 
 
 HR 
 0.0054 0.0015 \mathbf{0.0054_{0.0015}} 
 − 0.0006 0.0000 \mathbf{-0.0006_{0.0000}} 
 0.754 0.005 0.754_{0.005} 
 0.018 0.002 ¯ \underline{0.018_{0.002}} 
 1.01 0.04 1.01_{0.04} 
 
 DR-CP 
 0.0276 0.0072 0.0276_{0.0072} 
 0.0025 0.0013 0.0025_{0.0013} 
 0.770 0.005 0.770_{0.005} 
 0.024 0.001 0.024_{0.001} 
 0.88 0.00 \mathbf{0.88_{0.00}} 
 
 C-HDR 
 0.0353 0.0040 0.0353_{0.0040} 
 0.0019 0.0005 0.0019_{0.0005} 
 0.735 0.011 0.735_{0.011} 
 0.039 0.005 0.039_{0.005} 
 1.01 0.01 1.01_{0.01} 
 
 PCP 
 0.0298 0.0040 0.0298_{0.0040} 
 0.0003 0.0005 0.0003_{0.0005} 
 0.777 0.012 \mathbf{0.777_{0.012}} 
 0.031 0.001 0.031_{0.001} 
 1.00 0.01 1.00_{0.01} 
 
 HD-PCP 
 0.0215 0.0005 0.0215_{0.0005} 
 0.0002 0.0000 0.0002_{0.0000} 
 0.771 0.004 ¯ \underline{0.771_{0.004}} 
 0.024 0.001 0.024_{0.001} 
 0.91 0.00 ¯ \underline{0.91_{0.00}} 
 
 C-PCP 
 0.0275 0.0172 0.0275_{0.0172} 
 0.0007 0.0008 0.0007_{0.0008} 
 0.766 0.005 0.766_{0.005} 
 0.033 0.007 0.033_{0.007} 
 1.19 0.04 1.19_{0.04} 
 
 CP2-PCP 
 0.0166 0.0034 ¯ \underline{0.0166_{0.0034}} 
 − 0.0001 0.0001 ¯ \underline{-0.0001_{0.0001}} 
 0.739 0.007 0.739_{0.007} 
 0.015 0.001 \mathbf{0.015_{0.001}} 
 1.02 0.00 1.02_{0.00} 
 
 L-CP 
 0.0400 0.0028 0.0400_{0.0028} 
 0.0015 0.0003 0.0015_{0.0003} 
 0.741 0.003 0.741_{0.003} 
 0.024 0.003 0.024_{0.003} 
 1.10 0.04 1.10_{0.04} 
 
 STDQR 
 0.0272 0.0047 0.0272_{0.0047} 
 0.0007 0.0003 0.0007_{0.0003} 
 0.768 0.008 0.768_{0.008} 
 0.022 0.002 0.022_{0.002} 
 0.93 0.00 0.93_{0.00} 
 
 CopulaCPTS 
 0.0190 0.0005 0.0190_{0.0005} 
 0.0007 0.0000 0.0007_{0.0000} 
 0.743 0.009 0.743_{0.009} 
 0.018 0.000 0.018_{0.000} 
 1.01 0.03 1.01_{0.03} 
 
 
 
 house-8L 
 
 MSE 
 0.0543 0.0051 0.0543_{0.0051} 
 0.0047 0.0007 0.0047_{0.0007} 
 0.723 0.013 0.723_{0.013} 
 0.039 0.004 0.039_{0.004} 
 1.53 0.01 1.53_{0.01} 
 
 MVCS 
 0.0406 0.0076 0.0406_{0.0076} 
 0.0029 0.0007 0.0029_{0.0007} 
 0.763 0.012 0.763_{0.012} 
 0.023 0.004 0.023_{0.004} 
 1.47 0.01 1.47_{0.01} 
 
 HR 
 0.0138 0.0007 ¯ \underline{0.0138_{0.0007}} 
 0.0001 0.0001 ¯ \underline{0.0001_{0.0001}} 
 0.768 0.011 0.768_{0.011} 
 0.020 0.003 0.020_{0.003} 
 1.51 0.01 1.51_{0.01} 
 
 DR-CP 
 0.0351 0.0031 0.0351_{0.0031} 
 0.0018 0.0004 0.0018_{0.0004} 
 0.770 0.016 0.770_{0.016} 
 0.025 0.006 0.025_{0.006} 
 1.35 0.03 \mathbf{1.35_{0.03}} 
 
 C-HDR 
 0.0110 0.0008 \mathbf{0.0110_{0.0008}} 
 0.0001 0.0000 \mathbf{0.0001_{0.0000}} 
 0.807 0.005 \mathbf{0.807_{0.005}} 
 0.013 0.004 \mathbf{0.013_{0.004}} 
 1.47 0.04 1.47_{0.04} 
 
 PCP 
 0.0222 0.0079 0.0222_{0.0079} 
 0.0008 0.0003 0.0008_{0.0003} 
 0.786 0.007 0.786_{0.007} 
 0.024 0.003 0.024_{0.003} 
 1.65 0.01 1.65_{0.01} 
 
 HD-PCP 
 0.0231 0.0004 0.0231_{0.0004} 
 0.0006 0.0002 0.0006_{0.0002} 
 0.797 0.017 0.797_{0.017} 
 0.019 0.004 0.019_{0.004} 
 1.41 0.03 ¯ \underline{1.41_{0.03}} 
 
 C-PCP 
 0.0158 0.0015 0.0158_{0.0015} 
 0.0003 0.0001 0.0003_{0.0001} 
 0.806 0.017 ¯ \underline{0.806_{0.017}} 
 0.015 0.000 ¯ \underline{0.015_{0.000}} 
 1.72 0.01 1.72_{0.01} 
 
 CP2-PCP 
 0.0160 0.0018 0.0160_{0.0018} 
 0.0004 0.0001 0.0004_{0.0001} 
 0.799 0.015 0.799_{0.015} 
 0.016 0.001 0.016_{0.001} 
 1.74 0.03 1.74_{0.03} 
 
 L-CP 
 0.0214 0.0109 0.0214_{0.0109} 
 0.0032 0.0026 0.0032_{0.0026} 
 0.752 0.024 0.752_{0.024} 
 0.023 0.010 0.023_{0.010} 
 1.70 0.03 1.70_{0.03} 
 
 STDQR 
 0.0219 0.0021 0.0219_{0.0021} 
 0.0003 0.0000 0.0003_{0.0000} 
 0.782 0.013 0.782_{0.013} 
 0.019 0.003 0.019_{0.003} 
 1.45 0.00 1.45_{0.00} 
 
 CopulaCPTS 
 0.0269 0.0042 0.0269_{0.0042} 
 0.0007 0.0003 0.0007_{0.0003} 
 0.770 0.007 0.770_{0.007} 
 0.022 0.003 0.022_{0.003} 
 1.51 0.00 1.51_{0.00} 
 
 
 
 sulfur 
 
 MSE 
 0.0201 0.0063 0.0201_{0.0063} 
 0.0001 0.0003 0.0001_{0.0003} 
 0.753 0.017 0.753_{0.017} 
 0.021 0.005 0.021_{0.005} 
 1.45 0.04 1.45_{0.04} 
 
 MVCS 
 0.0116 0.0099 0.0116_{0.0099} 
 − 0.0002 0.0007 -0.0002_{0.0007} 
 0.744 0.020 0.744_{0.020} 
 0.025 0.003 0.025_{0.003} 
 1.41 0.04 \mathbf{1.41_{0.04}} 
 
 HR 
 0.0061 0.0010 0.0061_{0.0010} 
 − 0.0000 0.0001 -0.0000_{0.0001} 
 0.788 0.002 0.788_{0.002} 
 0.016 0.000 0.016_{0.000} 
 1.54 0.00 1.54_{0.00} 
 
 DR-CP 
 0.0001 0.0024 \mathbf{0.0001_{0.0024}} 
 − 0.0005 0.0001 -0.0005_{0.0001} 
 0.762 0.000 0.762_{0.000} 
 0.018 0.000 0.018_{0.000} 
 1.52 0.00 1.52_{0.00} 
 
 C-HDR 
 0.0186 0.0067 0.0186_{0.0067} 
 − 0.0009 0.0007 \mathbf{-0.0009_{0.0007}} 
 0.789 0.009 0.789_{0.009} 
 0.026 0.005 0.026_{0.005} 
 1.61 0.03 1.61_{0.03} 
 
 PCP 
 0.0159 0.0022 0.0159_{0.0022} 
 0.0004 0.0000 0.0004_{0.0000} 
 0.795 0.002 \mathbf{0.795_{0.002}} 
 0.028 0.001 0.028_{0.001} 
 1.75 0.01 1.75_{0.01} 
 
 HD-PCP 
 0.0126 0.0011 0.0126_{0.0011} 
 − 0.0005 0.0001 -0.0005_{0.0001} 
 0.793 0.004 ¯ \underline{0.793_{0.004}} 
 0.028 0.001 0.028_{0.001} 
 1.56 0.00 1.56_{0.00} 
 
 C-PCP 
 0.0037 0.0077 0.0037_{0.0077} 
 − 0.0006 0.0001 -0.0006_{0.0001} 
 0.771 0.005 0.771_{0.005} 
 0.017 0.003 0.017_{0.003} 
 1.80 0.02 1.80_{0.02} 
 
 CP2-PCP 
 0.0027 0.0030 0.0027_{0.0030} 
 − 0.0008 0.0002 ¯ \underline{-0.0008_{0.0002}} 
 0.791 0.002 0.791_{0.002} 
 0.021 0.001 0.021_{0.001} 
 1.71 0.00 1.71_{0.00} 
 
 L-CP 
 0.0230 0.0026 0.0230_{0.0026} 
 0.0007 0.0001 0.0007_{0.0001} 
 0.789 0.009 0.789_{0.009} 
 0.032 0.002 0.032_{0.002} 
 1.62 0.01 1.62_{0.01} 
 
 STDQR 
 0.0062 0.0003 0.0062_{0.0003} 
 − 0.0005 0.0000 -0.0005_{0.0000} 
 0.784 0.004 0.784_{0.004} 
 0.013 0.000 \mathbf{0.013_{0.000}} 
 1.52 0.00 1.52_{0.00} 
 
 CopulaCPTS 
 0.0007 0.0025 ¯ \underline{0.0007_{0.0025}} 
 − 0.0004 0.0001 -0.0004_{0.0001} 
 0.753 0.003 0.753_{0.003} 
 0.016 0.001 ¯ \underline{0.016_{0.001}} 
 1.51 0.00 1.51_{0.00} 
 
 
 
 
 Multivariate regression 
 
 The metrics EOC and SSC are used to compute the CovGap. 
 
 
 
 
 Dataset 
 Method 
 L 1 L_{1} -ERT 
 L 2 L_{2} -ERT 
 WSC 
 WCovGap 
 Size 
 
 
 
 CASP 
 
 MSE 
 0.0407 0.0040 0.0407_{0.0040} 
 0.0023 0.0004 0.0023_{0.0004} 
 0.818 0.007 0.818_{0.007} 
 0.025 0.003 0.025_{0.003} 
 1.26 0.01 1.26_{0.01} 
 
 MVCS 
 0.0304 0.0039 0.0304_{0.0039} 
 0.0016 0.0005 0.0016_{0.0005} 
 0.834 0.006 0.834_{0.006} 
 0.016 0.002 0.016_{0.002} 
 1.23 0.01 \mathbf{1.23_{0.01}} 
 
 HR 
 0.0252 0.0000 0.0252_{0.0000} 
 0.0008 0.0000 0.0008_{0.0000} 
 0.824 0.000 0.824_{0.000} 
 0.020 0.000 0.020_{0.000} 
 1.32 0.00 1.32_{0.00} 
 
 DR-CP 
 0.0177 0.0000 0.0177_{0.0000} 
 0.0005 0.0000 0.0005_{0.0000} 
 0.836 0.000 0.836_{0.000} 
 0.017 0.000 0.017_{0.000} 
 1.25 0.00 ¯ \underline{1.25_{0.00}} 
 
 C-HDR 
 0.0235 0.0000 0.0235_{0.0000} 
 0.0008 0.0000 0.0008_{0.0000} 
 0.862 0.000 \mathbf{0.862_{0.000}} 
 0.021 0.000 0.021_{0.000} 
 1.35 0.00 1.35_{0.00} 
 
 PCP 
 0.0147 0.0000 0.0147_{0.0000} 
 0.0003 0.0000 0.0003_{0.0000} 
 0.827 0.000 0.827_{0.000} 
 0.022 0.000 0.022_{0.000} 
 1.41 0.00 1.41_{0.00} 
 
 HD-PCP 
 0.0238 0.0000 0.0238_{0.0000} 
 0.0006 0.0000 0.0006_{0.0000} 
 0.817 0.000 0.817_{0.000} 
 0.018 0.000 0.018_{0.000} 
 1.30 0.00 1.30_{0.00} 
 
 C-PCP 
 0.0232 0.0000 0.0232_{0.0000} 
 0.0004 0.0000 0.0004_{0.0000} 
 0.845 0.000 0.845_{0.000} 
 0.009 0.000 \mathbf{0.009_{0.000}} 
 1.44 0.00 1.44_{0.00} 
 
 CP2-PCP 
 0.0154 0.0000 0.0154_{0.0000} 
 0.0003 0.0000 0.0003_{0.0000} 
 0.845 0.000 0.845_{0.000} 
 0.016 0.000 0.016_{0.000} 
 1.43 0.00 1.43_{0.00} 
 
 L-CP 
 0.0117 0.0000 \mathbf{0.0117_{0.0000}} 
 0.0002 0.0000 ¯ \underline{0.0002_{0.0000}} 
 0.846 0.000 ¯ \underline{0.846_{0.000}} 
 0.009 0.000 ¯ \underline{0.009_{0.000}} 
 1.42 0.00 1.42_{0.00} 
 
 STDQR 
 0.0158 0.0000 0.0158_{0.0000} 
 0.0004 0.0000 0.0004_{0.0000} 
 0.819 0.000 0.819_{0.000} 
 0.016 0.000 0.016_{0.000} 
 1.32 0.00 1.32_{0.00} 
 
 CopulaCPTS 
 0.0140 0.0000 ¯ \underline{0.0140_{0.0000}} 
 0.0002 0.0000 \mathbf{0.0002_{0.0000}} 
 0.831 0.000 0.831_{0.000} 
 0.013 0.000 0.013_{0.000} 
 1.34 0.00 1.34_{0.00} 
 
 
 
 bias 
 
 MSE 
 0.0268 0.0174 0.0268_{0.0174} 
 0.0011 0.0010 0.0011_{0.0010} 
 0.722 0.026 ¯ \underline{0.722_{0.026}} 
 0.017 0.005 0.017_{0.005} 
 1.03 0.01 \mathbf{1.03_{0.01}} 
 
 MVCS 
 0.0219 0.0096 0.0219_{0.0096} 
 0.0005 0.0009 0.0005_{0.0009} 
 0.726 0.021 \mathbf{0.726_{0.021}} 
 0.014 0.004 \mathbf{0.014_{0.004}} 
 1.04 0.02 ¯ \underline{1.04_{0.02}} 
 
 HR 
 0.0147 0.0000 0.0147_{0.0000} 
 − 0.0003 0.0000 -0.0003_{0.0000} 
 0.709 0.000 0.709_{0.000} 
 0.029 0.000 0.029_{0.000} 
 1.09 0.00 1.09_{0.00} 
 
 DR-CP 
 0.0338 0.0000 0.0338_{0.0000} 
 0.0027 0.0000 0.0027_{0.0000} 
 0.679 0.000 0.679_{0.000} 
 0.031 0.000 0.031_{0.000} 
 1.20 0.00 1.20_{0.00} 
 
 C-HDR 
 0.0387 0.0000 0.0387_{0.0000} 
 0.0018 0.0000 0.0018_{0.0000} 
 0.696 0.000 0.696_{0.000} 
 0.034 0.000 0.034_{0.000} 
 1.36 0.00 1.36_{0.00} 
 
 PCP 
 0.0181 0.0000 0.0181_{0.0000} 
 − 0.0003 0.0000 ¯ \underline{-0.0003_{0.0000}} 
 0.705 0.000 0.705_{0.000} 
 0.024 0.000 0.024_{0.000} 
 1.31 0.00 1.31_{0.00} 
 
 HD-PCP 
 0.0183 0.0000 0.0183_{0.0000} 
 0.0008 0.0000 0.0008_{0.0000} 
 0.692 0.000 0.692_{0.000} 
 0.029 0.000 0.029_{0.000} 
 1.19 0.00 1.19_{0.00} 
 
 C-PCP 
 0.0028 0.0000 \mathbf{0.0028_{0.0000}} 
 − 0.0006 0.0000 \mathbf{-0.0006_{0.0000}} 
 0.705 0.000 0.705_{0.000} 
 0.027 0.000 0.027_{0.000} 
 1.41 0.00 1.41_{0.00} 
 
 CP2-PCP 
 0.0199 0.0000 0.0199_{0.0000} 
 0.0029 0.0000 0.0029_{0.0000} 
 0.667 0.000 0.667_{0.000} 
 0.039 0.000 0.039_{0.000} 
 1.30 0.00 1.30_{0.00} 
 
 L-CP 
 0.0485 0.0000 0.0485_{0.0000} 
 0.0033 0.0000 0.0033_{0.0000} 
 0.646 0.000 0.646_{0.000} 
 0.042 0.000 0.042_{0.000} 
 1.24 0.00 1.24_{0.00} 
 
 STDQR 
 0.0139 0.0000 ¯ \underline{0.0139_{0.0000}} 
 − 0.0000 0.0000 -0.0000_{0.0000} 
 0.692 0.000 0.692_{0.000} 
 0.022 0.000 0.022_{0.000} 
 1.19 0.00 1.19_{0.00} 
 
 CopulaCPTS 
 0.0446 0.0000 0.0446_{0.0000} 
 0.0023 0.0000 0.0023_{0.0000} 
 0.675 0.000 0.675_{0.000} 
 0.044 0.000 0.044_{0.000} 
 1.21 0.00 1.21_{0.00} 
 
 
 
 house 
 
 MSE 
 0.0599 0.0070 0.0599_{0.0070} 
 0.0064 0.0017 0.0064_{0.0017} 
 0.682 0.021 0.682_{0.021} 
 0.044 0.006 0.044_{0.006} 
 1.09 0.02 1.09_{0.02} 
 
 MVCS 
 0.0605 0.0049 0.0605_{0.0049} 
 0.0071 0.0009 0.0071_{0.0009} 
 0.713 0.018 0.713_{0.018} 
 0.038 0.005 0.038_{0.005} 
 1.02 0.03 \mathbf{1.02_{0.03}} 
 
 HR 
 0.0339 0.0038 0.0339_{0.0038} 
 0.0026 0.0006 0.0026_{0.0006} 
 0.740 0.012 0.740_{0.012} 
 0.038 0.001 0.038_{0.001} 
 1.17 0.02 1.17_{0.02} 
 
 DR-CP 
 0.0450 0.0001 0.0450_{0.0001} 
 0.0037 0.0002 0.0037_{0.0002} 
 0.754 0.002 0.754_{0.002} 
 0.026 0.000 0.026_{0.000} 
 1.07 0.00 ¯ \underline{1.07_{0.00}} 
 
 C-HDR 
 0.0167 0.0054 ¯ \underline{0.0167_{0.0054}} 
 0.0008 0.0001 0.0008_{0.0001} 
 0.758 0.000 0.758_{0.000} 
 0.021 0.000 0.021_{0.000} 
 1.11 0.00 1.11_{0.00} 
 
 PCP 
 0.0513 0.0024 0.0513_{0.0024} 
 0.0049 0.0002 0.0049_{0.0002} 
 0.740 0.000 0.740_{0.000} 
 0.021 0.001 0.021_{0.001} 
 1.21 0.00 1.21_{0.00} 
 
 HD-PCP 
 0.0361 0.0037 0.0361_{0.0037} 
 0.0023 0.0001 0.0023_{0.0001} 
 0.746 0.007 0.746_{0.007} 
 0.026 0.001 0.026_{0.001} 
 1.12 0.00 1.12_{0.00} 
 
 C-PCP 
 0.0181 0.0030 0.0181_{0.0030} 
 0.0005 0.0000 \mathbf{0.0005_{0.0000}} 
 0.770 0.003 0.770_{0.003} 
 0.020 0.000 0.020_{0.000} 
 1.28 0.00 1.28_{0.00} 
 
 CP2-PCP 
 0.0166 0.0042 \mathbf{0.0166_{0.0042}} 
 0.0005 0.0002 ¯ \underline{0.0005_{0.0002}} 
 0.790 0.004 \mathbf{0.790_{0.004}} 
 0.018 0.002 ¯ \underline{0.018_{0.002}} 
 1.27 0.00 1.27_{0.00} 
 
 L-CP 
 0.0286 0.0006 0.0286_{0.0006} 
 0.0018 0.0001 0.0018_{0.0001} 
 0.774 0.000 ¯ \underline{0.774_{0.000}} 
 0.020 0.000 0.020_{0.000} 
 1.22 0.00 1.22_{0.00} 
 
 STDQR 
 0.0382 0.0006 0.0382_{0.0006} 
 0.0027 0.0001 0.0027_{0.0001} 
 0.751 0.007 0.751_{0.007} 
 0.028 0.000 0.028_{0.000} 
 1.14 0.00 1.14_{0.00} 
 
 CopulaCPTS 
 0.0212 0.0013 0.0212_{0.0013} 
 0.0008 0.0001 0.0008_{0.0001} 
 0.766 0.002 0.766_{0.002} 
 0.013 0.000 \mathbf{0.013_{0.000}} 
 1.22 0.00 1.22_{0.00} 
 
 
 
 
 
 Dataset 
 Method 
 L 1 L_{1} -ERT 
 L 2 L_{2} -ERT 
 WSC 
 WCovGap 
 Size 
 
 
 
 rf1 
 
 MSE 
 0.0647 0.0063 0.0647_{0.0063} 
 0.0068 0.0020 0.0068_{0.0020} 
 0.649 0.030 0.649_{0.030} 
 0.060 0.006 0.060_{0.006} 
 0.95 0.00 0.95_{0.00} 
 
 MVCS 
 0.1011 0.0074 0.1011_{0.0074} 
 0.0226 0.0034 0.0226_{0.0034} 
 0.588 0.032 0.588_{0.032} 
 0.071 0.007 0.071_{0.007} 
 0.40 0.01 \mathbf{0.40_{0.01}} 
 
 HR 
 0.0372 0.0134 ¯ \underline{0.0372_{0.0134}} 
 0.0029 0.0025 ¯ \underline{0.0029_{0.0025}} 
 0.717 0.027 0.717_{0.027} 
 0.036 0.000 0.036_{0.000} 
 0.57 0.02 0.57_{0.02} 
 
 DR-CP 
 0.0860 0.0016 0.0860_{0.0016} 
 0.0132 0.0002 0.0132_{0.0002} 
 0.689 0.010 0.689_{0.010} 
 0.053 0.009 0.053_{0.009} 
 0.46 0.02 0.46_{0.02} 
 
 C-HDR 
 0.0484 0.0059 0.0484_{0.0059} 
 0.0032 0.0001 0.0032_{0.0001} 
 0.765 0.021 \mathbf{0.765_{0.021}} 
 0.030 0.010 0.030_{0.010} 
 0.46 0.03 0.46_{0.03} 
 
 PCP 
 0.0958 0.0005 0.0958_{0.0005} 
 0.0101 0.0006 0.0101_{0.0006} 
 0.656 0.009 0.656_{0.009} 
 0.075 0.001 0.075_{0.001} 
 0.48 0.01 0.48_{0.01} 
 
 HD-PCP 
 0.0972 0.0066 0.0972_{0.0066} 
 0.0194 0.0030 0.0194_{0.0030} 
 0.612 0.004 0.612_{0.004} 
 0.070 0.005 0.070_{0.005} 
 0.47 0.01 0.47_{0.01} 
 
 C-PCP 
 0.0324 0.0088 \mathbf{0.0324_{0.0088}} 
 0.0019 0.0017 \mathbf{0.0019_{0.0017}} 
 0.744 0.009 ¯ \underline{0.744_{0.009}} 
 0.028 0.004 \mathbf{0.028_{0.004}} 
 0.48 0.01 0.48_{0.01} 
 
 CP2-PCP 
 0.0448 0.0121 0.0448_{0.0121} 
 0.0042 0.0030 0.0042_{0.0030} 
 0.717 0.000 0.717_{0.000} 
 0.029 0.013 ¯ \underline{0.029_{0.013}} 
 0.46 0.02 0.46_{0.02} 
 
 L-CP 
 0.0550 0.0004 0.0550_{0.0004} 
 0.0059 0.0014 0.0059_{0.0014} 
 0.702 0.005 0.702_{0.005} 
 0.030 0.013 0.030_{0.013} 
 0.46 0.02 0.46_{0.02} 
 
 STDQR 
 0.0934 0.0113 0.0934_{0.0113} 
 0.0152 0.0049 0.0152_{0.0049} 
 0.637 0.021 0.637_{0.021} 
 0.073 0.004 0.073_{0.004} 
 0.48 0.01 0.48_{0.01} 
 
 CopulaCPTS 
 0.0759 0.0005 0.0759_{0.0005} 
 0.0087 0.0023 0.0087_{0.0023} 
 0.733 0.075 0.733_{0.075} 
 0.038 0.006 0.038_{0.006} 
 0.48 0.02 0.48_{0.02} 
 
 
 
 rf2 
 
 MSE 
 0.0498 0.0096 0.0498_{0.0096} 
 0.0033 0.0021 0.0033_{0.0021} 
 0.737 0.016 0.737_{0.016} 
 0.024 0.003 0.024_{0.003} 
 0.95 0.00 0.95_{0.00} 
 
 MVCS 
 0.1100 0.0097 0.1100_{0.0097} 
 0.0329 0.0058 0.0329_{0.0058} 
 0.573 0.042 0.573_{0.042} 
 0.076 0.009 0.076_{0.009} 
 0.53 0.02 0.53_{0.02} 
 
 HR 
 0.0557 0.0019 0.0557_{0.0019} 
 0.0110 0.0001 0.0110_{0.0001} 
 0.714 0.014 0.714_{0.014} 
 0.020 0.005 0.020_{0.005} 
 1.07 0.01 1.07_{0.01} 
 
 DR-CP 
 0.0669 0.0015 0.0669_{0.0015} 
 0.0128 0.0001 0.0128_{0.0001} 
 0.727 0.020 0.727_{0.020} 
 0.022 0.001 0.022_{0.001} 
 0.39 0.00 \mathbf{0.39_{0.00}} 
 
 C-HDR 
 0.0524 0.0099 0.0524_{0.0099} 
 0.0049 0.0021 0.0049_{0.0021} 
 0.757 0.004 \mathbf{0.757_{0.004}} 
 0.021 0.001 0.021_{0.001} 
 0.40 0.00 0.40_{0.00} 
 
 PCP 
 0.0825 0.0007 0.0825_{0.0007} 
 0.0145 0.0027 0.0145_{0.0027} 
 0.701 0.005 0.701_{0.005} 
 0.025 0.007 0.025_{0.007} 
 0.42 0.01 0.42_{0.01} 
 
 HD-PCP 
 0.0722 0.0035 0.0722_{0.0035} 
 0.0115 0.0004 0.0115_{0.0004} 
 0.739 0.011 0.739_{0.011} 
 0.023 0.000 0.023_{0.000} 
 0.42 0.00 0.42_{0.00} 
 
 C-PCP 
 0.0366 0.0053 ¯ \underline{0.0366_{0.0053}} 
 0.0011 0.0000 \mathbf{0.0011_{0.0000}} 
 0.749 0.021 0.749_{0.021} 
 0.016 0.004 0.016_{0.004} 
 0.42 0.00 0.42_{0.00} 
 
 CP2-PCP 
 0.0291 0.0081 \mathbf{0.0291_{0.0081}} 
 0.0013 0.0004 ¯ \underline{0.0013_{0.0004}} 
 0.750 0.022 ¯ \underline{0.750_{0.022}} 
 0.015 0.004 ¯ \underline{0.015_{0.004}} 
 0.42 0.00 0.42_{0.00} 
 
 L-CP 
 0.0580 0.0186 0.0580_{0.0186} 
 0.0077 0.0030 0.0077_{0.0030} 
 0.714 0.015 0.714_{0.015} 
 0.013 0.001 \mathbf{0.013_{0.001}} 
 0.40 0.00 ¯ \underline{0.40_{0.00}} 
 
 STDQR 
 0.0702 0.0079 0.0702_{0.0079} 
 0.0095 0.0016 0.0095_{0.0016} 
 0.743 0.004 0.743_{0.004} 
 0.017 0.004 0.017_{0.004} 
 0.42 0.00 0.42_{0.00} 
 
 CopulaCPTS 
 0.0643 0.0024 0.0643_{0.0024} 
 0.0122 0.0026 0.0122_{0.0026} 
 0.707 0.087 0.707_{0.087} 
 0.031 0.002 0.031_{0.002} 
 1.43 1.02 1.43_{1.02} 
 
 
 
 taxi 
 
 MSE 
 0.0234 0.0031 0.0234_{0.0031} 
 0.0011 0.0001 0.0011_{0.0001} 
 0.815 0.003 0.815_{0.003} 
 0.018 0.001 0.018_{0.001} 
 1.88 0.00 \mathbf{1.88_{0.00}} 
 
 MVCS 
 0.0195 0.0027 0.0195_{0.0027} 
 0.0007 0.0002 0.0007_{0.0002} 
 0.816 0.006 0.816_{0.006} 
 0.020 0.001 0.020_{0.001} 
 3.12 0.01 3.12_{0.01} 
 
 HR 
 0.0044 0.0025 0.0044_{0.0025} 
 0.0001 0.0000 0.0001_{0.0000} 
 0.844 0.002 0.844_{0.002} 
 0.007 0.002 \mathbf{0.007_{0.002}} 
 3.48 0.00 3.48_{0.00} 
 
 DR-CP 
 0.0197 0.0015 0.0197_{0.0015} 
 0.0005 0.0002 0.0005_{0.0002} 
 0.830 0.004 0.830_{0.004} 
 0.014 0.001 0.014_{0.001} 
 2.70 0.01 ¯ \underline{2.70_{0.01}} 
 
 C-HDR 
 0.0128 0.0002 0.0128_{0.0002} 
 0.0002 0.0000 0.0002_{0.0000} 
 0.840 0.004 0.840_{0.004} 
 0.011 0.001 0.011_{0.001} 
 2.74 0.00 2.74_{0.00} 
 
 PCP 
 0.0152 0.0023 0.0152_{0.0023} 
 0.0007 0.0001 0.0007_{0.0001} 
 0.808 0.004 0.808_{0.004} 
 0.020 0.003 0.020_{0.003} 
 3.20 0.01 3.20_{0.01} 
 
 HD-PCP 
 0.0106 0.0011 0.0106_{0.0011} 
 0.0004 0.0002 0.0004_{0.0002} 
 0.833 0.005 0.833_{0.005} 
 0.015 0.002 0.015_{0.002} 
 2.99 0.02 2.99_{0.02} 
 
 C-PCP 
 0.0112 0.0068 0.0112_{0.0068} 
 0.0001 0.0001 0.0001_{0.0001} 
 0.846 0.001 0.846_{0.001} 
 0.009 0.001 ¯ \underline{0.009_{0.001}} 
 3.26 0.01 3.26_{0.01} 
 
 CP2-PCP 
 − 0.0026 0.0047 \mathbf{-0.0026_{0.0047}} 
 − 0.0000 0.0000 \mathbf{-0.0000_{0.0000}} 
 0.849 0.007 \mathbf{0.849_{0.007}} 
 0.012 0.002 0.012_{0.002} 
 3.24 0.02 3.24_{0.02} 
 
 L-CP 
 0.0088 0.0006 0.0088_{0.0006} 
 0.0001 0.0000 0.0001_{0.0000} 
 0.832 0.000 0.832_{0.000} 
 0.009 0.000 0.009_{0.000} 
 3.04 0.00 3.04_{0.00} 
 
 STDQR 
 0.0053 0.0004 0.0053_{0.0004} 
 0.0002 0.0001 0.0002_{0.0001} 
 0.839 0.002 0.839_{0.002} 
 0.010 0.000 0.010_{0.000} 
 3.04 0.01 3.04_{0.01} 
 
 CopulaCPTS 
 0.0030 0.0026 ¯ \underline{0.0030_{0.0026}} 
 0.0000 0.0000 ¯ \underline{0.0000_{0.0000}} 
 0.847 0.000 ¯ \underline{0.847_{0.000}} 
 0.010 0.001 0.010_{0.001} 
 3.38 0.03 3.38_{0.03} 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Generated on Fri Dec 12 18:47:42 2025 by L a T e XML