This is the repository associated with the paper Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025).

It includes:

• An implementation of several conformal methods for multi-output conformal regression, including RCP with different estimators.
• Several base predictors (Gaussian Mixture model, quantile model, and mean prediction model).
• Metrics for marginal coverage, conditional coverage and region size.

Datasets

All datasets except MEPS are directly available in this repository. See step 2 of the installation for downloading MEPS.

Refer to these repositories for more information on the datasets used in this study:

• https://github.com/tsoumakas/mulan
• https://github.com/Shai128/mqr
• https://github.com/Zhendong-Wang/Probabilistic-Conformal-Prediction
• https://github.com/aschnuecker/Superlevel-sets

Installation

Prerequisites

• Python (tested on 3.13.2)

Steps

1. After cloning the repository, install the package with its dependencies:

pip install .

For exact versions ensuring reproducibility, use instead:

pip install -r requirements.txt

2. (Optional) For running experiments on the MEPS dataset, download it according to these instructions, summarized below:

git clone https://github.com/yromano/cqr
cd cqr/get_meps_data/
Rscript download_data.R
# Type y when prompted to download the data.
python main_clean_and_save_to_csv.py
cd ../../
for id in 19 20 21; do mv "cqr/get_meps_data/meps_${id}_reg.csv" "data/feldman/meps_${id}.csv"; done
rm -rf cqr

Reproducing the results

To compute the main results of the paper:

python run.py name="rcp" tuning_type="rcp_all" repeat_tuning=10 device="cpu"

To run experiments with additional types of adjustments, run:

python run.py name="rcp_adjustments" tuning_type="rcp_adjustments" repeat_tuning=10 device="cpu"

To run comparisons with the CPCG method, run:

python run.py name="cpcg" tuning_type="rcp_cpcg" repeat_tuning=10 n_samples_for_region_size=null only_cheap_metrics=True device="cpu" manager="joblib"

To generate the figures based on these results, run analysis_rcp.ipynb in a Jupyter notebook environment.

Citation

If you use RCP we kindly ask you to cite:

@inproceedings{plassier2025,
  title={Rectifying Conformity Scores for Better Conditional Coverage},
  author={Plassier, Vincent and Fishkov, Alexander and Dheur, Victor and Guizani, Mohsen and Taieb, Souhaib Ben and Panov, Maxim and Moulines, Eric},
  booktitle={Forty-second International Conference on Machine Learning},
  year={2025}
}