|
|
--- |
|
|
license: mit |
|
|
tags: |
|
|
- arxiv:2602.01285 |
|
|
- conformal-inference |
|
|
- llm |
|
|
- maci |
|
|
--- |
|
|
|
|
|
# MACI: Multi-LLM Adaptive Conformal Inference |
|
|
|
|
|
This is the official repository for the paper **"Multi-LLM Adaptive Conformal Inference for Reliable LLM Responses"**. |
|
|
|
|
|
📄 **Paper**: [arXiv:2602.01285](https://arxiv.org/abs/2602.01285) |
|
|
💻 **Code**: [GitHub Repository](https://github.com/MLAI-Yonsei/MACI) |
|
|
|
|
|
## Abstract |
|
|
Ensuring factuality is essential for the safe use of Large Language Models (LLMs) in high-stakes domains such as medicine and law. Conformal inference provides distribution-free guarantees, but existing approaches are either overly conservative, discarding many true-claims, or rely on adaptive error rates and simple linear models that fail to capture complex group structures. To address these challenges, we reformulate conformal inference in a multiplicative filtering setting, modeling factuality as a product of claim-level scores. Our method, Multi-LLM Adaptive Conformal Inference (MACI), leverages ensembles to produce more accurate factuality-scores, which in our experiments led to higher retention, while validity is preserved through group-conditional calibration. Experiments show that MACI consistently achieves user-specified coverage with substantially higher retention and lower time cost than baselines. |
|
|
## Usage |
|
|
Please refer to our [GitHub Repository](https://github.com/MLAI-Yonsei/MACI) for installation and usage instructions. |
