Marginal conformal prediction has been widely adopted in drug discovery, yet a new study reveals its shortcomings, particularly for minority classes. On July 7, 2026, Muhammadjon Tursunbadalov and colleagues published their findings, demonstrating that standard methods can leave minority classes significantly underrepresented across various datasets.
Understanding the Shortcomings of Conformal Prediction
The research shows that while standard conformal prediction achieves a global coverage target of 90%, its effectiveness diminishes for minority classes. For instance, minority coverage plummeted to 64.8% on blood-brain-barrier penetration and a staggering 4.2% for clinical-trial toxicity, indicating a concerning trend where rare classes are nearly neglected.
This failure is consistent across different models, including random forests and graph networks, with statistical significance (p < 0.001). The study highlights that the issue is linked to baseline calibration on rare labels rather than the specific architecture of the models used.
The Impact of Imbalance on Prediction Accuracy
The authors explain that the shortfall in minority class coverage correlates with a surplus in majority class predictions, amplified by the imbalance ratio present in the datasets. This conservation identity allows for accurate predictions of the observed gaps, emphasizing the need for a critical examination of prediction methods.
Moreover, the failure persists even when realistic scaffold splits and alternative conformal scores are applied. Aggregate accuracy remains high, which can obscure the serious implications for minority class predictions, making it easy for researchers to overlook.
Class-Conditional Fixes for Improved Coverage
The study proposes a class-conditional approach, known as Mondrian conformal prediction, which successfully restores minority coverage to acceptable levels across all datasets. This method requires only a modest increase in prediction-set size, making it a practical solution for researchers.
Additionally, the authors identify generic molecular scaffolds—such as plain benzene and pyridine cores—as contributing factors to the coverage failures. They suggest a one-number diagnostic to assess these issues and demonstrate that abstaining from affected compounds can enhance the utility of screening campaigns.
Conclusion and Future Directions
This research emphasizes the need for a reevaluation of conformal prediction techniques in drug discovery, especially concerning imbalanced datasets. As Tursunbadalov et al. conclude, addressing the gap in minority class coverage is crucial for ensuring reliable and equitable predictions in the field.
🤖 This article was rewritten by Feed and Figures' editorial AI from a report originally published by arXiv Machine Learning. Facts and quotes are preserved from the original; the rewrite focuses on clarity and structure. For the unedited original, see the source link below.
