Model-free selective inference and its applications to drug discovery
Keywords: Conformal prediction; selective inference; model-free inference; multiple testing; false discovery rate
TL;DR: A method based on conformal prediction for selecting promising candidates (highly potent drugs) with any machine learning model while controlling the false discovery rate in finite samples.
Abstract: Decision making or scientific discovery pipelines such as drug discovery often involve multiple stages: before any resource-intensive step, there is often an initial screening that uses predictions from a machine learning model to shortlist a few candidates from a large pool. We study screening procedures that aim to select candidates whose unobserved outcomes exceed user-specified values. We develop a method that wraps around any prediction model to produce a subset of candidates while controlling the proportion of falsely selected units. Building upon the conformal inference framework, our method first constructs p-values that quantify the statistical evidence for large outcomes; it then determines the shortlist by comparing the p-values to a threshold introduced in the multiple testing literature. In many cases, the procedure selects candidates whose predictions are above a data-dependent threshold. Our theoretical guarantee holds under mild exchangeability conditions on the samples, generalizing existing results on multiple conformal p-values. We demonstrate the empirical performance of our method via applications to drug discovery datasets.
Submission Number: 3
Loading