Implicit Bayesian Markov Decision Process for Resource-Efficient Experimental Design in Drug Discovery

Tianchi Chen, Jan Bíma, Sean L. Wu, Otto Ritter, Bo Yuan, Bingjia Yang, Xiang Yu

Track: Biology: datasets and/or experimental results

Nature Biotechnology: Yes

Keywords: Implicit Bayesian Markov Decision Process, Sequential Decision Making, Resource Efficiency, Ensemble Methods, Drug Discovery

TL;DR: We propose an implicit Bayesian MDP that leverages historical data and ensemble methods to efficiently optimize experimental assay scheduling in drug discovery.

Abstract: In drug discovery, researchers make sequential decisions to schedule experiments, aiming to maximize probability of success towards drug candidates while simultaneously minimizing expected costs. However, such tasks pose significant challenges due to complex trade-offs between uncertainty reduction and allocation of constrained resources in a high-dimensional state-action space. Traditional methods based on simple rule-based heuristics or domain expertise often result in either inefficient resource utilization due to risk aversion or missed opportunities arising from reckless decisions. To address these challenges, we developed a Implicit Bayesian Markov Decision Process (IB-MDP) algorithm that constructs an implicit MDP model of the environment’s dynamics by integrating historical data through a similarity-based metric, and enables effective planning by simulating future states and actions. To enhance the robustness of the decision-making process, the IB-MDP also incorporates an ensemble approach that recommends maximum likelihood actions to effectively balance the dual objectives of reducing state uncertainty and optimizing expected costs. Our experimental results demonstrate that the IB-MDP algorithm offers significant improvements over traditional rule-based methods by identifying optimal decisions that ensure more efficient use of resources in drug discovery.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~Tianchi_Chen1

Format: Yes, the presenting author will attend in person if this work is accepted to the workshop.

Funding: No, the presenting author of this submission does *not* fall under ICLR’s funding aims, or has sufficient alternate funding.

Submission Number: 12