Autoregressive activity prediction for low-data drug discovery
Keywords: autoregressive inference, few-shot learning, low-data drug discovery
TL;DR: Autoregressive activity prediction modeling enables support set augmentation in an iterative fashion by including new pseudo-labeled samples which eventually improves the model performance.
Abstract: Autoregressive modeling is the main learning paradigm behind the currently so successful
large language models (LLM). For sequential tasks, such as generating natural language,
autoregressive modeling is a natural choice: the sequence is generated by continuously
appending the next sequence token. In this work, we investigate whether the autoregressive modeling paradigm could also be successfully used for molecular activity and property
prediction models, which are equivalent to LLMs in molecular sciences. To this end, we
formulate autoregressive activity prediction modeling (AR-APM), draw relations to transductive and active learning, and assess the predictive quality of AR-APM models in few-shot
learning scenarios. Our experiments show that using an existing few-shot learning system
without any other changes, except switching to autoregressive mode for inference, improves
∆AUC-PR up to ∼40%. Code is available here: https://github.com/ml-jku/autoregressive_activity_prediction.
Primary Subject Area: Active learning, Data cleaning, acquisition for ML
Paper Type: Research paper: up to 8 pages
DMLR For Good Track: Participate in DMLR for Good Track
Participation Mode: In-person
Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.
Submission Number: 63
Loading