AutoML Adoption in ML Software

Koen Van der Blom, Alex Serban, Holger Hoos, Joost Visser

Keywords: AutoML, software engineering, engineering practices, survey, interviews

TL;DR: This paper studies the adoption of AutoML in a software engineering context for applications with ML components through surveys, and reasons that influence the decision to adopt AutoML techniques or not through interviews.

Abstract: Machine learning (ML) has become essential to a vast range of applications, while ML experts are in short supply. To alleviate this problem, AutoML aims to make ML easier and more efficient to use. Even so, it is not clear to which extent AutoML techniques are actually adopted in an engineering context, nor what facilitates or inhibits adoption. To study this, we define AutoML engineering practices, measure their adoption through surveys, and distil first insights into factors influencing adoption from two initial interviews. Depending on the practice, results show around 20 to 30% of the respondents have not adopted it at all and many more only partially, leaving substantial room for increases in adoption. The interviews indicate adoption may in part be inhibited by usability issues with AutoML frameworks and the increased computational resources needed for adoption.

Ethics Statement: The data used for this work originates from survey and interview participants whom consented to using their data in publications. All participants are anonymous, and for demographics only the following high level information was collected and released: team size and experience, geographical region (on country level), and type of organisation. Free form text fields of the published survey data have been scrutinised with care, and any personally identifiable information has been redacted. We believe the collected data is sufficiently abstract such that the ethical risks are minimal.