Explaining machine learning models with interactive natural language conversations using TalkToModel

Dylan Slack, Satyapriya Krishna, Himabindu Lakkaraju, Sameer Singh

Abstract: Practitioners increasingly use machine learning (ML) models, yet models have become more complex and harder to understand. To understand complex models, researchers have proposed techniques to explain model predictions. However, practitioners struggle to use explainability methods because they do not know which explanation to choose and how to interpret the explanation. Here we address the challenge of using explainability methods by proposing TalkToModel: an interactive dialogue system that explains ML models through natural language conversations. TalkToModel consists of three components: an adaptive dialogue engine that interprets natural language and generates meaningful responses; an execution component that constructs the explanations used in the conversation; and a conversational interface. In real-world evaluations, 73% of healthcare workers agreed they would use TalkToModel over existing systems for understanding a disease prediction model, and 85% of ML professionals agreed TalkToModel was easier to use, demonstrating that TalkToModel is highly effective for model explainability. To ensure that a machine learning model has learned the intended features, it can be useful to have an explanation of why a specific output was given. Slack et al. have created a conversational environment, based on language models and feature importance, which can interactively explore explanations with questions asked in natural language.