ProKnow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance

Kaushik Roy, Manas Gaur, Misagh Soltani, Vipula Rawte, Ashwin Kalyan, Amit P. Sheth

Abstract: Current Virtual Mental Health Assistants (VMHAs) provide counseling and suggestive care. They refrain from patient diagnostic assistance because of a lack of training on safety-constrained and specialized clinical process knowledge. In this work, we define an ordered set of information that maps to evidence-based guidelines or categories of conceptual understanding to experts in a domain. We also introduce a new dataset of diagnostic conversations guided by safety constraints and process knowledge that healthcare professionals use. We develop a method for natural language question generation (NLG) that collects diagnostic information from the patient interactively. We demonstrate the limitations of using state-of-the-art large-scale language models (LMs) on this dataset. Our method called ProKnow, models the process knowledge through explicitly modeling safety, knowledge capture, and explainability. Using LMs with ProKnow generated 89% safer questions in the depression and anxiety domain. Further, LM based generations question did not adhere to clinical process knowledge. In comparison, ProKnow generations yield a 96% reduction in averaged squared rank error. The Explainability of the generated question is assessed by computing similarity with concepts in depression and anxiety knowledge bases. Overall, irrespective of the type of LMs, ProKnow achieved an averaged 82% improvement over simple pre-trained LMs on safety, explainability, and process-guided question generation. We qualitatively and quantitatively evaluate the efficacy of ProKnow by introducing three new evaluation metrics for safety, explainability, and process knowledge-adherence. For reproducibility, we will make the data and the code repository of ProKnow publicly available upon acceptance.