Are You Dictating to Me? Detecting Embedded Dictations in Doctor-Patient Conversations

Thomas Schaaf, Longxiang Zhang, Mark Fuhs, Shahid Durrani, Susanne Burger, Monika Woszczyna, Thomas Polzin

Abstract: Medical scribes chart doctor-patient conversations in real time or by listening to an audio recording afterwards. Doctors sometimes dictate during a patient encounter, a highly informative part for a scribe. We introduced a light-weight annotation schema and ana-lyzed recordings of 105 randomly selected doctor-patient encounters from 21 physicians to quantify the frequency and automatically de-tect dictated regions. Dictation behavior of individual doctors was consistent but varied among them. A linguistic analysis is provided to describe differences of doctors speech when talking to a patient or dictating. A description of the data is given, highlighting challenges of segmenting audio into conversation and dictation regions. We in-vestigate different features and methods to segment conversations including keyword spotting, acoustic features and class-conditioned language models. Results are anchored to a majority class base-line. Using only acoustic features allows to predict dictated speech without the need of a speech recognition system performing com-parable to a rule-based approach using lexical features derived from a speech recognition system. Performance is assessed using leave-one-physician-out cross validation and an analysis using a random forest classifier indicates that language model derived features are most useful, and that a combination of acoustic and lexical features performed best.