A Simulation-Based Research Strategy for Designing Complex NL Systems

Sharon Oviatt, Philip R. Cohen, Michelle Wang, Jeremy Gaston

Abstract: Basic research is critically needed to guide the development of a new generation of multimodal and multilingual NL systems. This paper summarizes the goals, capabilities, computing environment, and performance characteristics of a new semi-automatic simulation technique. This technique has been designed to support a wide spectrum of empirical studies on highly interactive speech, writing, and multi-modal systems incorporating pen and voice. Initial studies using this technique have provided information on people's language, performance, and preferential use of these communication modalities, either alone or in multimodal combination. One aim of this research has been to explore how the selection of input modality and presentation format can be used to reduce difficult sources of linguistic variability in people's speech and writing, such that more robust system processing results. The development of interface techniques for channeling users' language will be important to the ability of complex NL systems to function successfully in actual field use, as well as to the overall commercialization of this technology. Future extensions of the present simulation research also are discussed.