Deep Read: A Reading Comprehension System

Lynette Hirschman, Marc Light, Eric Breck, John D. Burger

Abstract: This paper describes initial work on Deep Read , an automated reading comprehension system that accepts arbitrary text input (a story) and answers questions about it. We have acquired a corpus of 60 development and 60 test stories of 3 rd to 6 th grade material; each story is followed by short-answer questions (an answer key was also provided). We used these to construct and evaluate a baseline system that uses pattern matching (bag-of-words) techniques augmented with additional automated linguistic processing (stemming, name identification, semantic class identification, and pronoun resolution). This simple system retrieves the sentence containing the answer 30--40% of the time.