A New Approach to Animacy Detection

Mark Finlayson

Abstract:

Animacy is a necessary property for a referent to be an agent, and thus animacy detection is
useful for a variety of natural language processing tasks, including word sense disambiguation,
co-reference resolution, semantic role labeling, and others. Prior work treated animacy as a
word-level property, and has developed statistical classifiers to classify words as either animate or
inanimate. We discuss why this approach to the problem is ill-posed, and present a new approach
based on classifying the animacy of co-reference chains. We show that simple voting approaches
to inferring the animacy of a chain from its constituent words perform relatively poorly, and then
present a hybrid system merging supervised machine learning (ML) and a small number of hand-
built rules to compute the animacy of referring expressions and co-reference chains. This method
achieves state of the art performance. The supervised ML component leverages features such as
word embeddings over referring expressions, parts of speech, and grammatical and semantic
roles. The rules take into consideration parts of speech and the hypernymy structure encoded in
WordNet. The system achieves an F1 of 0.88 for classifying the animacy of referring expressions,
which is comparable to state of the art results for classifying the animacy of words, and achieves
an F1 of 0.75 for classifying the animacy of coreference chains themselves. We release our
training and test dataset, which includes 142 texts (all narratives) comprising 156,154 words,
34,698 referring expressions, and 10,941 co-reference chains. We test the method on a subset of
the OntoNotes dataset, showing using manual sampling that animacy classification is 90%±2%
accurate for coreference chains, and 92%±1% for referring expressions. The data also contains
46 folktales, which present an interesting challenge because they often involve characters who
are members of traditionally inanimate classes (e.g., stoves that walk, trees that talk). We show
that our system is able to detect the animacy of these unusual referents with an F1 of 0.95.