SHADE: Semantic Hypernym Annotator for Domain-Specific Entities - Dungeons and Dragons Domain Use Case

Akila Peiris, Nisansa de Silva

Abstract: Manual data annotation is an important NLP task but one that takes a considerable amount of resources and effort. In spite of the costs, labelling and categorizing entities are essential for NLP tasks such as semantic evaluation. Even though annotation can be done by non-experts in most cases, due to the fact that this requires human labour, the process is costly. Another major challenge encountered in data annotation is maintaining annotation consistency. Annotation efforts are typically carried out by teams of multiple annotators. The annotations need to maintain consistency in relation to both the domain truth and annotation format while reducing human errors. Annotating a specialized domain that deviates significantly from the general domain, such as fantasy literature, will see a significant amount of human error and annotator disagreement. So it is vital that proper guidelines and error reduction mechanisms are enforced. One way to enforce these constraints is by using a specialized application. Such an app can ensure that the notations are consistent, and the labels can be pre-defined or restricted reducing the room for errors. In this paper, we present SHADE, an annotation software that can be used to annotate entities in the high fantasy literature domain Dungeons and Dragons extracted from the Forgotten Realms Fandom Wiki.