Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling

Ouyu Lan*, Xiao Huang*, Bill Yuchen Lin, He Jiang, Xiang Ren

Keywords: crowdsourcing, domain adaptation, sequence labeling, named entity recognition, weak supervision

TL;DR: A model to contextually aggregate multi-source supervision for sequence learning.

Abstract: Sequence labeling is a fundamental framework for various natural language processing problems including part-of-speech tagging and named entity recognition. Its performance is largely influenced by the annotation quality and quantity in supervised learning scenarios. In many cases, ground truth labels are costly and time-consuming to collect or even non-existent, while imperfect ones could be easily accessed or transferred from different domains. A typical example is crowd-sourced datasets which have multiple annotations for each sentence which may be noisy or incomplete. Additionally, predictions from multiple source models in transfer learning can be seen as a case of multi-source supervision. In this paper, we propose a novel framework named Consensus Network (CONNET) to conduct training with imperfect annotations from multiple sources. It learns the representation for every weak supervision source and dynamically aggregates them by a context-aware attention mechanism. Finally, it leads to a model reflecting the consensus among multiple sources. We evaluate the proposed framework in two practical settings of multi-source learning: learning with crowd annotations and unsupervised cross-domain model adaptation. Extensive experimental results show that our model achieves significant improvements over existing methods in both settings.

Code: https://www.dropbox.com/sh/ru7vdss4xsv2j29/AADji_r6MXGVi5-97mngNCV3a?dl=0

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1910.04289/code)