Efficient Learning of Sparse Conditional Random Fields for Supervised Sequence Labeling

Nataliya Sokolovska, Thomas Lavergne, Olivier Cappé, François Yvon

Abstract: Conditional random fields (CRFs) constitute a popular and efficient approach for supervised sequence labeling. CRFs can cope with large description spaces and can integrate some form of structural dependency between labels. In this paper, we address the issue of efficient feature selection for CRFs based on imposing sparsity through an <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ℓ</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> penalty. We first show how sparsity of the parameter set can be exploited to significantly speed up training and labeling. We then introduce coordinate descent parameter update schemes for CRFs with ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> regularization. We finally provide some empirical comparisons of the proposed approach with state-of-the-art CRF training strategies. In particular, it is shown that the proposed approach is able to take profit of the sparsity to speed up processing and handle larger dimensional models.