Combining pattern-based CRFs and weighted context-free grammars

Rustem Takhanov, Vladimir Kolmogorov

Abstract: We consider two models for the sequence labeling (tagging) problem. The first one is a Pattern-Based Conditional Random Field (PB), in which the energy of a string (chain labeling) x = x 1 ⁢ … ⁢ x n ∈ D n is a sum of terms over intervals [ i , j ] where each term is non-zero only if the substring x i ⁢ … ⁢ x j equals a prespecified word w ∈ Λ . The second model is a Weighted Context-Free Grammar (WCFG) frequently used for natural language processing. PB and WCFG encode local and non-local interactions respectively, and thus can be viewed as complementary. We propose a Grammatical Pattern-Based CRF model (GPB) that combines the two in a natural way. We argue that it has certain advantages over existing approaches such as the Hybrid model of Benedí and Sanchez that combines N -grams and WCFGs. The focus of this paper is to analyze the complexity of inference tasks in a GPB such as computing MAP. We present a polynomial-time algorithm for general GPBs and a faster version for a special case that we call Interaction Grammars .