Approximate Structured Diffusion for Sequence Labelling

ACL ARR 2026 January Submission9314 Authors

Keywords: Deep learning, NLP, Diffusion, Sequence tagging, POS tagging

Abstract:

Sequence labelling, a core task of Natural Language Processing (NLP), consists in assigning each token of an input sentence a label.
From a Machine Learning point of view, sequence labelling is often cast as a Linear-Chain Conditional Random Field (CRF) parametrized by a neural network.
While this approach gives good empirical results, CRFs assume a finite decision span (\eg{} label bigrams) which limits their expressivity and hurts performance when long-range dependencies are required.

We show we can leverage diffusion to train a CRF conditioned on an entire label sequence, with the caveat that the condition is on a \emph{noisy} version of labels.
We show experimentally that this method, in conjunction with Mean-Field inference, improves label accuracy while remaining faster than traditional CRF decoding.

Paper Type: Short

Research Area: Machine Learning for NLP

Research Area Keywords: Deep learning, NLP, Diffusion, Sequence tagging, POS tagging

Contribution Types: NLP engineering experiment

Languages Studied: German, English, French, Dutch

Submission Number: 9314