Tree-Structured Non-Autoregressive Decoding for Sequence-to-Sequence Text Generation

ACL ARR 2025 May Submission7333 Authors

Abstract: Autoregressive Transformer (AT) dominates sequence-to-sequence generation tasks but suffers from high inference latency due to sequential token generation. Non-Autoregressive Transformer (NAT) improves inference efficiency by parallelizing token prediction, yet degrades generation quality. To address these limitations, we propose Tree-structured Non-Autoregressive Decoding (TNAD), a novel paradigm that bridges autoregressive and non-autoregressive decoding. TNAD generates a sentence through a top-down, layer-wise expansion of its constituency parse tree, enabling parallel generation within each layer while preserving contextual dependencies across layers. Experimental results on machine translation and paraphrase generation demonstrate that TNAD outperforms AT in efficiency and NAT in generation quality, thus offering a new alternative to AT and NAT in the trade-off between efficiency and quality.

Paper Type: Short

Research Area: Syntax: Tagging, Chunking and Parsing

Research Area Keywords: syntax-based generation, sequence-to-sequence generation, non-autoregessive decoding

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models

Languages Studied: English, German, Romanian

Submission Number: 7333