Riemannian Flow Matching on the Fisher–Rao Sphere for Non-Autoregressive Conditional Text Generation

ACL ARR 2025 May Submission2043 Authors

Abstract:

Diffusion models and Linear Flow matching
have emerged as a promising framework for
fast and high-quality conditional text generation, yet, current approaches often overlook
the inherent geometric structure of text embeddings. In this work, we introduce GeoFM, a
novel flow matching model that directly leverages the Riemannian geometry induced by
the Fisher–Rao metric. Specifically, GeoFM
projects token embeddings onto a Fisher–Rao
sphere via the square-root transform, and learns
a neural velocity field that precisely aligns with
spherical geodesics connecting noisy priors and
target embeddings. Additionally, we propose
a spherical trajectory loss that maintains lexical fidelity and encourages direct, minimally distorted trajectories on the manifold. Our empirical evaluation demonstrates GeoFM’s effectiveness and significant speedups over state-of the-art non-autoregressive baselines.

Paper Type: Short

Research Area: Generation

Research Area Keywords: Dialogue and Interactive Systems, Generation, Machine Learning for NLP, Question Answering

Contribution Types: Approaches low compute settings-efficiency, Theory

Languages Studied: English

Submission Number: 2043