INRFlow: Flow Matching for INRs in Ambient Space

Yuyang Wang, Anurag Ranjan, Joshua M. Susskind, Miguel Ángel Bautista

TL;DR: A flow-matching model for learning probabilistic INRs at high resolution for multiple data domains

Abstract: Flow matching models have emerged as a powerful method for generative modeling on domains like images or videos, and even on irregular or unstructured data like 3D point clouds or even protein structures. These models are commonly trained in two stages: first, a data compressor is trained, and in a subsequent training stage a flow matching generative model is trained in the latent space of the data compressor. This two-stage paradigm sets obstacles for unifying models across data domains, as hand-crafted compressors architectures are used for different data modalities. To this end, we introduce INRFlow, a domain-agnostic approach to learn flow matching transformers directly in ambient space. Drawing inspiration from INRs, we introduce a conditionally independent point-wise training objective that enables INRFlow to make predictions continuously in coordinate space. Our empirical results demonstrate that INRFlow effectively handles different data modalities such as images, 3D point clouds and protein structure data, achieving strong performance in different domains and outperforming comparable approaches. INRFlow is a promising step towards domain-agnostic flow matching generative models that can be trivially adopted in different data domains.

Lay Summary: We present a new generative model called INRFlow, which is designed to generate different types of data—like images, 3D objects, or protein structures—in a flexible and unified way. Currently popular generative models involve two separate steps: one model to compress the data, and another model to learn to generate the compressed data. However, this approach often requires handcrafting different models for different types of data which makes it hard to use the same method across domains. INRFlow avoids this issue by skipping the compression step and working directly with the original data. Inspired by a concept called "implicit neural representations" (INRs), it learns to understand and generate data continuously across space, treating data points (like pixels) as coordinate-value pairs. We show that INRFlow works well across different kinds of data and doesn't need custom tweaks for each domain. This makes it a promising step toward building a single, general-purpose model that can generate diverse types of data without specialized design.

Primary Area: Deep Learning->Generative Models and Autoencoders

Keywords: Flow Matching, INRs

Submission Number: 7206