Adaptive Training of INRs via Pruning and Densification

ICLR 2026 Conference Submission18700 Authors

Keywords: implicit neural representation, sinusoidal neural networks, coordinate-based neural networks

TL;DR: We present an adaptive training of low-dimension signals consisting of pruning and densifying the neurons of sinusoidal neural networks.

Abstract:

Encoding input coordinates with sinusoidal functions into multilayer perceptrons
(MLPs) has proven effective for implicit neural representations (INRs) of low-
dimensional signals, enabling the modeling of high-frequency details. However,
selecting appropriate input frequencies and architectures while managing parameter
redundancy remains an open challenge, often addressed through heuristics and
heavy hyperparameter optimization schemes. In this paper, we introduce AIRe
(**A**daptive **I**mplicit neural **Re**presentation), an adaptive training scheme that refines
the INR architecture over the course of optimization. Our method uses a neuron
pruning mechanism to avoid redundancy and input frequency densification to
improve representation capacity, leading to an improved trade-off between network
size and reconstruction quality. For pruning, we first identify less-contributory
neurons and apply a targeted weight decay to transfer their information to the
remaining neurons, followed by structured pruning. Next, the densification stage
adds input frequencies to spectrum regions where the signal underfits, expanding
the representational basis. Through experiments on images and SDFs, we show
that AIRe reduces model size while preserving, or even improving, reconstruction
quality. Code and pretrained models will be released for public use.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 18700