NeuForm: Adaptive Overfitting for Neural Shape Editing

Connor Zhizhen Lin, Niloy Mitra, Gordon Wetzstein, Leonidas Guibas, Paul Guerrero

Keywords: shape editing, localized overfitting, neural shape representations

Abstract: Neural representations are popular for representing shapes as they can be used for data cleanup, model completion, shape editing, and shape synthesis. Current neural representations can be categorized as either overfitting to a single object instance, or representing a collection of objects. However, neither allows accurate editing of neural scene representations: on the one hand, methods that overfit objects achieve highly accurate reconstructions but do not support editing, as they do not generalize to unseen object configurations; on the other hand, methods that represent a family of objects with variations do generalize but produce approximate reconstructions. We propose NeuForm to combine the advantages of both overfitted and generalizable representations by adaptively overfitting a generalizable representation to regions where reliable data is available, while using the generalizable representation everywhere else. We achieve this with a carefully designed architecture and an approach that blends the network weights of the two representations. We demonstrate edits that successfully reconfigure parts of human-made shapes, such as chairs, tables, and lamps, while preserving the accuracy of an overfitted shape representation. We compare with two state-of-the-art competitors and demonstrate clear improvements in terms of plausibility and fidelity of the resultant edits.

TL;DR: We propose NeuForm, a neural 3D shape representation that combines advantages of a generalizable representation (like the ability to handle previously unseen configurations) and overfitted representations (like accurate detail representation).