Improving Set Function Approximation with Quasi-Arithmetic Neural Networks

Tomas Tokar, Scott Sanner

Keywords: representation learning, set function learning

TL;DR: We introduce quasi-arithmetic neural networks (QUANNs), which use Neuralized Kolmogorov Mean---newly introduced trainable pooling mechanism---to improve set functions approximation.

Abstract: Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such as DeepSets and PointNet rely on fixed, non-learnable pooling operations (e.g., sum or max) -- a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed as more expressive alternatives. In this work, we advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) -- a novel, trainable framework for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as a learnable aggregation function. We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions and, thanks to their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 3243