The Responsibility Problem in Neural Networks with Unordered Targets

Ben Hayes, Charalampos Saitis, György Fazekas

Keywords: set prediction, responsibility problem, order theory

TL;DR: We prove that there must be infinitely many discontinuities in the mapping from a set to the output layer of a feed-forward network.

Abstract: We discuss the discontinuities that arise when mapping unordered objects to neural network outputs of fixed permutation, referred to as the responsibility problem. Prior work has proved the existence of the issue by identifying a single discontinuity. Here, we show that discontinuities under such models are uncountably infinite, motivating further research into neural networks for unordered data.