Likelihood-based Permutation Invariant Loss Function for Probability DistributionsDownload PDF

27 Sep 2018 (modified: 21 Dec 2018)ICLR 2019 Conference Blind SubmissionReaders: Everyone
  • Abstract: We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation. Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any additional sequential algorithm. The proposed method, Set Cross Entropy, has a natural information-theoretic interpretation and is related to the metrics defined for sets. We evaluate the proposed approach in two object reconstruction tasks and a rule learning task.
  • Keywords: Set reconstruction, maximum likelihood, permutation invariance
  • TL;DR: The proposed method, Set Cross Entropy, measures the information-theoretic similarity of sets in a permutation-invariant manner.
15 Replies