Unlocking Slot Attention by Changing Optimal Transport CostsDownload PDF

Published: 21 Oct 2022, Last Modified: 21 Apr 2024nCSI WS @ NeurIPS 2022 OralReaders: Everyone
Keywords: slot attention, optimal transport, object-centric, attention, multiset, equivariance
TL;DR: Slot attention can do tiebreaking by changing the costs for optimal transport to minimize entropy, which improves results significantly on object detection
Abstract: Slot attention is a successful method for object-centric modeling with images and videos for tasks like unsupervised object discovery. However, set-equivariance limits its ability to perform tiebreaking, which makes distinguishing similar structures difficult – a task crucial for vision problems. To fix this, we cast cross-attention in slot attention as an optimal transport (OT) problem that has solutions with the desired tiebreaking properties. We then propose an entropy minimization module that combines the tiebreaking properties of unregularized OT with the speed of regularized OT. We evaluate our method on CLEVR object detection and observe significant improvements from 53% to 91% on a strict average precision metric.
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 1 code implementation](https://www.catalyzex.com/paper/arxiv:2301.13197/code)
4 Replies