Doppelgänger Saliency: Towards More Ethical Person Re-Identification

Brandon Richard Webster, Brian Hu, Keith Fieldhouse, Anthony Hoogs

Abstract: Modern surveillance systems have become increasingly dependent on artificial intelligence to provide actionable information for real-time decision making. A critical question relates to how these systems handle difficult ethical dilemmas, such as the re-identification of similar looking individuals. Potential misidentification of individuals can have severe negative consequences, as evidenced by recent headlines of individuals who were wrongly targeted for crimes they did not commit based on false matches. A computer vision-based saliency algorithm is proposed to help identify pixel-level differences in pairs of images containing visually similar individuals, which we term "doppelgängers." The computed saliency maps can alert human users of the presence of doppelgängers and provide important visual evidence to reduce the potential of false matches in these high-stakes situations. We show both qualitative and quantitative saliency results on doppelgängers found in a video-based person re-identification dataset (MARS) using three different state-of-the-art models. Our results suggest that this novel use of visual saliency can improve overall outcomes by helping human users in the person re-identification setting, while assuring the ethical and trusted operation of surveillance systems.