Vulnerability of Privacy-Preserving Visual Localization against Diffusion-based Attacks

Maxime Pietrantoni, Torsten Sattler, Gabriela Csurka

Keywords: Privacy-preserving visual localization, benchmark, diffusion, segmentation, geometric obfuscation

Abstract:

Driven by the increasing use of visual localization (VL) in AR/VR and autonomous systems, privacy-preserving localization is a critical societal necessity. Current VL systems rely on cloud-based 3D scene representation storage and client-side feature extraction, thus creating significant privacy risks. A privacy breach is framed as a malicious actor recovering privacy-preserving representations being sent from the client to the server. This paper therefore aims at finding out what can be recovered from these representations and comparing the multiple privacy-preserving solutions within the literature. We define privacy as the inability to recover personally identifiable information from image representations, acknowledging that general scene details do not inherently represent a privacy breach. We assess the degree of privacy of a representation by evaluating the amount of sensitive information it contains. To that end, we introduce a new privacy attack in which we train a diffusion model to reconstruct images through conditioning on different groups of privacy-preserving representations.
We then measure what can be recovered in the images through a set of comprehensive experiments, which effectively act as a proxy to evaluate the degree of privacy of the initial representations. We apply this comprehensive evaluation protocol on different privacy-preserving representations and provide the first comparison between multiple branches of privacy-preserving visual localization methods. We plan on releasing code and trained checkpoints.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 9416