Robustness Certification of Visual Perception Models via Camera Motion Smoothing

Hanjiang Hu, Zuxin Liu, Linyi Li, Jiacheng Zhu, Ding Zhao

Keywords: Certifiable Robustness, Camera Motion Perturbation, Robotic Perception

TL;DR: We study the robustness of the visual perception model under camera motion perturbations and we propose a motion smoothing technique to give a certification guarantee under camera motion perturbations for any black-box image classification models.

Abstract: A vast literature shows that the learning-based visual perception model is sensitive to adversarial noises, but few works consider the robustness of robotic perception models under widely-existing camera motion perturbations. To this end, we study the robustness of the visual perception model under camera motion perturbations to investigate the influence of camera motion on robotic perception. Specifically, we propose a motion smoothing technique for arbitrary image classification models, whose robustness under camera motion perturbations could be certified. The proposed robustness certification framework based on camera motion smoothing provides effective and scalable robustness guarantees for visual perception modules so that they are applicable to wide robotic applications. As far as we are aware, this is the first work to provide robustness certification for the deep perception module against camera motions, which improves the trustworthiness of robotic perception. A realistic indoor robotic dataset with a dense point cloud map for the entire room, MetaRoom, is introduced for the challenging certifiable robust perception task. We conduct extensive experiments to validate the certification approach via motion smoothing against camera motion perturbations. Our framework guarantees the certified accuracy of 81.7% against camera translation perturbation along depth direction within -0.1m ~ 0.1m. We also validate the effectiveness of our method on the real-world robot by conducting hardware experiments on the robotic arm with an eye-in-hand camera. The code is available at https://github.com/HanjiangHu/camera-motion-smoothing.

Student First Author: yes

Code: https://github.com/HanjiangHu/camera-motion-smoothing

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2210.04625/code)