​Privacy-Aware Lip Reading: Depth-Sensing with Adaptive Perturbation for Silent Speech Recognition​

Rong Fu, weizhi Tang, Simon James Fong

Keywords: ​Privacy-Preserving Machine Learning, Adaptive Perturbation, Depth-Sensing for Speech, Causal Reasoning in Vision, Lightweight Proxy Alignment, Silent Speech Benchmarking​

TL;DR: Depth-sensing silent speech recognition with adaptive perturbation preserves user privacy while maintaining lip-reading accuracy across diverse hardware deployments.

Abstract: Silent speech recognition using depth sensing offers inherent advantages for privacy-sensitive applications by eliminating sensitive RGB data. However, existing systems remain vulnerable to adversarial inference from raw lip movement patterns. We propose ​Privacy-Aware DepthSpeech, a novel framework integrating ​frequency-domain perturbation​ and ​causality-weighted noise injection​ to protect user identity while maintaining recognition fidelity. By transforming lip sequences into perturbed point clouds, our method dynamically corrupts high-frequency components and causal-sensitive regions via transfer entropy analysis. A ​lightweight proxy model​ trained on non-sensitive data further aligns outputs through multi-scale feature constraints, enabling robust cross-device deployment (on-wrist, on-head, in-environment). Evaluations confirm superior privacy-utility trade-offs against RGB baselines, with enhanced generalizability across physiological diversities and lighting conditions.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: true

Submission Guidelines: true

Anonymous Url: true

No Acknowledgement Section: true

Submission Number: 2624