ShadowSpeak: Is It Possible to Communicate Cross-Room Solely by Decoding Gesture Shadows?

Zhiwen Zheng, Yubo Chen, Shaowei Jiang, Huiyu Zhou, Zhao Huang, Tao Zhang, Jin Liu, Guangyuan Zhang, Xiaoshuai Zhang, Xingru Huang

Keywords: Gesture recognition, Computer Vision, Deep Learning, Pattern recognition

Abstract: Accurately decoding hidden information in dynamic shadows for Non-Line-of-Sight (NLOS) imaging enables us to overcome visual occlusions and perceive or reconstruct obscured targets. This breakthrough holds significant potential for real-world applications such as disaster rescue, autonomous driving, and security surveillance. Conventional algorithms struggle to model the physical propagation of light in space. Furthermore, the signal distortions introduced by nonlinear transformations incur the loss of geometric information about the source scene, limiting sensitivity to subtle shadow variations. To overcome these challenges, we present Radiation-constraint Network (RacoNet) that marries physical propagation simulation with geometric-information recovery to interpret minute gesture signals embedded in dynamic shadows. In RacoNet, Radiance-Constrained Light-Transportation (RCLT) optical propagation is proposed to capture complete light-space information. Meanwhile, Geometric Information Aliment Operation (GIAO) restores source-scene geometry lost in the modulated shadow through layer-by-layer refined prior attention. Moreover, Kolmogorov-Arnold Enhanced Layerwise Nonlinear Reorganization (KA-ELNR) fuses light-space and geometric cues to produce the final decoded output. Extensive experiments show that RacoNet markedly surpasses existing approaches in both accuracy and robustness for dynamic-shadow decoding, confirming the possibility of gesture-based information interaction via shadows.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 10388