Creating Super-Hearing Capabilities With Real-Time AI

Malek Itani, Bandhav Veluri, Tuochao Chen, Shyamnath Gollakota

Abstract: Sound is a fundamental medium through which we perceive our environment. However, today we are surrounded by a cacophony of sounds that can overwhelm our senses. Indeed, human auditory perception can be limited in noisy environments. Imagine being in a crowded room with a cacophony of sounds and having the ability to focus on or remove sounds from a specific person or based on their semantic descriptions. This requires understanding and manipulating an acoustic scene, isolating each sound, and associating a spatial context or semantic meaning with each constituent sound or speaker—a challenging set of tasks even for the human brain in noisy environments.

External IDs:dblp:journals/pervasive/ItaniVCG24