HearHere: smartphone based audio localization using time difference of arrival: demo

Ellington Kirby, Seoyoon Park, Yan Wang, Yingying Chen

Abstract: Recent advancements in audio recording on mobile devices have improved audio localization capabilities using phones. Previous research has shown that millimeter level accuracy is capable using an off-the-shelf smartphone. This work demonstrates that such valuable resources in smartphones are readily available for developing consumer applications. We develop a smartphone application called HearHere that utilizes geometric features of sound to categorize a tapping sound within a grid. The geometric features are based on the Time Difference of Arrival (TDoA) of the sound between two microphones. By using this classification, any off the shelf phone with stereo recording capability can be made into a music producing device. The application consists of three major components: system calibration, data collection, and audio localization engine based on TDoA. We demonstrate that we can accurately map the tapping sound six region on various solid surface and turn it into a MIDI controller with six different types of instruments.