Behaviourally-based Synthesis of Scene-aware Footstep Sound

Haonan Cheng, Zhaoye Wang, Hengyan Huang, Juanjuan Cai, Long Ye

Abstract: This paper proposes a behaviourally-based scene-aware (BBSA) footstep sound synthesis method for congruent interaction in virtual environments. Most existing methods focus on how to simulate timbre of footstep sound appropriately, but ignore the impact of user behaviour and virtual scenes on footstep sound, which reduces the sense of immersion and presence. To tackle this issue, we classify common user behaviours and design a set of mapping functions between different behaviours and footsteps. To further enhance user immersion, we propose a ray-casting based scene aware sound synthesis method, which allows to synthesize corresponding footstep sounds in real time for indoor and outdoor scenes with different material surfaces. User studies demonstrate that our proposed BBSA method achieves a higher level of immersion and presence. The complete project code is available: https://github.com/OlyMarco/Behaviourally-based-Synthesis-of-Scene-aware-Footstep-Sound-Demo-.