Video-Audio Multimodal Fall Detection Method

Mahtab Jamali, Paul Davidsson, Reza Khoshkangini, Radu-Casian Mihailescu, Elin Sexton, Viktor Johannesson, Jonas Tillström

Abstract: Falls frequently present substantial safety hazards to those who are alone, particularly the elderly. Deploying a rapid and proficient method for detecting falls is a highly effective approach to tackle this concealed peril. The majority of existing fall detection methods rely on either visual data or wearable devices, both of which have drawbacks. This research presents a multimodal approach that integrates video and audio modalities to address the issue of fall detection systems and enhances the accuracy of fall detection in challenging environmental conditions. This multimodal approach, which leverages the benefits of attention mechanism in both video and audio streams, utilizes features from both modalities through feature-level fusion to detect falls in unfavorable conditions where visual systems alone are unable to do so. We assessed the performance of our multimodal fall detection model using Le2i and UP-Fall datasets. Additionally, we compared our findings with other fall detection methods. The outstanding results of our multimodal model indicate its superior performance compared to single fall detection models.

External IDs:dblp:conf/pricai/JamaliDKMSJT24