HNet: Irregular Posture Detection by Understanding Human Character and Core Structures

Seungha Noh, Kangmin Bae, Yuseok Bae, Byong-Dai Lee

Abstract: This paper proposes H3Net that considers detecting people in irregular postures by utilizing human structures and characters. To handle both features, we introduce two attention modules: 1) Human Structure Attention Module (HSAM), which is introduced to focus on the spatial aspects of a person, and 2) Human Character Attention Module (HCAM), which is designed to address the issue of repetitive appearance. HSAM effectively handles both foreground and background information about a human instance and utilizes keypoints to provide additional guidance to predict irregular postures. Meanwhile, HCAM employs ID information obtained from the tracking head, enriching the posture prediction with high-level semantic information. Furthermore, gathering images of people in irregular postures is a challenging task. Therefore, many conventional datasets consist of images with the same actors simulating varying postures in distinct images. To address this problem, we propose a Human ID Dependent Posture (HID2) loss that handles repeated instances. The HID2 loss generates a regularization term by considering duplicated instances to reduce bias. Our experiments demonstrate the effectiveness of H3Net compared to existing algorithms on irregular posture datasets. Furthermore, we show the qualitative results using color-coded masks and bounding boxes. We also provide ablation studies to highlight the significance of our proposed methods.