Detecting and grouping keypoints for multi-person pose estimation using instance-aware attention

Sen Yang, Ze Feng, Zhicheng Wang, Yanjie Li, Shoukui Zhang, Zhibin Quan, Shutao Xia, Wankou Yang

Abstract: Highlights•Exploiting the pairwise attention scores between keypoints as the criterion to judge whether they belong to the same person or not.•Using the instance masks to supervise the self-attention to ensure the instance- discriminative characteristics for the use of keypoint grouping.•Using a very simple architecture design to simultaneously detect and group instance-agnostic keypoints into person skeletons.•The instance segmentation results of any number of people can be directly and simply obtained from the supervised attention matrix.