CENTROID-BASED JOINT REPRESENTATION FOR HUMAN POSE ESTIMATION AND INSTANCE SEGMENTATION

Niaz Ahmad, Jawad Khan, Jeremy Yuhyun Kim, Youngmoon Lee

Keywords: Representation Learning, Pose Estimation, Instance Segmentation, Clustering, Feature Extraction

Abstract:

Joint pose estimation and instance segmentation combines keypoint heatmaps
with segmentation masks for multi-person pose and instance-level segmenta-
tion. Unlike easy cases with explicit heatmap activation, hard cases with im-
plicit heatmap due to multi-person entanglement, overlap, and occlusions requires
joint representation with a segmentation mask in end-to-end training. This pa-
per presents a new centroid-based joint representation method called CENTER-
FOCUS. It follows a bottom-up paradigm to generate Strong Keypoint Feature
Maps for both soft and hard keypoints and improve keypoints detection accuracy
as well as the confidence score by introducing KeyCentroids and a Body Heat
Map. CENTERFOCUS then uses the high-resolution representation of keypoint as
a center of attraction for the pixels in the embedding space to generate MaskCen-
troid to cluster the pixels to a particular human instance to whom it belongs, even
if 70% of the body is occluded. Finally, we propose a new PoseSeg algorithm
that collects the feature representation of a 2D human pose and segmentation for
the joint structure of the pose and instance segmentation. We then experimentally
demonstrate the effectiveness and generalization ability of our system on chal-
lenging scenarios such as occlusions, entangled limbs, and overlapping people.
The experimental results show the effectiveness of CENTERFOCUS outperforms
representative models on the challenging MS COCO and OCHuman benchmarks
in terms of both accuracy and runtime performance, Ablation experiments analyze
the impact of each component of the system. The code will be released publicly.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)