Crowd Counting by Using Top-k Relations: A Mixed Ground-Truth CNN Framework

Li Dong, Haijun Zhang, Kai Yang, Dongliang Zhou, Jianyang Shi, Jianghong Ma

Abstract: Crowd counting has important applications in the environments of smart cities, such as intelligent surveillance. In this paper, we propose a novel convolutional neural network (CNN) framework for crowd counting with mixed ground-truth, called top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relation-based network (TKRNet). Specifically, the estimated density maps generated in a coarse-to-fine manner are treated as coarse locations for crowds so as to assist our TKRNet to regress the scattered point-annotated ground truth. Moreover, an adaptive top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relation module (ATRM) is proposed to enhance feature representations by leveraging the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> dependencies between the pixels with an adaptive filtering mechanism. Specifically, we first compute the similarity between two pixels so as to select the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations for each position. Then, a weight normalization operation with an adaptive filtering mechanism is proposed to make the ATRM adaptively eliminate the influence from the low correlation positions in the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations. Finally, a weight attention mechanism is introduced to make the ATRM pay more attention to the positions with high weights in the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations. Extensive experimental results demonstrate the effectiveness of our proposed TKRNet on several public datasets in comparison to state-of-the-art methods.