HF-SRGR: a new hybrid feature-driven social relation graph reasoning model

Lindong Li, Linbo Qing, Yuchen Wang, Jie Su, Yongqiang Cheng, Yonghong Peng

Abstract: Social relations and interactions between persons form the foundation of human society. Effective recognition of social relationships has great potential for understanding and improving people’s psychology and behaviors, e.g., mental health and activity analysis, and further improving social resilience. Existing work of social relation recognition (SRR) mainly focuses on exploiting two or three types of features to recognize social relations without considering the relations between features. In this paper, we proposed a new framework for extraction and fusion of the hybrid features, namely Social Relation Graph Reasoning model driven by Hybrid-Features (HF-SRGR). For the proposed method, a social relation graph was constructed first using relation and scene features as nodes. An attention mechanism was then designed to incorporate into graph neural networks (GNNs), generating inter-pair features and interactions between relation nodes and the scene node, respectively. Besides, the propagation of scene information further strengthens the rationality of interaction reasoning. Extensive experiments on PISC and PIPA datasets show that our proposed approach achieves better performance over the state-of-the-art methods in terms of accuracy.