Visual 3D tracking of child-adult social interactions

Eunji Chong, Audrey Southerland, Abhijit Kundu, Rebecca M. Jones, Agata Rozga, James M. Rehg

Abstract: We describe an approach to continuously capture children's 3D head pose and location during a tabletop social interaction with an adult examiner. Our approach, called face plus context, utilizes a fixed room camera in conjunction with a head-worn camera on the examiner to simultaneously capture the child's face along with the toys and social partners that provide context. Our system performs head tracking and pose estimation along with multi-target tracking to provide 3D localization and disambiguate identity. We evaluated our method on a dataset of 16 children, including both typically developing and autistic children. We present encouraging results for measuring children's social behaviors, along with validation results using an IMU.