Reconstruction of Human Body Pose and Appearance Using Body-Worn IMUs and a Nearby Camera View for Collaborative Egocentric Telepresence

Qian Zhang, Akshay Paruchuri, Young-Woon Cha, Jiabin Huang, Jade Kandel, Howard Jiang, Adrian Ilie, Andrei State, Danielle Albers Szafir, Daniel Szafir, Henry Fuchs

Abstract: We envision a future in which telepresence is available to users any-time and anywhere, enabled by sensors and displays embedded in accessories worn daily, such as watches, jewelry, belt buckles, shoes, and eyeglasses. We present a collaborative approach to 3D reconstruction that combines a set of IMUs worn by a target person with an external view from another nearby person wearing an AR headset, used for estimating the target person's body pose and reconstructing their appearance, respectively.