Do You See What I See: Using Augmented Reality and Artificial Intelligence

Anonymous

Keywords: AI Ethics, Computer Vision, AI, AR

TL;DR: We explore the challenges of real-world applications of augmented reality (AR) and artificial intelligence (AI) through experiments that demonstrate interactions with an augmented world.

Abstract:

Do You See What I See: Using Augmented Reality and Artificial Intelligence

We explore the challenges of real-world applications of augmented reality (AR) and artificial intelligence (AI) through experiments that demonstrate interactions with an augmented world. We interrogate applications of AR and AI, their limitations, and social impacts amplified by the pandemic. We share code, explore ethical considerations, and future projects.

We conduct three experiments to apply the theory of pose estimation with deep learning [1][2][3][4]. In our first experiment, we implement AR using segmentation to augment the scene captured by laptop webcam. In the second experiment, we use keypoint estimation using a deep neural network for pose estimation. In the last experiment, we implement pose estimation against different backgrounds. These experiments enable us to reflect on how context matters.[5] We ask, who is at the table when applications are built and deployed, and invite you too to reflect on the challenges associated with using AR and AI and our responsibilities as builders as tech.