Towards a HOI application for digital home assistants

Megani Rajendran, Chek Tien Tan, Indriyati Atmosukarto, Aik Beng Ng, Simon See

Abstract: Human-object interaction detection (HOI) aims to visually deduce how people interact with surrounding objects. Although there is significant progress in advancing classification models for HOI, data remains a key problem limiting the application of HOI for real-world problems, which in turn results in insufficient applied research in HOI. In this position paper, we hence present a research agenda to aid the creation of this system - a data-centric HOI-based digital home assistant framework. The eventual goal of this digital home assistant is to understand how to apply HOI to enhance daily living experiences for the elderly. The research will embark in two phases - (1) create varied relevant HOI datasets through a novel synthetic data generator, and (2) iteratively develop the digital assistant software artifact using a system-based, user-focused design approach.