Estimating Context Aware Human-Object Interaction Using Deep Learning-Based Object Recognition Architectures

Iván San Martín Fernández, Sergiu Oprea, John Alejandro Castro-Vargas, Pablo Martinez-Gonzalez, José García Rodríguez

Abstract: In this work, we propose an architecture for predicting plausible person-object interactions based on image visible objects and room recognition. First, the system detects objects in the video using a popular framework named “YOLO” (You Only Look Once) and associates each object with their possible interactions. Then, making use of a convolutional neural network, our algorithm recognizes which is the room that appears in the image and filters possible context aware human-object interactions. The main purpose of this project is helping people with memory failures to perform daily activities. Many people have problems carrying out actions that can be natural for the rest. With the aim to assist them, we are interested in the development of methods which allow remembering them the actions they may have forgotten.