Combining Image Regions and Human Activity for Indirect Object Recognition in Indoor Wide-Angle Views

Patrick Peursum, Geoff A. W. West, Svetha Venkatesh

Abstract: Traditional methods of object recognition are reliant on shape and so are very difficult to apply in cluttered, wide angle and low detail views such as surveillance scenes. To address this, a method of indirect object recognition is proposed, where human activity is used to infer both the location and identity of objects. No shape analysis is necessary. The concept is dubbed 'interaction signatures', since the premise is that a human interacts with objects in ways characteristic of the function of that object - for example, a person sits in a chair and drinks from a cup. The human-centred approach means that recognition is possible in low detail views and is largely invariant to the shape of objects within the same functional class. This paper implements a Bayesian network for classifying region patches with object labels, building upon our previous work in automatically segmenting and recognising a human's interactions with the objects. Experiments show that interaction signatures can successfully find and label objects in low detail views and are equally effective at recognising test objects that differ markedly in appearance from the training objects.