Exploring Spatial Relation Awareness Through Virtual Indoor Environments

Swe Nwe Nwe Htun, Shusaku Egami, Takanori Ugai, Yijun Duan, Ken Fukuda

Abstract: This research addresses the critical challenge of understanding spatial relations in virtual indoor environments. The proposed methodology comprises two core modules: object clustering and spatial relation extraction. Object clustering employs a Bayesian probabilistic model to group closely positioned objects, facilitating a more contextual understanding and simplifying subsequent spatial relation analyses. The spatial relation extraction module integrates depth information with cluster bounding box data, enabling a nuanced assessment of relationships such as above, below, inside, and close between objects. The effectiveness of the proposed approach is evaluated using the VirtualHome2KG dataset, demonstrating its robust performance in capturing spatial relationships. A gold standard is prepared to assess the results of relation extraction, representing a precise set of spatial relationships derived from the 3D object locations in VirtualHome. Experimental phases include clustering stability analysis and 2D spatial relation extraction, providing valuable insights into the temporal dynamics of scenes and the precision of the method. The results underscore the methodology’s efficacy, as evidenced by precision, recall, and accuracy metrics, establishing its potential for evolving intelligent monitoring systems in home environments.