X-NR: Towards An Extended Reality-Driven Human Evaluation Framework for Neural-Rendering

Lorenzo Stacchio, Emanuele Balloni, Lucrezia Gorgoglione, Marina Paolanti, Emanuele Frontoni, Roberto Pierdicca

Abstract: The joint usage of Extended Reality (XR) and Artificial Intelligence (AI) has enabled different Metaverse-related use cases. Such paradigms were recently adopted for immersive content creation, particularly considering Neural Rendering (NR) techniques to project scenes from the real world in the 3D realm. These methods are particularly beneficial in the field of Cultural Heritage (CH), where digitizing and visualizing cultural assets in 3D is crucial. However, current evaluation protocols lack a robust integration of human judgments through a Human-In-The-Loop (HITL) approach to humanly evaluate the quality of the generated 3D models, which could also support model optimization. To bridge this gap, we here introduce X-NR, a novel XR framework designed to evaluate and compare 3D reconstruction methodologies, including NR in the context of CH. We contextualize and validate such a framework through case studies on cultural heritage sites in the Marche region (Italy), employing various data-capturing and 3D reconstruction methodologies. The study concludes with a validation of the framework by CH domain experts, underscoring its potential advantages over traditional 3D editing software.