A Real-Time 3D Avatar System for Immersive Virtual Try-On on Public Displays

Yueqian Guo, Tianzhao Li, Xin Lv

Keywords: Virtual Try-On (VTON), Human-Computer Interaction (HCI), Gesture Interaction, Digital Human, Usability Evaluation, Social Acceptability

TL;DR: We built a real-time 3D avatar Virtual Try-On system on public displays, featuring digital human and mid air gesture.

Abstract:

Virtual Try-On (VTON) technology is emerging as a solution for public "mirror" displays, but interaction remains a key challenge. Touchscreens raise hygiene concerns, while mid-air gestures can suffer from fatigue and low social acceptability. This paper presents a real-time 3D avatar system designed to provide an immersive and practical VTON experience. The system's core contribution is the integration of a personalized 3D digital human with real-time motion capture, creating an intuitive "mirror" effect. A key feature is its touchless mid-air gesture framework, designed as a hygienic and immersive alternative to traditional inputs.
To evaluate the system's effectiveness and user experience, we conducted a user study (N=20) comparing our optimized Gesture-Based system (A) against a Touch baseline (B) and a high-latency Gesture system (C). The results validated our system's design: the optimized Gesture system (A) achieved subjective usability (SUS) scores largely equivalent to the Touch baseline (B). Furthermore, our system (A) scored significantly highest on immersion and was rated highest for perceived hygiene.
Crucially, we found that physical fatigue was rated low for the optimized system (A), but high for the high-latency system (C). This demonstrates that user fatigue in gesture interfaces is primarily a product of system latency, not the physical act of gesturing. Our work confirms the viability of 3D avatar systems for public VTON, showing that a responsive gesture-based system can provide a highly immersive and hygienic experience that rivals the usability of touch.

Submission Number: 50