HAPhy: AI-Assisted Content Creator Supporting Tools for Multi-sensory Haptic Experiences in Immersive Media

Seoyong Nam, Hyunwook Jung, Yongjae Yoo

Abstract: Multi-sensory haptic experience is a key component in immersive media (IM), yet effective methods for authoring such sensory stimuli remain underexplored. In this paper, we introduce HAPhy, an AI-assisted co-creative authoring system that enables users to easily design multi-sensory haptic feedback. Leveraging large language models (LLMs), HAPhy first generates semantic descriptions of virtual objects and then infers suitable haptic rendering parameters and physical properties using both the descriptions and multi-view images. We conducted two user studies (N = 20 and N = 16) to evaluate the usability of HAPhy and the quality of the stimuli generated. Results show that HAPhy outperformed traditional manual authoring methods in usability, effectiveness, and time efficiency. A blinded comparison further showed that HAPhy-authored stimuli were rated comparably to manually authored stimuli and significantly better than AI-only outputs. The findings demonstrate that HAPhy significantly lowers the barrier to creating high-quality immersive media involving rich, diverse haptic feedback, offering a promising approach for advanced immersive media experiences.