PM-Jewelry: Personalized Multimodal Adaptation for Virtual Jewelry Try-On with Latent Diffusion

Yangfan He, Yinghui Xia, Jinfeng Wei, TIANYU SHI, Yang Jingsong

Keywords: Multimodal Learning Latent Diffusion Personalized Adaptation Virtual Jewelry Try-On

Abstract: Virtual jewelry try-on systems offer an innovative way for users to experience personalized, realistic jewelry interactions in an online setting. This paper introduces PM-Jewelry, a virtual try-on framework designed to leverage multimodal learning and personalized adaptation techniques for an enhanced user experience. By integrating data from multiple modalities—such as images, text descriptions, and user interaction data—the model generates lifelike simulations of jewelry on various users. Using a latent diffusion framework, PM-Jewelry captures the intricate details of different jewelry types (e.g., rings, earrings, necklaces), ensuring high precision in aspects like texture, shine, and fit. The model further incorporates personalized adaptation mechanisms, allowing users to tailor the virtual experience to their preferences. Extensive experiments demonstrate the system’s ability to handle diverse jewelry types while preserving critical details, making PM-Jewelry a scalable and robust solution for virtual jewelry try-on. This work also explores challenges such as realistic rendering, jewelry alignment, and material texture simulation, offering insights into future developments in multimodal virtual try-on technologies.

Submission Number: 13