Art3D-Fusion: A Hybrid Framework for Visual Synthesis with Artistic Control

Kohou Wang, Ping Chen, Zhaoxiang Liu, Xiang Liu, Zezhou Chen, Huan Hu, Xin Wang, Kai Wang, Shiguo Lian

Abstract: Traditional visual synthesis methods suffer from blurring, ghosting, and significant artistic deprivation due to their sole reliance on physical disparity reconstruction. These methods fail to capture the artistic elements that directors carefully design in classic 3D films. To address these issues, we propose Art3D-Fusion, a hybrid framework combining geometric processing with advanced diffusion models. This framework integrates physical depth information with global artistic control, particularly in 0-plane selection. Through depth estimation and optical flow-based disparity matching, we create a pseudo-realistic disparity map that reflects the director’s artistic adjustments. This approach enables precise depth-to-disparity transformation and ensures natural detail restoration and effective occlusion handling. Art3D-Fusion generates high-quality right-view images by driving a diffusion model under an enhanced ControlNet architecture using the artistic disparity map and original left-view image as dual-conditional inputs. This approach accurately represents the director’s artistic vision and allows for the transfer of artistic styles from classic 3D films to new scenes, ensuring the consistent reproduction of different directors’ artistic styles through the estimation and transfer of camera parameters.

External IDs:dblp:conf/icig/WangCLLCHWWL25