StoryCtrl: Customized Story Visualization with Fine-Grained Control

Chubin Chen, Jiashu Zhu, Chenyang Zhu, Jiangshan Wang, Nisha Huang, Chengyu Fang, Jiahong Wu, Xiangxiang Chu, Xiu Li

Keywords: Diffusion Models

Abstract: Recent advancements in story visualization have achieved significant progress through text-to-image models that generate coherent image sequences aligned with narratives. However, despite these advancements, generating customized story visualizations remains challenging. Current methods primarily address identity (ID) fidelity and consistency across frames but overlook the fine-grained control of character attributes, leading to suboptimal generation results. To tackle these limitations, we propose StoryCtrl, an innovative framework that not only preserves identity fidelity but also enables fine-grained control over specific character attributes in generated stories. The proposed framework consists of four key components. First, the CtrlGAN Encoder extracts ID information and inverts visual features to the W+ latent space. Second, the Story-aware Module (SaM) captures attribute changes within the narrative context and assists CtrlGAN in making adjustments during the image encoding stage, enabling fine-grained attribute control. Third, we introduce ID-Consistency Attention (ICA), which ensures consistency in generated story sequences. Finally, we incorporate Customized Guidance Fusion (CGF), which integrates reference image features and prompts to enhance customization. To the best of our knowledge, we are the first to introduce an expanded definition of story visualization and present a method for generating fine-grained character attributes. Extensive qualitative and quantitative experiments demonstrate that our method delivers superior performance in customized story visualization.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2026/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6697