Decomposition of Graphic Design with Unified Multimodal Model

Hui Nie, Zhao Zhang, Yutao Cheng, Maoke Yang, Gonglei Shi, Qingsong Xie, Jie Shao, Xinglong Wu

Abstract: We propose Layer Decomposition of Graphic Designs (LDGD), a novel vision task that converts composite graphic design (e.g., posters) into structured representations comprising ordered RGB-A layers and metadata. By transforming visual content into structured data, LDGD facilitates precise image editing and offers significant advantages for digital content creation, management, and reuse. This task presents two core challenges: (1) predicting the attribute information (metadata) of each layer, and (2) recovering the occluded regions within overlapping layers to enable high-fidelity image reconstruction. To address this, we present the Decompose Layer Model (DeaM), a large unified multimodal model that integrates a conjoined visual encoder, a language model, and a condition-aware RGB-A decoder. DeaM adopts a two-stage processing pipeline: first generates layer-specific metadata containing information such as spatial coordinates and quantized encodings, and then reconstructs pixel-accurate layer images using a condition-aware RGB-A decoder. Beyond full decomposition, the model supports interactive decomposition via textual or point-based prompts. Extensive experiments demonstrate the effectiveness of the proposed method. The code is accessed at https://github.com/witnessai/DeaM.

Lay Summary: We introduce a new way to break down complex poster designs into separate visual pieces, like text and images, along with useful information about each piece—such as where it appears and how it looks. This helps make editing and reusing designs much easier and more precise. To do this, we built a smart system called DeaM that can understand and separate these design elements. It first figures out the details of each part of a design, then rebuilds each piece as an image layer. It can also respond to user instructions, like clicking on an area or describing what they want to change.

Primary Area: Applications->Computer Vision

Keywords: Layer Decomposition, Graphic Design, Unified Multimodal Model

Submission Number: 9140