CPT: Controllable & Editable Design Variations with Language Models

Karthik Suresh, Amine Ben Khalifa, Li Zhang, Wei-ting Hsu, Fangzheng Wu, Vinay More, Asim Kadav

Track: Regular paper

Keywords: generative AI, large language models, design variations, controllable design generation, editable design variations, document layout, graphic design automation, color and typography, brand consistency, structured representations, human evaluation

TL;DR: Turn design templates into a tokenized markup so an LLM can controllably vary colors, fonts, and layouts—outputting fully editable, brand-consistent designs.

Abstract: Designing visually diverse and high-quality designs remains a manual, time-consuming process, limiting scalability and personalization in creative workflows. We present a system for generating editable design variations using a decoder-only language model – the Creative Pre-trained Transformer (CPT) – trained to predict visual style attributes in design templates. At the core of our approach is a new representation called Creative Markup Language (CML), a compact, machine-learning–friendly format that captures canvas-level structure, page layout, and element-level details (text, images, and vector graphics), including both content and style. We fine-tune CPT on a large corpus of design templates authored by professional designers, enabling it to learn meaningful, context-aware predictions for attributes such as color schemes and font choices. The model produces semantically structured and stylistically coherent outputs, preserving internal consistency across elements. Unlike generative image models, our system yields fully editable design documents rather than pixel-only images, allowing users to iterate, personalize within a design editor. In experiments, our approach generates contextual color and font variations for existing templates and shows promise in adjusting layouts, all while maintaining design principles.

Submission Number: 53