Lyra: Lifelong Chart-to-Code Generation via LLM-Driven Multimodal Data Synthesis

ACL ARR 2026 January Submission9592 Authors

Keywords: Multimodal learning, LLM-Driven data synthesis, Chart-to-code, Lifelong learning, Iterative training

Abstract:

Current vision-language models struggle to parse the dense visual content in charts.
The chart-to-code task addresses this by converting visual data into executable rendering code.
Previous approaches rely on static and synthetic datasets, but these collections often lack diversity in chart types and underlying data distributions. This limitation contrasts with real-world charts that evolve continuously across domains.
To bridge this gap, we propose Lyra, a fully automated lifelong learning framework for chart-to-code generation.
We employ a large language model as a strategic data generator that generates chart-code pairs to train the vision-language model.
The vision-language model assesses these samples to identify challenging instances and guides the language model to optimize its generation strategy while simultaneously refining its own charting capabilities through training on the synthesized data.
Our experiments demonstrate that Lyra progressively improves performance across three out-of-domain benchmarks while successfully mitigating catastrophic forgetting and achieving superior sample efficiency compared to prior work.

Paper Type: Long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: multimodality

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 9592