ChartReformer: Natural Language-Driven Chart Image Editing

Pengyu Yan, Mahesh Bhosale, Jay Lal, Bikhyat Adhikari, David S. Doermann

Abstract: Chart visualizations are essential for data interpretation and communication; however, most charts are only accessible in image format and lack the corresponding data tables and supplementary information, making it difficult to alter their appearance for different scenarios of application. To eliminate the need for original underlying data and information to perform chart editing, we propose ChartReformer, a natural language-driven chart image editing solution that directly edits the charts from the input images with the given instruction prompts. Instead of predicting the plotting code, the key in this method is that we allow the model to comprehend the chart and reason over the prompt to generate the corresponding underlying data table and visual attributes for new charts, enabling a precise and stable editing result. To generalize ChartReformer, we define and standardize the chart editing category and generate the ChartCraft dataset, covering style, layout, format, and data-centric edits. The experiments show promising results for the natural language-driven chart image editing. Our datasets and model are available at: https://github.com/pengyu965/ChartReformer.