FigGen: Text to Scientific Figure Generation

Juan A. Rodriguez, David Vazquez, Issam H. Laradji, Marco Pedersoli, Pau Rodriguez

Keywords: text-to-image, generative models, diffusion

TL;DR: FigGen is a diffusion model that generates scientific figures of papers conditioned on the text from the papers (text-to-figure).

Abstract: The generative modeling landscape has experienced tremendous growth in recent years, particularly in generating natural images and art. Recent techniques have shown impressive potential in creating complex visual compositions while delivering impressive realism and quality. However, state-of-the-art methods have been focusing on the narrow domain of natural images, while other distributions remain unexplored. In this paper, we introduce the problem of text-to-figure generation, that is creating scientific figures of papers from text descriptions. We present FigGen, a diffusion-based approach for text-to-figure as well as the main challenges of the proposed task. Code and models are available at https://github.com/joanrod/figure-diffusion

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/figgen-text-to-scientific-figure-generation/code)