Editable Image Geometric Abstraction via Neural Primitive Assembly.

Ye Chen, Bingbing Ni, Xuanhong Chen, Zhangli Hu

Abstract: This work explores a novel image geometric abstraction paradigm based on assembly out of a pool of pre- defined simple parametric primitives (i.e., triangle, rectangle, circle and semicircle), facilitating controllable shape editing in images. While cast as a mixed combinatorial and continuous optimization problem, the above task is approximately reformulated within a token translation neural framework that simultaneously outputs primitive assignments and corresponding transformation and color parameters in an image-to-set manner, thus bypassing complex/non-differentiable graph-matching iterations. To relax the searching space and address the vanishing gradient issue, a novel Neural Soft Assignment scheme that well explores the quasi-equivalence between the assignment in Bipartite b-Matching and opacity-aware weighted multiple rasterization combination is introduced, drastically reducing the optimization complexity. Without ground- truth image abstraction labeling (i.e., vectorized representation), the whole pipeline is end-to-end trainable in a self- supervised manner, based on the linkage of differentiable rasterization techniques. Extensive experiments on several datasets well demonstrate that our framework is able to pre- dict highly compelling vectorized geometric abstraction re- sults with a combination of ONLY four simple primitives, also with VERY straightforward shape editing capability by simple replacement of primitive type, compared to previous image abstraction and image vectorization methods.