Recognizing Vector Graphics without Rasterization

XINYANG JIANG, Lu Liu, Caihua Shan, Yifei Shen, Xuanyi Dong, Dongsheng Li

Keywords: Vector Graphics, Object Detection, Image Recognition

TL;DR: We propose an efficient vector graphics based recognition method without rasterization, and takes textual document of the vector graphics as input.

Abstract: In this paper, we consider a different data format for images: vector graphics. In contrast to raster graphics which are widely used in image recognition, vector graphics can be scaled up or down into any resolution without aliasing or information loss, due to the analytic representation of the primitives in the document. Furthermore, vector graphics are able to give extra structural information on how low-level elements group together to form high level shapes or structures. These merits of graphic vectors have not been fully leveraged in existing methods. To explore this data format, we target on the fundamental recognition tasks: object localization and classification. We propose an efficient CNN-free pipeline that does not render the graphic into pixels (i.e. rasterization), and takes textual document of the vector graphics as input, called YOLaT (You Only Look at Text). YOLaT builds multi-graphs to model the structural and spatial information in vector graphics, and a dual-stream graph neural network is proposed to detect objects from the graph. Our experiments show that by directly operating on vector graphics, YOLaT outperforms raster-graphic based object detection baselines in terms of both average precision and efficiency. Code is available at https://github.com/microsoft/YOLaT-VectorGraphicsRecognition.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/microsoft/YOLaT-VectorGraphicsRecognition

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/arxiv:2111.03281/code)