SSR-GNNs: Stroke-based Sketch Representation with Graph Neural Networks

Sheng Cheng, Yi Ren, Yezhou Yang

Keywords: Stroke-based representation, Spatial robustness, Robust feature learning, Novel pattern generation

Abstract: Existing end-to-end visual recognition models do not possess innate spatial invariance and are thus vulnerable to out-of-training attacks. This suggests the need of a better representation design. This paper follows existing cognitive studies to investigate a sketch representation that specify stroke information on vertices and inter-stroke information on edges. The resultant representation, combined with a graph neural network, achieves both high classification accuracy and high robustness against translation, rotation, and stroke-wise parametric and topological attacks thanks to the use of spatially invariant stroke features and GNN architecture. While prior studies demonstrated similar sketch representations for classification and generation, these attempts heavily relied on run-time statistical inference rather than more efficient bottom-up computation via GNN. The presented sketch representation poses good structured expression capability as it enables generation of sketches semantically different from the training dataset. Lastly, we show SSR-GNNs are able to accomplish all tasks (classification, robust feature learning, and novel pattern generation), which shows that the representation is task-agnostic.

One-sentence Summary: The paper presents a Stroke-based Sketch Representation with Graph Neural Networks which is spatially robust, with structured expression capability and is task-agnostic.