Automated Test Input Generation for Convolutional Neural Networks by Implementing Multi-objective Evolutionary Algorithms

Lingfeng Zhang, Hiroyuki Sato

Abstract: Deep Neural Networks (DNNs) have been widely applied in safety- and security-critical aspects, where the robustness of the system is of great significance, especially for corner case inputs. Traditionally, a DNN is tested with manually labeled data, which is not only labor-consuming, but also unable to contain statistically rare case inputs.In our work, we design, implement and evaluate the test input generation framework guided by multi-objective functions. The multi-objective functions are formed from neuron coverage, behavioral divergence and perturbation degree. We leverage evolutionary algorithms (EAs) to resolve such optimization problem by generating approximation to Pareto-optimal solutions. By implementing our framework, we successfully generated more than 6,000 test inputs for a convolutional neural network. And the generated test inputs help to improve the system’s accuracy by up to 4.4%.