A Tale of Two Cities: Data and Configuration Variances in Robust Deep Learning

Guanqin Zhang, Jiankun Sun, Feng Xu, Yulei Sui, H. M. N. Dilum Bandara, Shiping Chen, Tim Menzies

Abstract: Deep neural networks (DNNs) have widespread applications in industries such as image recognition, supply chain, medical diagnosis, and autonomous driving. However, previous work has shown that the high accuracy of a DNN model does not imply high robustness (i.e., consistent performances on new and future datasets) because the input data and external environment (e.g., software and model configurations) for a deployed model are constantly changing. Therefore, ensuring robustness is crucial to enhance business and consumer confidence. Previous research focuses mostly on the data aspect of model variance. This article takes a holistic view of DNN robustness by summarizing the issues related to both data and software configuration variances. We also present a predictive framework using search-based optimization to generate representative variances for robust learning, considering data and configurations.