Innocent Until Proven Guilty (IUPG): Building Deep Learning Models with Embedded Robustness to Out-Of-Distribution Content

Brody Kutt, William Hewlett, Oleksii Starov, Yuchen Zhou

Abstract: Deep Neural Network classifiers trained with the conventional Categorical Cross-Entropy loss face problems in real-world environments such as a tendency to produce overly confident posterior distributions on out-of-distribution inputs, sensitivity to adversarial noise, and lost performance due to distributional shift. We hypothesize that a central shortcoming -an inability to effectively process out-of-distribution content within inputs-exacerbates each of these setbacks. In response, we propose a novel learning framework called Innocent Until Proven Guilty which prototypes training data clusters or classes within the input space while uniquely leveraging noise and inherently random classes to discover noise-resistant, uniquely identifiable features of the modeled classes. In evaluation, we leverage both academic computer vision datasets and realworld JavaScript and URL datasets for malware classification. Across these interdisciplinary settings, we observe favorable classification performance on test data, decreased loss of performance due to recency bias, decreased false-positive responses on noise samples, and decreased vulnerability in several noisebased attack simulations when compared to a baseline network of equal topology trained with Categorical Cross-Entropy. To the best of our knowledge, ours is the first work that demonstrates significantly decreased vulnerability to blackbox append attacks on malware. By applying the well-known FastGradient Sign Method, we show the potential to combine our framework with existing adversarial learning techniques and discover favorable performance by a significant margin. Our framework is general enough for use with any network topology that could otherwise be trained with Categorical Cross-Entropy.