Unifying Syntactic and Semantic Abstractions for Deep Neural Networks

Sanaa Siddiqui, Diganta Mukhopadhyay, Mohammad Afzal, Hrishikesh Karmarkar, Kumar Madhukar

Abstract: Deep Neural Networks (DNNs) are being trained and trusted for performing fairly complex tasks, even in safety-critical applications such as autonomous driving, medical diagnosis, and air traffic control. However, these real-world applications tend to rely on very large DNNs to achieve the desired accuracy, making it a challenge for them to be executed in resource-constrained and real-time settings. The size of these networks is also a bottleneck in proving their trustworthiness through formal verification or explanation, limiting the deployability of these networks in safety-critical domains. Therefore, it is imperative to be able to compress these networks while maintaining a strong formal connection while preserving desirable safety properties. Several syntactic abstraction techniques have been proposed that produce an abstract network with a formal guarantee that safety properties will be preserved. These, however, do not take the semantic behaviour of the network into account and thus produce suboptimally large networks. On the other hand, compression and semantic abstraction techniques have been proposed that achieve a significant reduction in network size but only weakly preserve a limited set of safety properties. In this paper, we propose to combine the semantic and syntactic approaches into a single framework to get the best of both worlds. This allows us to guide the abstraction using global semantic information while still providing concrete soundness guarantees based on syntactic constraints. Our experiments on standard neural network benchmarks show that this can produce smaller abstract networks than existing methods while preserving safety properties.

External IDs:dblp:conf/fmics/SiddiquiMAKM24