HeatC: A Variable-Grained Coverage Criterion for Deep Learning Systems

Weidi Sun, Yuteng Lu, Xiaokun Luan, Meng Sun

Abstract: Deep learning (DL) systems have achieved significant success in numerous cutting-edge fields. However, the deployment of DL systems in safety-critical areas has raised public concerns about their correctness and robustness. To provide testing evidence for the dependable behavior of Deep Neural Networks (DNNs), various DL coverage criteria have been proposed. These coverage criteria are often “ad-hoc” in terms of granularity for different tasks, but designing appropriate criteria for every possible usage scenario is infeasible and will make the coverage testing lack of uniform standards. In this paper, we proposes a variable-grained DL coverage criterion named HeatC as a common solution for different coverage testing tasks. HeatC leverages class-activation-map-based features from neural networks and clusters these features to generate test targets. Experiments demonstrate that HeatC outperforms existing mainstream coverage criteria in assessing the adequacy of test suites and selecting high-value test samples from unlabeled datasets.