Randomized Channel-Pass Mask for Channel-Wise Explanation of Black-Box Models

Hirotaka Hachiya, Daiki Nisawa

Abstract: In recent years, there has been active research on interpreting the classification results of deep models. Among these methods, MC-RISE enables pixel-color-wise interpretation based on the model output for images where pixels have been randomly replaced with a predetermined color. However, this approach requires manually preparing the appropriate color candidates. This study proposes a pixel-channel-wise interpretation method using a Randomized Channel-pass Mask (RaCM), which directly evaluates the importance of the original RGB values of an image through randomly generated masks that pass or exclude color channels of each pixel. Experiments are conducted using the German Traffic Sign Recognition Benchmark and ImageNet datasets. The effectiveness of the proposed method is demonstrated through evaluation metrics such as Insertion, Deletion, and Average DCC.