Abstract: Highlights•Empirical observations on convolutional neural networks are presented.•The size of the receptive field did not guarantee improved accuracy.•We observed that dead pixels exist in modern convolutional neural networks.•The effect of dead pixels is discussed with a solution to resolve them.