On the importance of severely testing deep learning models of cognition

Jeffrey S. Bowers, Gaurav Malhotra, Federico Adolfi, Marin Dujmovic, Milton Llera Montero, Valerio Biscione, Guillermo Puebla, John E. Hummel, Rachel F. Heaton

Abstract: Highlights:•We argue that many unwarranted conclusions regarding deep neural network (DNN) and human similarities are drawn because of a lack of severe testing of hypotheses.•We attribute the lack of severe testing to bias of publishing ‘positive’ results that highlight DNN-human similarities.•We argue that a better appreciation of severe testing is needed at both the research and evaluation stages to obtain a better characterization of DNN-human similarities.