A mixture of sparse coding models explaining properties of face neurons related to holistic and parts-based processing

Haruo Hosoya, Aapo Hyvärinen

Abstract: Author summary Does the brain represent an object as a combination of parts or as a whole? Past experiments have found both types of representation; but how can such opposing notions coexist in a single visual system? Here, we introduce a novel theory called mixture of sparse coding models for investigating the possible computational principles underlying the primate visual object processing. We constructed a hierarchical network combining two sparse coding modules that each represented one feature set, of either facial parts or non-facial object parts. Competitive computation between the modules, formalized as Bayesian inference, enabled parts to be recognized with a strong top-down influence from the category of the whole input. We show that the latter computation is crucial to explain in detail neural selectivity and tuning properties that were experimentally reported for a particular face processing region called the middle patch. Thus, we offer the first theoretical account of neural face processing in relation to parts-based and holistic representations.