Modularity is the Bedrock of Natural and Artificial Intelligence

TMLR Paper4858 Authors

Abstract: The astonishing performance showcased by AI systems in the last decade has been achieved through the use of massive amounts of data, computation, and, in turn, energy, which vastly exceed what human intelligence requires. This wide gap underscores the need for further research and points to leveraging brains as a valuable source of guiding principles. On the other hand, the No Free Lunch Theorem highlights that effective inductive biases must be problem-specific. This suggests designing architectures with specialized components that can solve subproblems --- namely, modular architectures. Interestingly, modularity is an established principle of brain organization that is considered essential for supporting the efficient learning and strong generalization abilities consistently demonstrated by humans. However, despite its recognized importance in natural intelligence and the proven benefits it has shown across various seemingly unrelated AI research areas, modularity remains somewhat underappreciated in AI. In this work, we review several research threads in artificial intelligence and neuroscience through a conceptual framework that highlights the central role of modularity in supporting both artificial and natural intelligence. In particular, we examine what computational advantages modularity provides, how it has emerged as a solution in several AI research areas, which modularity principles the brain exploits, and how modularity can help bridge the gap between natural and artificial intelligence.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: We have added tables that highlight the main contributions of the paper and link to relevant paper sections. We have also expanded the section 2.2.3, on the Energetic and Computational Benefits of Modular Brain Networks, discussing new topics such as amortized control, partial autonomy, and dynamical complexity. Finally, we have expanded the Implicit Modularity and the Explicit Modularity sections, clarifying the concepts further and introducing new connections with the lottery ticket hypothesis and mechanistic interpretability works.

Assigned Action Editor: ~Razvan_Pascanu1

Submission Number: 4858