Toward an Architecture for Robots in the Era of Foundation Models

Mohan Sridharan

Keywords: Knowledge representation, interactive learning, ecological rationality, human-robot collaboration

TL;DR: Essay on key principles to embed in an architecture for reasoning, control, collaboration, and learning in robotics

Abstract: Robots and AI systems are increasingly being used to collaborate with humans in different application domains. This development has largely been due to the impressive performance of deep networks and foundation models that are considered to be state of the art for many problems in robotics and AI. However, these methods and models are resource-hungry and opaque, and they are known to provide arbitrary decisions in previously unknown situations, whereas practical robot application domains require transparent, multi-step, multi-level decision-making and ad hoc collaboration under resource constraints and open world uncertainty. In this paper, I argue that for widespread use of robots in assistive roles, we need to revisit principles that can be traced back to the early pioneers of AI who had a deep understanding of cognition and control in humans. We also need to embed these principles in the architectures we develop for robots, using modern data-driven methods as one of many tools that build on this foundation. I then briefly summarize the benefits of this approach in the context of core problems in robotics such as visual scene understanding, planning, changing-contact manipulation, and multiagent/human-agent collaboration.

Paper Track: Commentary

Submission Number: 72