Sensory Resilience based on Synesthesia

Eric Platon, Tom Sonoda

Keywords: perception, resilience, robotics, synesthesia

Abstract: Situated cognition depends on accessing environmental state through sensors. Engineering and cost constraints usually lead to limited “pathways” where, for example, a vision sub-system only includes a camera and the software to deal with it. This traditional and rational design style entails any hardware defect on the pathway causes the system to grind to a halt until repair. We propose a “sensoriplexer” as drop-in neural component architecture to address this issue, under the common scenario of multiple sensors availability. This component architecture learns to mix and relate pathways, such that an agent facing failure in a sensory sub-system can degrade gracefully and coherently by relying on its other sub- systems. The architecture is inspired by the concept of synesthesia, and relies on statistical coupling between sensor signals. We show the benefit and limitation of the architecture on a simple shape recognition and a more complex emotion recognition scenarios.

One-sentence Summary: An artificial neural component inspired by synesthesia for agents to deal with sensory failures.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=b8Uj56NkRf