ES-Parkour: Advanced Robot Parkour with Bio-Inspired Event Camera and Spiking Neural Network
Abstract: In recent years, significant progress has been made in the field of quadruped robotics, particularly in perception and motion control algorithms powered by reinforcement learning. It has shown that these robots are capable of executing complex motions in challenging environments with the help of visual sensors like depth cameras, demonstrating outstanding stability and robustness. However, these sensors typically operate at lower frequencies compared to the control frequency of the robot's joints and are susceptible to lighting conditions, making them challenging to deploy outdoors. Moreover, the computational load on the robot's side is increased by the deep neural networks used in multiple sensor systems and control units. To address these challenges, we, \textit{for the first time}, introduce the spiking neural networks (SNNs) and event cameras to accomplish a complex quadruped robot parkour task. This combination leverages the efficiency of SNN in processing spike sequences and the capability of event cameras to capture dynamic visual information, thus showing great potential in emulating biological perception and processing mechanisms. Our experimental results indicate that employing event cameras and SNN yields excellent performance in challenging parkour tasks. Compared to traditional deep neural networks, our ES-Parkour presents significantly lower energy consumption, amounting to merely 11.7% of that exhibited by the ANN model. This corresponds to an extreme energy-saving 88.3% by utilizing SNN. By integrating the strengths of event cameras and SNN, our work expands the possibilities for the further development of robotic reinforcement learning algorithms and explores their future applications in various challenging environments.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Dmitry_Kangin1
Submission Number: 3282
Loading