Multi-Objective Evolutionary Neural Architecture Search for Liquid State Machine

Sida Xin, Renzhi Chen, Xun Xiao, Yuan Li, Lei Wang

Abstract: Liquid State Machine (LSM) is a brain-inspired computational model that has proven highly effective in various applications, owing to its intrinsic capability to process spatiotemporal information and its minimal training complexity. However, the performance of LSMs significantly depends on the design of their network architecture, which is overly reliant on existing human experience. Furthermore, as the network scale increases, the computing resources required for deployment and operation also increase, so we regarded the network design as a multi-objective problem. To address these challenges, we introduced an effective surrogate-assisted multi-objective evolutionary neural architecture search algorithm that balanced the accuracy and network scale. Our approach utilized parameter sensitivity analysis followed by the upper confidence bound algorithm to reduce the search space. Experimental results demonstrate that we successfully reduced the dimensions of the search space by 11% and the size of the entire search space by 75%. Compared to the state-of-the-art, our approach offered better trade-off solutions, such as a solution that reduced network scale by 32.5% while maintaining the same accuracy, and another that improved accuracy by 1.4% without changing the network scale. Furthermore, the knee point reduced network scale by 25 % and simultaneously increased accuracy by 0.7%. The source code can be accessed at https://github.com/XinSida/MOENAS-PSA.