Optimization Design of Adaptive Loss Function Using Evolutionary Neural Networks

Xiang Meng, Zhaoyang Hai, Xiabi Liu, Yan Pei

Abstract: This study introduces the evolutionary loss function (ELF), a novel framework that dynamically optimizes loss functions using evolutionary computation. Unlike traditional loss functions based on fixed, predefined formulas, ELF employs a parameterized neural network capable of adapting to diverse data distributions and task-specific requirements. By leveraging operations of evolutionary computation such as mutation and selection, ELF explores a broad parameter space, effectively addressing the inherent limitations of gradient-based optimization methods. These methods, which require differentiable objectives, often struggle with non-smooth functions and are prone to local optima, limiting their effectiveness in complex or irregular optimization landscapes. In contrast, ELF utilizes evolutionary optimization to perform a global search across the parameter space, enabling it to overcome these challenges and dynamically optimize loss functions. To validate its effectiveness, ELF is evaluated across multiple tasks, with experimental results consistently demonstrating superior performance compared to both traditional and state-of-the-art dynamic loss functions.

External IDs:dblp:conf/iconip/MengHLP25