Formal analysis on the DNN-kWTA model with non-ideal transfer function and noisy integrator

Wenhao Lu, Yuanjin Zheng, Chi-Sing Leung

Abstract: The dual neural network-based (DNN)-kWTA model, based on a DNN architecture, serves as a continuous-time neural approach for identifying the k largest values from a set of n input values. This model possesses two notable characteristics. Firstly, it exhibits significantly fewer interconnections compared to other neural models. Secondly, it demonstrates the capability to accurately detect the k largest values within a finite time. However, the latter quality hinges on the assumption of perfect realization, which is often unattainable in practice. Imperfections such as the inability to achieve an ideal step function for neuron transfer functions and the presence of time-varying multiplicative noise, stemming from random fluctuations at the recurrent neuron’s output, can impact its accuracy, i.e., the ability of correctly finding out the k largest values. This paper analyzes the behaviors of the DNN-kWTA model under the aforementioned imperfections, including multiplicative noise and a non-ideal transfer function. We first derive the neural dynamics for the DNN-kWTA model under the mentioned imperfections. When the noise level is excessively high or the transfer function gain is too low, the neural dynamics fail to converge. Consequently, we theoretically demonstrate that under specific conditions, the neural dynamics converge to an equilibrium point. Leveraging the characteristics of the dynamics, we develop a practical method to assess the model’s ability to accurately identify the k largest numbers. This method enables us to effectively estimate the likelihood of the model correctly identifying these numbers. Additionally, when the input numbers adhere to a uniform distribution, we establish a lower bound expression for the probability of the model correctly identifying the k largest numbers. It is worth noting that, according to basic probability theory, any probability density function can be transformed into a uniform distribution without affecting the ordering of random variables. Simulations are conducted to validate our theoretical analysis.