The Relationship Between the Distribution of Neural Network Weights and Model Accuracy Using Benford’s Law

TMLR Paper1676 Authors

Abstract:

Context: Benford’s Law describes the distribution of atypical patterns of numbers. It focuses on
the occurrence of the first digit in a natural population of numbers. When these numbers are divided
into nine categories based on their first digit, the largest category consists of numbers that start with
1, followed by those starting with 2, and so on. Objective: Each neuron in a Neural Network
(NN) holds a mathematical value, often referred to as a weight, which is updated according to
certain parameters. This study explores the Degree of Benford’s Law Existence (DBLE) within
Convolutional Neural Networks (CNNs). Additionally, the experiment investigates the correlation
between the DBLE and NN’s accuracy. Methods: A (CNN) is subjected to testing 15 times using
various datasets and hyperparameters. The DBLE is calculated for each CNN variation, and the
correlation between the CNN’s performance and DBLE is examined. To further explore the presence
of Benford’s Law in CNN models, nine transfer learning models are also tested for. Results: The
experiment suggests: 1) Benford’s Law is observed in the weights of neural networks, and in most
cases, the DBLE increases as the training progresses. 2) It is observed that models with significant
differences in performance tend to demonstrate relatively high divergence in DBLE.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=1PrOv3kOIk

Changes Since Last Submission:

My first submission was desk-rejected before it was sent for review due to the following reasons:

1- It was not anonymous.
2- Formatting Issues. White space
3- References.

All the above issues are now fixed the correct style for this journal is selected now. The article is anonymous.

Thank you very much for consideration and my sincere apologies on sending the incomplete submission.

Assigned Action Editor: ~Brian_Kulis1

Submission Number: 1676