Norm-count Hypothesis: On the Relationship Between Norm and Object Count in Visual Representations
Abstract: We present a novel hypothesis on norms of representations produced by convolutional neural networks (CNNs). In particular, we propose the norm-count hypothesis (NCH), which states that there is a monotonically increasing relationship between the number of certain objects in the image, and the norm of the corresponding representation. We formalize and prove our hypothesis in a controlled setting, showing that the NCH is true for linear and batch normalized CNNs followed by global average pooling, when they are applied to a certain class of images. Further, we present experimental evidence that corroborates our hypothesis for CNN-based representations. Our experiments are conducted with several real-world image datasets, in both supervised and self-supervised learning -- providing new insight on the relationship between object counts and representation norms.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=0hJGrRuhEA
Changes Since Last Submission: **2024.05.10:**
1. Added additional experiments on the Multi Salient Object (MSO) dataset. This dataset is derived from the Salient Object Subitizing (SOS) dataset, which was suggested by the AE for additional experiments.
1. Added norm vs. size experiments on a real-world dataset (COCO), as suggested by the AE and one of the reviewers.
1. Changed the experimental setup such that the experimental setting is more representative of the theoretical analysis.
1. Removed experiments on classification accuracy before and after L2-normalization, as the line of reasoning behind these experiments was not sufficiently rigorous as pointed out by the AE and reviewer. Instead we have strengthened the experiment part that can directly be attributed to the proposed NCH through the addition of additional datasets and the requested norm vs. size evaluations.
**2024.07.08:**
1. Updated Definition of relaxed detector to include addition of constant in distributive property (Eq. (5)). Results and proofs affected by this change have also been updated.
1. Fixed error in Proposition 4 (LeakyReLU is a relaxed detector).
1. Added a note on padding following proposition about convolution being a strict detector.
1. Added extra motivation and explanation of the definitions of strict and relaxed detectors.
1. Fixed several typos and notational inconsistencies identified by the reviewers.
**2024.07.09**
1. Added MNIST experiment where digits are allowed to overlap (Appendix B).
**2024.07.16**
1. Changed final paragraph in conclusion.
1. Added brief discussion on object images vs. labels for SSL models.
1. Highlighted changes made as response to Reviewer bML5.
**2024.07.18**
1. Added stepwise evaluation results in appendix.
Assigned Action Editor: ~Neil_Houlsby1
Submission Number: 2664
Loading