CNNGEN: A GENERATOR AND BENCHMARK FOR SUSTAINABLE CONVOLUTIONAL NEURAL NETWORK SEARCH
Primary Area: datasets and benchmarks
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: CNN, NAS, Neural Architecture Search, convolutional neural network, energy consumption, sustainability
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Neural Architecture Search (NAS) emerged as a promising approach to search for
optimal neural network architectures in a limited, predefined architecture space.
One popular method to form such a space is to derive a known architecture in
which we insert cells where NAS algorithms can automatically combine network
functions and connections. Cell-based methods yielded hundreds of thousands
of trained architectures whose specifications and performance are available to de-
sign performance prediction models. Cell-based approaches come with three main
limitations: i) generated networks have limited diversity resulting in very sim-
ilar performances, in turn hampering the generalization of trained performance
models, ii) networks’ implementations are missing hampering performance un-
derstanding, and iii) they solely focus on performance metrics (e.g., accuracy)
ignoring the growing sustainability concern. We propose CNNGen, an approach
that addresses: i) by leveraging a domain-specific language (DSL) to automat-
ically generate convolutional neural networks (CNNs) without predefined cells
or base skeleton. It allows the exploration of diverse and potentially unknown
topologies; ii) CNNGen’s comprehensive pipeline stores the network description
(textual and image representation) and the fully executable generated Python code
(integrated with popular deep-learning frameworks) for analysis or retraining, and
iii) in addition to training and performance metrics, CNNGen also computes en-
ergy consumption and carbon impact for green machine learning endeavors. We
demonstrate the possibilities of CNNGen by designing two performance predic-
tors and comparing them to the state of the art.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3267
Loading