The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones for Progress Towards More Efficient Model Inference

Zeyu Yang, Wesley Armour

Abstract: Deep learning has achieved significant success but poses increasing concerns about energy consumption and sustainability. Despite these concerns, there is a lack of understanding of their energy efficiency during inference. In this study, we conduct a comprehensive analysis of the inference energy consumption of 1,200 ImageNet classification models—the largest evaluation of its kind to date. Our findings reveal a steep decline in accuracy gains relative to the increase in energy usage, highlighting sustainability concerns in the pursuit of marginal improvements. We identify key factors contributing to energy consumption and demonstrate methods to improve energy efficiency. To promote more sustainable AI practices, we introduce an energy efficiency scoring system and develop an interactive web application that allows users to compare models based on accuracy and energy consumption. By providing extensive empirical data and practical tools, we aim to facilitate informed decision-making and encourage collaborative efforts in the development of energy-efficient AI technologies.

Lay Summary:

Deep learning powers many of the smart technologies we use today—from voice assistants to image recognition—but running these models can use a lot of energy. As artificial intelligence (AI) becomes more widely used, concerns are growing about its environmental impact, especially during inference (the phase when trained models are actually used to make predictions).

In our research, we studied the energy use of 1,200 image classification models, making this the largest study of its kind so far. We found that while newer models often achieve slightly better accuracy, they tend to consume much more energy, raising questions about whether these small improvements are worth the environmental cost.

We also identified what makes some models more energy-hungry than others and showed how certain design choices can improve energy efficiency. To help the AI community make more sustainable choices, we created an energy efficiency score and built a web tool that lets users compare models based on both their accuracy and energy use.

Our goal is to provide practical tools and clear data to help researchers and developers build AI systems that are not only powerful but also environmentally responsible.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/JimZeyuYang/DL-Inference-Energy-Efficiency

Primary Area: General Machine Learning->Evaluation

Keywords: Energy Efficiency, Green AI, Efficient and Scalable Vision, Sustainability

Submission Number: 12604