Deep Neural Network Model Compression and Signal Processing

Arijit Ukil, Angshul Majumdar, Antonio J. Jara, João Gama

Abstract: Deep neural networks (DNN) are used to analyze images, videos, signals and texts require a lot of memory and intensive computing power. For example, the very successful GPT4 model contains more than a few trillion parameters. Although such models are of great impact, but they have been used very little in real-world applications, including industrial Internet of Things, self-driving cars, algorithmic health monitoring for use in limited mobile or edge devices. The requirement to run large models on resource-constrained peripherals has led to significant research interest in compressing DNN models. Signal processing researchers have traditionally advocated data (image/video/audio) compression, and by the way, many of these techniques are used for DNN compression. For example, source coding is a basic technique that has been widely used to compress various DNN models. In this paper, we present our views on the use of signal processing methods for DNN model compression.

External IDs:dblp:conf/icassp/UkilMJG24