Exponential Quantum Advantage in Communication for Distributed Learning

Dar Gilboa, Jarrod Ryan McClean

Keywords: Distributed learning, communication complexity, quantum computing

TL;DR: We show that distributed, parameterized models can be trained using gradient descent with exponentially less communication if realized using quantum computers that can exchange quantum states, with additional privacy benefits.

Abstract:

Training and inference with large machine learning models that far exceed the memory capacity of
individual devices necessitates the design of distributed architectures, forcing one to contend with
communication constraints. We present a framework for distributed computation over a quantum
network in which data is encoded into specialized quantum states. We prove that for certain models
within this framework, inference and training using gradient descent can be performed with exponentially less communication compared to their classical analogs, and with relatively modest time
and space complexity overheads relative to standard gradient-based methods. To our knowledge,
this is the first example of exponential quantum advantage for a generic class of machine learning
problems with dense classical data that holds regardless of the data encoding cost. Moreover, we
show that models in this class can encode highly nonlinear features of their inputs, and their expressivity increases exponentially with model depth. We also find that, interestingly, the communication
advantage nearly vanishes for simpler linear classifiers. These results can be combined with natural
privacy advantages in the communicated quantum states that limit the amount of information that
can be extracted from them about the data and model parameters. Taken as a whole, these findings
form a promising foundation for distributed machine learning over quantum networks.

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1142