Nearest Unitary and Toeplitz matrix techniques for adaptation of Deep Learning models in photonic FPGA

Georgios Agrafiotis, Eftychia Makri, Ilias Kalamaras, Antonios Lalas, Konstantinos Votis, Dimitrios Tzovaras

Abstract: Photonic circuits pave the way to extremely quick computation and real-time inference in critical applications, such as imaging flow cytometry (IFC). Nevertheless, current photonic FPGA implementations display intrinsic limitations that restrict the complexity of Deep Learning (DL) models that could be sustained. One of these restrictions implies the weight matrices to be unitary. Thus, machine learning mechanisms to transform weight matrices to their nearest unitary one, are essential for the effective deployment of such demanding tasks. Furthermore, DL models that perform convolutions, require special handling so as to fit in the photonic system. In this work, several methods have been investigated for conversion of non-unitary matrices to unitary ones, as well as, linear algebra techniques for the transformation of Convolutional Neural Networks (CNNs) to Feed-Forward models, under the prism of discovery of the best candidate for the photonic FPGA in terms of accuracy and restrictions. Experimental results proved that post-training or iterative techniques to find the nearest unitary weight matrix can be applied for photonic chips with the minimum loss in accuracy, while CNNs adapted well in a photonic configuration employing a Toeplitz matrix implementation. The proposed approach envisions efficient tackling of DL models limitations for deployment in photonic FPGAs.

External IDs:dblp:conf/nldl/AgrafiotisMKLVT23