Constructing the Basis Path Set by Eliminating the Path Dependency

Juanping Zhu, Qi Meng, Wei Chen, Yue Wang, Zhiming Ma

Abstract: The newly appeared $${\cal G}$$ G -SGD algorithm can only heuristically find the basis path set in a simple neural network, so its generalization to a more practical network is hindered. From the perspective of graph theory, the BasisPathSetSearching problem is formulated to find the basis path set in a complicated fully connected neural network. This paper proposes algorithm DEAH to hierarchically solve the BasisPathSetSearching problem by eliminating the path dependencies. For this purpose, the authors discover the underlying cause of the path dependency between two independent substructures. The path subdivision chain is proposed to effectively eliminate the path dependency, both inside the chain and between chains. The theoretical proofs and the analysis of time complexity are presented for Algorithm DEAH. This paper therefore provides one methodology to find the basis path set in a general and practical neural network.