A Deep Architecture for Log-Linear ModelsDownload PDF

Published: 07 Nov 2020, Last Modified: 05 May 2023NeurIPSW 2020: DL-IG PosterReaders: Everyone
Keywords: deep learning, log-linear model, partial order structure, energy based model, EM algorithm
TL;DR: A deep learning architectures using a log-linear model and partial order structure that does not require any gradients to update.
Abstract: We present a novel perspective on deep learning architectures using a partial order structure, which is naturally incorporated into the information geometric formulation of the log-linear model. Our formulation provides a different perspective of deep learning by realizing the bias and weights as different layers on our partial order structure. This formulation of the neural network does not require any gradients and can efficiently estimate the parameters using the EM algorithm.
3 Replies