A two-parameter learnable Logmoid Activation Unit

xuemei Zhou, Lingfang Li, Xingzhou Zheng, Mingxing Luo

Keywords: Learnable activation function, image classification, object detection

TL;DR: A novel learnable Logmoid Activation Unit (LAU) is by parameterizing Logmoid with two hyper-parameters.

Abstract: A novel learnable Logmoid Activation Unit (LAU) is proposed as, $f(x)=x\ln(1+\alpha\textrm{sigmoid}(\beta x))$, by parameterizing Sigmoid with two hyper-parameters $\alpha$ and $\beta$ that are optimized by the back-propagation algorithm. The end-to-end deep neural networks with learnable LAUs can increase the predictive performances beyond well-known activation functions for different tasks.