Fractional Rectifier Activations for Off-Policy Reinforcement Learning: A Systematic Empirical Study

TMLR Paper8086 Authors

Abstract:

Activation functions play a key role in deep reinforcement learning by shaping how neural networks approximate policies and value functions. The Rectified Linear Unit (ReLU) remains the dominant choice due to its simplicity and computational efficiency. However, ReLU and its common variants rely on piecewise linear transformations, which may limit representational flexibility when modelling complex nonlinear dynamics in continuous-control tasks. In this work, we investigate fractional-order nonlinearities that extend rectifier-style activations while preserving their computational simplicity. We investigate three fractional variants: Fractional ReLU (FReLU), Fractional Leaky ReLU (FLReLU), and Fractional Parametric ReLU (FPReLU), which incorporate a fractional exponent that enables smooth and continuously adjustable nonlinear transformations.

We conduct a systematic empirical study of these activations in off-policy reinforcement learning by integrating them into two widely used algorithms, TD3 and SAC. Experiments are performed on continuous-control benchmarks from MuJoCo and the DeepMind Control Suite. Across tasks, architectures, and algorithms, fractional activations frequently outperform conventional rectifier functions, with the best fractional configuration yielding an average improvement of about 21\% in normalized Area Under the learning Curve (AUC) relative to the ReLU baseline. These results suggest that fractional rectifier activations provide a simple architectural modification that can improve function approximation and learning efficiency in deep reinforcement learning.

Submission Type: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=omjtYwnofM&noteId=omjtYwnofM

Changes Since Last Submission:

The previous submission was desk rejected with the comment: “An important contribution is the empirical study, but code is not provided.”

In the original submission, we did not include a code repository in order to strictly preserve anonymity during the review process. However, we acknowledge that the empirical nature of this work makes code availability essential for proper evaluation.

In this revised submission, we have addressed this concern by providing the complete implementation as anonymized supplementary material. This allows reviewers to fully assess the empirical results while maintaining anonymity.

In addition, we have clarified this in the manuscript by including the following statement:

“The implementation code will be made publicly available on GitHub after the anonymous review process.”

No other changes have been made to the technical content of the paper.

Assigned Action Editor: ~Youngchul_Sung1

Submission Number: 8086