Fourier Features in Reinforcement Learning with Neural Networks
Abstract: In classic Reinforcement Learning (RL), the performance of algorithms depends critically on data representation, i.e., the way the states of the system are represented as features. Choosing appropriate features for a task is an important way of adding prior domain knowledge since cleverly distributing information into states facilitates appropriate generalization. For linear function approximations, the representation is usually hand-designed according to the task at hand and projected into a higher-dimensional space to facilitate linear separation. Among the feature encodings used in RL for linear function approximation, we can mention in a non-exhaustive way Polynomial Features or Tile Coding. However, the main bottleneck of such feature encodings is that they do not scale to high-dimensional inputs as they grow exponentially in size with the input dimension.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Minor typo change.
Code: https://github.com/DavidBrellmann/Fourier_Features_in_RL_with_NN
Assigned Action Editor: ~Caglar_Gulcehre1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 918
Loading