Autonomous Particles

Nikola Andrejić, Vitaly Vanchurin

Abstract:

Consider a reinforcement learning problem where an agent has access
to a very large amount of information about the environment, but it can only take
very few actions to accomplish its task and to maximize its reward. Evidently, the
main problem for the agent is to learn a map from a very high-dimensional space
(which represents its environment) to a very low-dimensional space (which represents
its actions). The high-to-low dimensional map implies that most of the information
about the environment is irrelevant for the actions to be taken, and only a small fraction
of information is relevant. In this paper we argue that the relevant information need not
be learned by brute force (which is the standard approach), but can be identified from
the intrinsic symmetries of the system. We analyze in details a reinforcement learning
problem of autonomous driving, where the corresponding symmetry is the Galilean
symmetry, and argue that the learning task can be accomplished with very few relevant
parameters, or, more precisely, invariants. For a numerical demonstration, we show
that the autonomous vehicles (which we call autonomous particles since they describe
very primitive vehicles) need only four relevant invariants to learn how to drive very
well without colliding with other particles. The simple model can be easily generalized
to include different types of particles (e.g. for cars, for pedestrians, for buildings,
for road signs, etc.) with different types of relevant invariants describing interactions
between them. We also argue that there must exist a field theory description of the
learning system where autonomous particles would be described by fermionic degrees
of freedom and interactions mediated by the relevant invariants would be described
by bosonic degrees of freedom. This suggests that the effectiveness of field theory
descriptions of physical systems might be connected to the learning dynamics of some
kinds of autonomous particles, supporting the claim that the entire universe is a neural
network.