Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problem

Hélène Plisnier, Denis Steckelmacher, Jeroen Willems, Bruno Depraetere, Ann Nowe

Abstract:

Many instances of similar or almost-identical industrial machines or tools are often deployed
at once, or in quick succession. For instance, a particular model of air compressor may be
installed at hundreds of customers. Because these tools perform distinct but highly similar
tasks, it is interesting to be able to quickly produce a high-quality controller for machine
N+1 given the controllers already produced for machines 1..N . This is even more important
when the controllers are learned through Reinforcement Learning, as training takes time,
energy and other resources. In this paper, we apply Policy Intersection, a Policy Shaping
method, to help a Reinforcement Learning agent learn to solve a new variant of a compressors
control problem faster, by transferring knowledge from several previously learned controllers.
We show that our approach outperforms loading an old controller, and significantly improves
performance in the long run.