Shielded Deep Reinforcement Learning for Multi-Sensor Spacecraft Imaging

Islam Nazmy, Andrew Harris, Morteza Lahijanian, Hanspeter Schaub

Abstract: This paper considers the problem of autonomous spacecraft control for imaging missions, subject to safety constraints. The controller chooses between discrete flight modes to image a target with different sensor types. The safety constraints include maintaining safe battery levels, reaction wheel speeds, and body rates. The proposed approach applies shielded deep reinforcement learning (SDRL) to autonomously command spacecraft flight modes, where the imaging requirements are communicated to the agent through a finite-state machine (FSM). The training is done in a target- and orbit-agnostic manner to create a single artificial neural network that can operate in a range of conditions. The FSM specifies which sensor type the agent should use for the next image. Simulation results based on a spacecraft tasked on Boulder, CO, USA demonstrate that this approach is effective for commanding a spacecraft safely while meeting predefined imaging requirements. This work also demonstrates how an agent trained on Boulder is capable of being applied to other Earth-targets as well as targets on the Moon with similar performance.