Learning Underlying Physical Properties From Observations For Trajectory Prediction

Ekaterina Nikonova, Jochen Renz

Keywords: Physical Games, Deep Learning, Physical Reasoning, Transfer of Knowledge

TL;DR: We present a model that learns physical properties from observations and uses them for trajectory prediction in physical games.

Abstract:

In this work we present an approach that combines deep learning together with
laws of Newton’s physics for accurate trajectory predictions in physical games.
Our model learns to estimate physical properties and forces that generated given
observations, learns the relationships between available player’s actions and estimated
physical properties and uses these extracted forces for predictions. We
show the advantages of using physical laws together with deep learning by evaluating
it against two baseline models that automatically discover features from
the data without such a knowledge. We evaluate our model abilities to extract
physical properties and to generalize to unseen trajectories in two games with a
shooting mechanism. We also evaluate our model capabilities to transfer learned
knowledge from a 2D game for predictions in a 3D game with a similar physics.
We show that by using physical laws together with deep learning we achieve a better
human-interpretability of learned physical properties, transfer of knowledge to
a game with similar physics and very accurate predictions for previously unseen
data.