A Sequence Prediction Perspective for Intuitive Physics Reasoning and Interaction

Du Jiang

Keywords: intuitive physics, sequence prediction

Abstract:

In this essay, we discuss a sequence prediction-based approach to empower deep learning systems with intuitive physics reasoning capabilities. Our approach involves pre-training a physics-informed model to predict future states based on real-world observations. This model is then fine-tuned using reinforcement learning (RL) techniques, aligning it with RL’s success in training large language models.
Thus it is possible for our systems to better understand complex, dynamic environments. This approach holds potential across diverse domains, including robotics, simulations, and autonomous systems.

Submission Number: 55