Multi-task Learning with Future States for Vision-Based Autonomous Driving

Inhan Kim, Hyemin Lee, Joonyeong Lee, Eunseop Lee, Daijin Kim

Abstract: Human drivers consider past and future driving environments to maintain stable control of a vehicle. To adopt a human driver’s behavior, we propose a vision-based autonomous driving model, called Future Actions and States Network (FASNet), which uses predicted future actions and generated future states in multi-task learning manner. Future states are generated using an enhanced deep predictive-coding network and motion equations defined by the kinematic vehicle model. The final control values are determined by the weighted average of the predicted actions for a stable decision. With these methods, the proposed FASNet has a high generalization ability in unseen environments. To validate the proposed FASNet, we conducted several experiments, including ablation studies in realistic three-dimensional simulations. FASNet achieves a higher Success Rate (SR) on the recent CARLA benchmarks under several conditions as compared to state-of-the-art models.