ESRA: a Neuro-Symbolic Relation Transformer for Autonomous Driving

Alessandro Sebastian Russo, Lia Morra, Fabrizio Lamberti, Paolo Emmanuel Ilario Dimasi

Abstract: Scene Graph Generation (SGG) is a powerful tool for autonomous vehicles to understand their environment. In this paper, a novel one-stage neuro-symbolic architecture called nEuro-Symbolic Relation trAnsformer (ESRA) is proposed and its applications to SGG in the field of autonomous driving are investigated. This one-stage architecture can perform both object and relation recognition in a single step, attempting to incorporate prior knowledge in the form of logical propositions grounded by a Logic Tensor Network (LTN). To the best of our knowledge, this is the first attempt to combine a transformer-based architecture with an LTN for SGG. The results show that the integration of LTN increases mean recall (mR) by up to 21% in the best configuration, with mAP achieving an increase of up to 19%.