FORT-RAJ: a fisheye-optimized deep learning model for real-time trajectory prediction

Sarra Bouzayane, Mouad Kahouadji, Baptiste Magnier

Abstract: Fisheye cameras, renowned for their ultra-wide \(180^\circ \) panoramic field of view, have become essential tools in applications requiring extensive area monitoring. However, their unique optical properties introduce significant image distortions, rendering traditional trajectory prediction algorithms ineffective when applied to fisheye imagery. To address this challenge, we introduce Fisheye-Optimized Real-Time Trajectory prediction (FORT-RAJ), a novel hybrid artificial intelligence approach capable of both adapting to fisheye distortions and accurately predicting pedestrian trajectories in real time. From a technical perspective, FORT-RAJ combines the strengths of two state-of-the-art models: Fisheye Online Realtime Tracking (FORT), a distortion-aware model designed for fisheye cameras but limited to pedestrian tracking, and Graph- and Attention-based multi-agent Trajectory prediction model (GATraj), a powerful prediction framework not originally compatible with fisheye distortions. By integrating the fisheye adaptation capabilities of FORT with the predictive power of GATraj, FORT-RAJ overcomes their individual limitations and provides an end-to-end solution that is both distortion-aware and capable of accurate future movement prediction. The effectiveness of the proposed approach was rigorously evaluated on two distinct datasets, namely HABBOF and Caplogy. Experimental results revealed the superiority of FORT-RAJ, achieving an Average Displacement Error of 0.39 ms and 0.38 ms, and a Final Displacement Error of 0.43 ms and 0.42 ms for HABBOF and Caplogy, respectively. These results underscore the model’s high precision in predicting pedestrian trajectories, even under challenging conditions.

External IDs:dblp:journals/paa/BouzayaneKM25