Simulating City-Scale Aerial Data Collection Using Unreal Engine

Dhruv Agarwal, Taci Kucukpinar, Joshua Fraser, Jeffrey Kerley, Andrew R. Buck, Derek T. Anderson, Kannappan Palaniappan

Abstract: The rendering quality of game engines has significantly improved in realism. Physically-based rendering techniques started to overtake traditional rasterization-based graphics rendering pipelines in interactive rendering applications. We leverage these advancements to replicate our real-world city-scale aerial image data collections in the Unreal Engine. The use of city-scale synthetic data has many benefits in computer vision and remote sensing research. We can generate ground truth data for many applications, such as feature extraction, bundle adjustment, 3D reconstruction, object tracking, shadow detection, and shadow removal. This is crucial since sufficient ground truth data is not always publicly available for these use cases. In this paper, we introduce an easy-to-use Unreal Engine Blueprint pipeline that replicates drone and plane flight data collections in a circular orbit. Then, we highlight the issues arising from the approximate algorithms used by game engines as proxies for real-world data, and offer solutions to these challenges. Finally, we present our use cases, showing how we use the synthetic data generated by this approach in various research applications.