HIRL-GAN：Hierarchical Mask-Guided Inpainting via GAN and Reinforcement Learning for Urban Occlusion Removal

Peng Sun, Hui Li, Shuangchen Zhao, Haili Sun, Zhenxin Zhang, Yuhang Dong, Bojun Li, Jiatong Zhang

Keywords: Image Inpainting；Generative Adversarial Networks；Reinforcement Learning；Deep Learning

Abstract: In urban 3D reconstruction tasks, occlusions in architectural images often result in missing or distorted structures during point cloud reconstruction from images, thereby significantly affecting the accuracy of the final reconstruction. To address this issue, we propose HIRL-GAN, a **HI**erarchical and progressive inpainting framework that synergizes **R**einforcement **L**earning with **GAN**s, specifically designed for structured reconstruction of occluded building images. The proposed framework incorporates three key components: a hierarchical mask decomposition strategy that partitions complex occlusions into smaller sub-regions and restores them progressively to enhance structural stability; a reinforcement learning-based policy optimization mechanism that dynamically guides the reconstruction process at the sub-region level to improve restoration quality; and a self-attention-enhanced generator network that jointly models global semantics and local textures. In addition, we introduce a soft-mask guided training scheme to ensure smooth transitions and natural texture blending between restored and original regions. Extensive experiments on multiple image inpainting benchmarks demonstrate that HIRL-GAN achieves superior performance in structural recovery, detail reconstruction, and perceptual quality compared to existing state-of-the-art methods. Furthermore, ablation studies validate the effectiveness and robustness of the proposed RL-driven progressive mask restoration strategy under complex occlusion scenarios.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 11303