Under the Shadow: Exploiting Opacity Variation for Fine-grained Shadow Detection

Xiaotian Qiao, Ke Xu, Xianglong Yang, Ruijie Dong, Xiaofang Xia, Jiangtao Cui

Keywords: Shadow Detection, Shadow Removal, Scene Understanding

TL;DR: We make the first attempt to investigate fine-grained shadow detection by exploiting opacity variations, enhancing downstream applications like shadow detection, removal, and editing.

Abstract:

Shadow characteristics are of great importance for scene understanding.
Existing works mainly consider shadow regions as binary masks, often leading to imprecise detection results and suboptimal performance for scene understanding.
We demonstrate that such an assumption oversimplifies light-object interactions in the scene, as the scene details under either hard or soft shadows remain visible to a certain degree.
Based on this insight, we aim to reformulate the shadow detection paradigm from the opacity perspective, and introduce a new fine-grained shadow detection method.
In particular, given an input image, we first propose a shadow opacity augmentation module to generate realistic images with varied shadow opacities.
We then introduce a shadow feature separation module to learn the shadow position and opacity representations separately, followed by an opacity mask prediction module that fuses these representations and predicts fine-grained shadow detection results.
In addition, we construct a new dataset with opacity-annotated shadow masks across varied scenarios.
Extensive experiments demonstrate that our method outperforms the baselines qualitatively and quantitatively, enhancing a wide range of applications, including shadow removal, shadow editing, and 3D reconstruction.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Flagged For Ethics Review: true

Submission Number: 11711