Restoring Images in Adverse Weather Conditions via Histogram Transformer

Shangquan Sun, Wenqi Ren, Xinwei Gao, Rui Wang, Xiaochun Cao

Abstract: Transformer-basedimagerestorationmethodsinadversewea- ther have achieved significant progress. Most of them use self-attention along the channel dimension or within spatially fixed-range blocks to reduce computational load. However, such a compromise results in lim- itations in capturing long-range spatial features. Inspired by the ob- servation that the weather-induced degradation factors mainly cause similar occlusion and brightness, in this work, we propose an efficient Histogram Transformer (Histoformer) for restoring images affected by adverse weather. It is powered by a mechanism dubbed histogram self- attention, which sorts and segments spatial features into intensity-based bins. Self-attention is then applied across bins or within each bin to selectively focus on spatial features of dynamic range and process simi- lar degraded pixels of the long range together. To boost histogram self- attention, we present a dynamic-range convolution enabling conventional convolution to conduct operation over similar pixels rather than neighbor pixels. We also observe that the common pixel-wise losses neglect linear association and correlation between output and ground-truth. Thus, we propose to leverage the Pearson correlation coefficient as a loss function to enforce the recovered pixels following the identical order as ground- truth. Extensive experiments demonstrate the efficacy and superiority of our proposed method.