Domain Shared and Specific Prompt Learning for Incremental Monocular Depth Estimation

Zhiwen Yang, Liang Li, Jiehua Zhang, Tingyu Wang, Yaoqi Sun, Chenggang Yan

Abstract:

Incremental monocular depth estimation aims to continuously learn from new domains while maintaining their performance on old
domains. The catastrophic forgetting problem is the key challenge when the model adapts the dynamic scene variations. Previous
methods usually address this forgetting problem by storing raw samples from the old domain, allowing the model to review the
knowledge of the old domain. However, due to the concerns of data privacy and security, our objective is to tackle the incremental
monocular depth estimation problem in more stringent scenarios without the need for replaying samples. In this paper, we attribute
the cross-domain catastrophic forgetting to the domain distribution shifts and continuous variations of depth space. To this end,
we propose Domain Shared and Specific Prompt Learning (DSSP) for incremental monocular depth estimation. In detail, to alleviate
the domain distribution shift, complementary domain prompt are designed to learn the domain-shared and domain-specific knowledge which are optimized by the inter-domain alignment and intra-domain orthogonal loss. To mitigate the depth space variations, we first introduce a pre-trained model to generate the domain-shared depth space. Then, we design $S^2$-Adapter that quantizes
depth space variations with scale&shift matrices and converts the domain-shared depth space to domain-specific depth space. Our
method achieves state-of-the-art performance under various scenarios such as different depth ranges, virtual and real, different
weather conditions, and the few-shot incremental learning setting
on 12 datasets. We will release the source codes and pre-trained
models.

Primary Subject Area: [Content] Media Interpretation

Relevance To Conference: Our method aims to tackle the incremental monocular depth estimation problem in more stringent scenarios without the need for replaying samples due to the data privacy and security.

Submission Number: 2804