Reducing Distributional Uncertainty by Mutual Information Maximisation and Transferable Feature Learning

Guosheng Hu

Abstract: Distributional uncertainty exists broadly in many real-worldapplications, one of which in the form of domain discrepancy. Yet in theexisting literature, the mathematical definition of it is missing. In thispaper, we propose to formulate the distributional uncertainty both be-tween the source(s) and target domain(s) and within each domain usingmutual information. Further, to reduce distributional uncertainty (e.g.domain discrepancy), we (1) maximise the mutual information betweensource and target domains and (2) propose a transferable feature learningscheme, balancing two complementary and discriminative feature learn-ing processes (general texture learning and self-supervised transferableshape learning) according to the uncertainty. We conduct extensive ex-periments on both domain adaption and domain generalisation usingchallenging common benchmarks: Office-Home and DomainNet. Resultsshow the great effectiveness of the proposed method and its superiorityover the state-of-the-art methods.