Unsupervised Representation Learning: Target Regularization for Cross-Domain Sentiment Classification

Michal Perelkiewicz, Rafal Poswiata, Jakub Kierzkowski

Abstract: This article proposes an autoencoder-based domain invariant feature representation learning approach to domain adaptation and the cross-domain text classification problem. Finding domain invariant feature representations is a transfer learning method for transmitting knowledge between source and target domain data. Our method aims to avoid the overfitting of an autoencoder model on source domain training data in a trained embedded feature space using a target regularization technique. We hypothesize that when forcing the semantic similarity of target domain representation to source domain representation by adding the source domain similarity penalty to reconstruction loss during autoencoder training, the penalty is greater when the the domain’s representations separability is. In this work, we contribute to domain adaptation by demonstrating that a regularization technique based on an auxiliary pretrained domain classification model can be used to build robust, shared domain feature representations. Our model achieves a classification accuracy improvement in standard cross-domain sentiment classification tasks over the baseline model in most cases.