Alleviating Dimensional Collapse Problem in Deep Recommender Models by Designing Uniformity Layers

Yabin Zhang, Jiakai Tang, Xu Chen

Abstract: Deep recommender models have shown remarkable successes in real-world applications. Despite effectiveness, it has been observed that as the number of model layers goes larger, the learned embeddings may become more and more similar (called the dimensional collapse problem), which may fail to discriminate different users/items, and severely impact the recommendation performance. To solve this problem, in this paper, we design a general uniformity-driven layer to equip existing deep recommender models. In specific, we first design an objective to encourage diversities between different sample embeddings. Then, we convert this objective into a neural layer, which is identical to propagating one-step gradient information. Comparing with the final objective, the converted layer can be seamlessly incorporated into the model architecture, which can influence the model parameters in a more direct and effective manner. To further enhance the training efficiency, we introduce a non-linear projection strategy and a re-weighting mechanism to reduce the computational cost. Beyond the above design, we also theoretically discuss our method on alleviating the dimensional collapse problem by analyzing its effective rank. In the experiments, we apply our method to different base models, and demonstrate its effectiveness based on four real-world datasets.

External IDs:dblp:conf/dasfaa/ZhangTC24