Deviation-based multiple coefficient item mixer for heterogeneous set-to-set matching

Hirotaka Hachiya, Kajishiro Yukito

Abstract: Heterogeneous set-to-set matching tasks such as fashion outfit recommendation, require permutation-invariant and dynamic item-wise transformations to bring compatible sets closer while pushing incompatible ones apart. While attention-based methods satisfy the permutation invariance requirement, they often suffer from convex hull limitations due to their reliance on softmax-based dot-product operations. On the other hand, MLP-based methods like DuMLP-Pin avoid such constraints but tend to lose critical item-wise structure through global aggregation. To address these limitations, we propose DeviMix (Deviation-based multiple coefficient item Mixer), a novel MLP-based architecture that performs item-wise dynamic transformations. Our approach generates multiple item-mixing coefficients by applying MLPs to cross-deviation vectors computed from all possible item pairs in sets. Extensive experiments on fashion outfit and furniture coordination matching tasks demonstrate that DeviMix consistently outperforms attention-based and global pooling-based baselines, validating the effectiveness of our MLP-based item-wise aggregation using cross-deviation for heterogeneous set matching.

Submission Number: 47