Denoising cosine similarity: A theory-driven approach for efficient representation learning

Takumi Nakagawa, Yutaro Sanada, Hiroki Waida, Yuhui Zhang, Yuichiro Wada, Kosaku Takanashi, Tomonori Yamada, Takafumi Kanamori

Abstract: Highlights•A modified cosine-similarity loss with a denoising property is proposed.•The denoising property of the cosine similarity loss is theoretically investigated.•An estimator of the modified loss is introduced with statistical guarantees.•The quality enhancement of representations learned by the modified loss is observed.