Irrelevant Patch-Masked Autoencoders for Enhancing Vision Transformers under Limited Data

Qiang Ren, Junli Wang

Abstract: Highlights•We propose IP-MAE, enhancing ViT-based models under data-limited scenarios.•A novel alignment method masks irrelevant patches based on feature discrepancies.•A variance-weighting strategy mitigates the impact of irrelevant features.•Extensive experiments show IP-MAE significantly improves performance on small datasets.