Memory-Modular Classification: Learning to Generalize with Memory Replacement
Keywords: few-shot classification, zero-shot classification, memory-based classification, image classification, vision and language
Abstract: We introduce a memory-modular learner for image classification that externalizes
knowledge memorization from reasoning and thus effectively generalizes to new
classes by replacing memory contents. Instead of statically compiling the world
knowledge and task skills into model weights during training, the proposed model
stores the knowledge in an external memory of image/text data and learns to dy-
namically select relevant contents from the memory according to an input image.
The meta-learned model performs robust classification with a memory of noisy
web-crawled data and adapts to new classes without re-training when the memory
is replaced. Experimental results show the promising performance of our method
on diverse scenarios, including zero-shot/few-shot classification of unseen classes,
fine-grained classification, and class-incremental classification.
Primary Area: transfer learning, meta learning, and lifelong learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4930
Loading