Plug-and-Play Compositionality for Boosting Continual Learning with Foundation Models

ICLR 2026 Conference Submission12857 Authors

Keywords: Continual learning

TL;DR: We introduce CompSLOT, a universal concept learning method to continual learning with foundation models system to establish a concept-level understanding of class prediction for alternative continual learners.

Abstract:

Vision learners often struggle with catastrophic forgetting due to their reliance on class recognition by comparison, rather than understanding classes as compositions of representative concepts.
This limitation is prevalent even in state-of-the-art continual learners with foundation models and worsens when current tasks contain few classes.
Inspired by the recent success of concept-level understanding in mitigating forgetting, we design a universal framework CompSLOT to guide concept learning across diverse continual learners.
Leveraging the progress of object-centric learning in parsing semantically meaningful slots from images, we tackle the challenge of learning slot extraction from ImageNet-pretrained vision transformers by analyzing meaningful concept properties.
We further introduce a primitive selection and aggregation mechanism to harness concept-level image understanding.
Additionally, we propose a method-agnostic self-supervision approach to distill sample-wise concept-based similarity information into the classifier, reducing reliance on incorrect or partial concepts for classification.
Experiments show CompSLOT significantly enhances various continual learners and provides a universal concept-level module for the community.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 12857