# CLIP model is an Efficient Online Continual Learner

## Abstract

Online continual learning addresses the challenge of learning from continuous, non-stationary data streams. Existing online continual learning frameworks are classification-based and assume a pre-defined number of classes. In this study, we propose that vision-language models (VLMs) are more suitable candidates for online continual learning. Compared to traditional classification-based frameworks, VLMs such as CLIP model is not limited by the maximum number of classes or constrained by rigid model architectures, enabling it to generalize across both known and emerging classes. However, we find that naively tuning the CLIP for online continual learning results in asymmetric image-text matching. This asymmetric matching will consistently poses negative suppression on the previously learned classes, leading to catestrophic forgetting. To address this issue, we propose a simple yet effective method, the symmetric image-text (SIT) tuning strategy, which mitigates the adverse impact of negative samples by excluding asymmetric text during online learning. Additionally, we introduce a more challenging blurred boundary online continual learning setup, namely MiD-Blurry, which mixes multiple data distributions to mimic real-world scenarios. We conducted extensive experiments on several continual learning benchmarks as well as the MID-Blurry setting, evaluating both inference-at-any-time performance and generalization to future data. Our results demonstrate that the SIT strategy effectively preserves memory stability while maintaining learning plasticity.

## Introduction

Official repository for online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning. Code is based on [Si-Blurry](https://github.com/naver-ai/i-Blurry) and [MaPLe](https://github.com/muzairkhattak/multimodal-prompt-learning). 

## Requirements

- Pytorch
- timm
- clip

or you can install a conda environment with :

```Bash
   conda env create -f environment.yml
```

## Run

```Bash
   bash scripts/lora_clip.sh
```

