Continually learn to map visual concepts to language models in resource-constrained environments

Clea Rebillard, Julio Hurtado, Andrii Krutsylo, Lucia C. Passaro, Vincenzo Lomonaco

Abstract: Highlights•A small visual model can be trained with knowledge space created by a frozen LM.•CVM improves performance and mitigating forgetting in standard benchmarks.•We study the generalization and transfer capabilities of our proposal.•CL method based on a large pre-trained model fails in fine-grained datasets.•CVM achieves similar results with lower inference time in fine-grained datasets.

External IDs:dblp:journals/ijon/RebillardHKPL25