Exploring Stable Meta-optimization Patterns via Differentiable Reinforcement Learning for Few-shot Classification

Zheng Han, Xiaobin Zhu, Chun Yang, Hongyang Zhou, Jingyan Qin, Xu-Cheng Yin

Abstract: Existing few-shot learning methods generally focus on designing exquisite structures of meta-learners for learning task-specific prior to improve the discriminative ability of global embeddings. However, they often ignore the importance of learning stability in meta-training, making it difficult to obtain a relatively optimal model. From this key observation, we propose an innovative generic differentiable Reinforcement Learning (RL) strategy for few-shot classification. It aims to explore stable meta-optimization patterns in meta-training by learning generalizable optimizations for producing task-adaptive embeddings. Accordingly, our differentiable RL strategy models the embedding procedure of feature transformation layers in meta-learner to optimize the gradient flow implicitly. Also, we propose a memory module to associate historical and current task states and actions for exploring inter-task similarity. Notably, our RL-based strategy can be easily extended to various backbones. In addition, we propose a novel task state encoder to encode task representation, which fully explores inner-task similarities between support set and query set. Extensive experiments verify that our approach can improve the performance of different backbones and achieve promising results against state-of-the-art methods in few-shot classification. Our code is available at an anonymous site: https://anonymous.4open.science/r/db8f0c012/.

Primary Subject Area: [Content] Media Interpretation

Secondary Subject Area: [Content] Media Interpretation, [Engagement] Summarization, Analytics, and Storytelling

Relevance To Conference: Our work holds particular significance in real-world multimedia scenarios where resource constraints, data scarcity, or domain-specific challenges may limit the applicability of exclusively relying on large-scale foundation models. Meta-learning is the bridge to help large-scale models transfer vast knowledge from a data-abundant field to a data-scarce field. Besides, meta-learned few-shot learners can induce a good initial point for training large-scale models, greatly reducing the huge amount of needed samples and large training time. We aim to solve the instability issue of meta-learning. Future works on multimedia can benefit from our method when facing training instability problems, especially in meta-learning. Our method can also help alleviate the instability issue and huge cost of training large-scale models.

Submission Number: 956