Learnware Specification via Dual Alignment

Wei Chen, Jun-Xiang Mao, Xiaozheng Wang, Min-Ling Zhang

TL;DR: This paper proposes a learnware specification via dual alignment, demonstrating superior performance in the learnware paradigm.

Abstract: The learnware paradigm aims to establish a learnware dock system that contains numerous leanwares, each consisting of a well-trained model and a specification, enabling users to reuse high-performing models for their tasks instead of training from scratch. The specification, as a unique characterization of the model's specialties, dominates the effectiveness of model reuse. Existing specification methods mainly employ distribution alignment to generate specifications. However, this approach overlooks the model's discriminative performance, hindering an adequate specialty characterization. In this paper, we claim that it is beneficial to incorporate such discriminative performance for high-quality specification generation. Accordingly, a novel specification approach named Dali, i.e., Learnware Specification via Dual ALIgnment, is proposed. In Dali, the characterization of the model's discriminative performance is modeled as discriminative alignment, which is considered along with distribution alignment in the specification generation process. Theoretical and empirical analyses clearly demonstrate that the proposed approach is capable of facilitating model reuse in the learnware paradigm with high-quality specification generation.

Lay Summary: The learnware system lets people reuse existing machine learning models instead of building new ones. Each model has a "specification" describing its capability. Existing methods focused on matching data patterns but ignored how well the model works on actual tasks. The new method, Dali, looks at both data patterns and real-world performance to create a better specification. Experiment and theory show this approach works better for finding useful models.

Primary Area: General Machine Learning->Transfer, Multitask and Meta-learning

Keywords: Learnware paradigm, learnware specification, model reuse

Submission Number: 10789