Query by Self

Tong Zhao, Haibo Li, Weile Jia, Guangming Tan

Keywords: Active learning, Kalman Filter, Variance

Abstract: Training with hard-to-obtain and therefore valuable data, improving generalization performance, and accelerating training speed are all challenging problems in machine learning community. Active learning , whose performance depends on its query strategy, is a powerful tool for these challenges. Unlike the famous Query by Committee strategy aimed at classification problem and dependent on a committee of student, our proposed query by self is suitable to both classification and regression problem and requires only a student, which benefits from estimated output variance, the intermediate product of Kalman filtering optimization. This means larger scope of application and less requirement for computation and the number of data. Besides, this strategy reduces training time and improves accuracy via filtration of similar data and better generalization. We theoretically explain query by self strategy from the perspective of entropy. To verify effectiveness of query by self empirically, we conduct several experiments on two classical models in machine learning.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning