Licon: A Diverse, Controllable and Challenging Linguistic Concept Learning Benchmark

Shenglong Yu, Ying Zhang, wenya guo, Zhengkun Zhang, Ru Zhou, Xiaojie Yuan

Submission Type: Regular Long Paper

Submission Track: Resources and Evaluation

Keywords: Concept Learning, Zero-shot Learning, Linguistic Description

Abstract: Concept Learning requires learning the definition of a general category from given training examples. Most of the existing methods focus on learning concepts from images. However, the visual information cannot present abstract concepts exactly, which struggles the introduction of novel concepts related to known concepts (e.g., ‘Plant’→‘Asteroids’). In this paper, inspired by the fact that humans learn most concepts through linguistic description, we introduce Linguistic Concept Learning benchmark (Licon), where concepts in diverse forms (e.g., plain attributes, images, and text) are defined by linguistic descriptions. The difficulty to learn novel concepts can be controlled by the number of attributes or the hierarchical relationships between concepts. The diverse and controllable concepts are used to support challenging evaluation tasks, including concept classification, attribute prediction, and concept relationship recognition. In addition, we design an entailment-based concept learning method (EnC) to model the relationship among concepts. Extensive experiments demonstrate the effectiveness of EnC. The benchmark will be released to the public soon.

Submission Number: 1431