Zero-Shot Learning with Common Sense Knowledge Graphs

Nihal V. Nayak, Stephen Bach

Abstract: Zero-shot learning relies on semantic class representations such as hand-engineered attributes or learned embeddings to predict classes without any labeled examples. We propose to learn class representations by embedding nodes from common sense knowledge graphs in a vector space. Common sense knowledge graphs are an untapped source of explicit high-level knowledge that requires little human effort to apply to a range of tasks. To capture the knowledge in the graph, we introduce ZSL-KG, a general-purpose framework with a novel transformer graph convolutional network (TrGCN) for generating class representations. Our proposed TrGCN architecture computes non-linear combinations of node neighbourhoods. Our results show that ZSL-KG improves over existing WordNet-based methods on five out of six zero-shot benchmark datasets in language and vision.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

1. Revised the manuscript to incorporate the Action Editor's requested changes (see Appendix C).
2. Included new results based on the discussion with the reviewers (see Appendix B).

Video: https://youtu.be/VXt3MucMWvY

Code: https://github.com/BatsResearch/zsl-kg

Assigned Action Editor: ~Zsolt_Kira1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 66