SkillBERT: "Skilling" the BERT to classify skills using Electronic Recruitment Records

Amber Nigam, Shikha Tyagi, Kuldeep Tyagi

Keywords: Competency, Electronic Recruitment Record, Recruitment, SkillBERT, Skill embeddings, Skill semantics

Abstract:

In this work, we show how the Electronic Recruitment Records (ERRs) that store the information related to job postings and candidates can be mined and analyzed to provide assistance to hiring managers in recruitment. These ERRs are captured through our recruitment portal, where hiring managers can post the jobs and candidates can apply for various job postings. These ERRs are stored in the form of tables in our recruitment database and whenever there is a new job posting, a new ERR is added to the database.
We have leveraged the skills present in the ERRs to train a BERT-based model, SkillBERT, the embeddings of which are used as features for classifying skills into groups referred to as “competency groups”. A competency group is a group of similar skills, and it is used as matching criteria (instead of matching on skills) for finding the overlap of skills between the candidates and the jobs. This proxy match takes advantage of the BERT’s capability of deriving meaning from the structure of competency groups present in the skill dataset. The skill classification is a multi-label classification problem as a single skill can belong to multiple competency groups. To solve multi-label competency group classification using a binary classifier, we have paired each skill with each competency group and tried to predict the probability of that skill belonging to that particular competency group. SkillBERT, which is trained from scratch on the skills present in job requisitions, is shown to be better performing than the pre-trained BERT (Devlin et al., 2019) and the Word2Vec (Mikolov et al., 2013). We have also explored K-means clustering (Lloyd, 1982) and spectral clustering (Chung, 1997) on SkillBERT embeddings to generate cluster-based features. Both algorithms provide similar performance benefits. Last, we have experimented with different classification models like Random Forest (Breiman, 2001), XGBoost (Chen and Guestrin, 2016), and a deep learning algorithm Bi-LSTM (Schuster and Paliwal, 1997; Hochreiter and Schmidhuber, 1997) for the tagging of competency groups to skill. We did not observe a significant performance difference among the algorithms, although XGBoost and Bi-LSTM perform slightly better than Random Forest. The features created using SkillBERT are most predictive in the classification task, which demonstrates that the SkillBERT is able to capture the information pertaining to skill ontology from the data. We have made the source code, the trained models, and the dataset.

URL: https://www.dropbox.com/s/wcg8kbq5btl4gm0/code_data_pickle_files.zip?dl=0