Evaluating Web Crawlers with Machine Learning Algorithms for Accurate Location Extraction from Job Offers

Pawel Drozda, Bartosz A. Nowak, Arkadiusz Talun, Leszek Bukowski

Abstract: This article focuses on systems designed to extract the location of a job listing from a job advertisement on web pages. It presents the use of classifiers to improve the reliability of automata used to collect this information. Three different algorithms - SVM, Random Forest and XGBoost - were used for this purpose. Accuracy, precision, recall and F1 score were used to evaluate the performance of each algorithm. While XGBoost performed best with an accuracy and F1 score of nearly 94%, all three algorithms showed very similar results. This suggests that each of the three algorithms can be effectively used to improve the accuracy of indexing robots in identifying jobs in job listings.