Data Analytics in Text Messages: A Mobile Network Operator Case Study

Mohammadreza Babaee, Hamidreza Sarabadani

Abstract: This paper explores the application of different data mining and machine learning algorithms to propose an effective technique to filter out spam SMSs. Due to high competitive nature of MNO business; filtering spam SMSs will have a great impact on the protection of business and profit making. This is mostly because subscribers refuse to use the services of MNOs that are not vigilant about spam SMSs. Based on the CRISP-DM method which is an open standard process model for data analytics projects, machine learning algorithms and data preparation methods have been conducted on a MNO unstructured dataset to transform characters, delete stop words, extract word stems, roots, N-Grams, and classification. Next, numerical Vector Space Models were created utilizing all four types of word vector creation methods. After producing test and train models with machine learning algorithms; accuracy and error rate, recall, precision and the area under curve for each classification algorithm has been measured. Finally, the Bagging algorithm by implementing Binary Term Occurrence vector space creation method showed the highest efficiency rate which can have the highest application in the big data ecosystem of the industry for spam filtering.