Enhanced Analysis of User Perceptions Through Natural Language Processing Approaches

Gabriel Prenassi, Ana Machado, Davi Freitas, André Lima, Raquel O. Prates, Antonio Landim, Leonardo Rocha, Elisa Tuler

Abstract: In this paper, we present a framework that assists in analyzing textual data collected from user comments to complement the analysis obtained with the System Usability Score (SUS), a traditional methodology in Human-Computer Interaction (HCI) for evaluating computer systems. Our framework performs textual analysis through the coordinated use of different Natural Language Processing (NLP) techniques, such as; (i) Topic modeling; (ii) Text Summarization; and (iii) Sentiment analysis. Topic modeling finds semantic topics from textual comments. Text Summarization is applied in each set of documents for each topic, allowing their explainability. The sentiment analysis detects the sentiment associated with each topic to identify which ones correlate with positive aspects and which ones must be improved. We also propose a visual interface that provides an intuitive visualization for all evaluations. Our framework has been applied in a real scenario. The initial analysis conducted indicated that the framework allowed for a thorough analysis of the most critical issues of the functionality, highlighting more precise feedback for the development team.

External IDs:dblp:conf/interact/PrenassiMFLPLRT25