Localizing Non-functional Code Bugs in User Interfaces Using Deep Learning Techniques

Arwa Ahmed, Ahmed Tamer Salah, Ghada Khoriba, Tamer Arafa

Abstract: In the context of modern business digitization, non-functional code issues in User Interface (UI) exert a substantial impact on User Experience(UX). UX-related problems can generate annoyance and potentially result in revenue erosion. However, conventional approaches for detecting and resolving these defects tend to be prolonged, mandate a substantial degree of expertise, and divert precious time and resources away from vital software development tasks, ultimately impairing business operations. A deep learning-based approach automatically detects and localizes non-functional code flaws in user interface (UI) screens. The automated approach reduces the time and effort required to fix non-functional code bugs in UI screens, improving overall quality control and testing and decreasing overall UI testing time. Our model can identify non-functional code flaws by analyzing many UI screens and pinpointing their location on UI screens. We evaluate our proposed approach on a variety of UI screens. The proposed approach achieves better performance in classification by 4% and in localization by 16%.