Enhancing UI Tests Robustness With Graph Convolutional Networks

Maroun Ayli, Youssef Bakouny, Hani Seifeddine, Nader Jalloul, Rima Kilany

Abstract: Web application testing faces significant challenges due to the dynamic nature of modern interfaces, often leading to fragile test scripts and increased maintenance overhead. This paper introduces a novel approach to enhance Selenium’s web element localization capabilities using Graph Convolutional Networks (GCNs). We propose a method that generates robust embeddings for web elements by integrating textual, visual, and structural features. Our GCN-based model constructs a graph representation of web pages, capturing complex relationships between elements. We present a recovery mechanism implemented in our tool WebEmbed that utilizes these embeddings to locate elements when traditional locators fail. Evaluation on a dataset of 20 manually modified open-source web applications demonstrates that our approach significantly outperforms baseline methods, achieving a 92.5% recovery rate without any prior annotation. This research contributes to more resilient automated testing practices, reducing script maintenance and improving test reliability in dynamic web environments.

External IDs:doi:10.1007/978-3-031-93598-5_27