Tree-Based Synthesis of Web Test Sequences from Manual Actions

Pengkun Jiang, Sinan Wang, Yepang Liu

Abstract: The thrive of web technologies and applications demands effective testing methods for quality assurance. For this purpose, researchers have proposed various testing techniques to automate the interaction with web applications in recent years. While such techniques help save human testers from repetitive and tedious manual testing tasks, they often only explore limited usage scenarios due to the lack of domain knowledge. In practice, constructing test sequences that align with application business logic still requires substantial manual efforts. In this work, we explore a new paradigm that leverages the synergy between humans and machines. Our key observation is that in many web applications, interactive elements within or across web pages often exhibit similarities in functionality and interaction style. Leveraging this observation, we propose a novel testing method, Fret, which combines manual recording of an example test sequence with automated synthesis of multiple new test sequences that have similar intentions through action mutation. To detect similar elements to the example, Fret employs a programming-by-example technique, in which similar elements are described as the intersection within a well-defined version space. To avoid redundant mutations, Fret builds a tree representation of the mutation process, which helps preserve the logical dependence between actions and monitor the progress of test generation. The experiment results on 32 top-ranking websites show that, given an example sequence, Fret can effectively generate an average of 138 new sequences to interact with the web application under test with a false positive rate of only 0.69% on similar element searching, significantly outperforming the baseline method in terms of covered elements and generated valid test sequences. Additionally, Fret has also led to the discovery of 86 real errors in 19 web applications, which demonstrates its practical usefulness.