An empirical study of async wait flakiness in front-end testing

Automated front-end regression testing is an essential part of web development, allowing fast release cycles while maintaining high-quality requirements. However, due to the asynchronous nature of web applications, front-end testing is sensitive to Async Wait flakiness which reduces the usefulness of such test suites by introducing false alarms. In this work, we conducted an empirical study to investigate the causes and the impact of Async Wait flakiness in front-end testing. To do so, we build a dataset of 62 tests exhibiting reproducible Async Wait flakiness associated with a clean fix commit, which becomes the foundation of our study. Our preliminary results suggest that tests relying on an explicit time to wait in order to synchronize the tests tend to create more flakiness (38 instances) than synchronizing on the status of DOM elements (24 instances). Further study shows that where time-based issues are typically addressed by increasing the time to wait, DOM-based issues are resolved by actually introducing a missing synchronization point. We conclude our study with an analysis of the different implementations of synchronization mechanisms to provide tool manufacturers with concrete insights on how to improve their solutions.