Automating Visual Testing in Android Applications

Cherkashyn, Oleksii

doi:Under indexing

Proceedings of International Conference on Applied Innovation in IT · 2026/04/22 · Vol. 14 · Issue 2 · pp. 195–202

Automating Visual Testing in Android Applications

Oleksii Cherkashyn

📄 Download PDF DOI: Under indexing

Abstract

Automated visual testing has become increasingly important in Android application development due to the rapid growth of device fragmentation, diverse screen resolutions, multiple operating system versions, and the widespread adoption of cross-platform frameworks. Ensuring consistent user interface (UI) rendering across different environments is a critical quality attribute, particularly for applications distributed to large and heterogeneous user bases. Traditional functional testing approaches are often insufficient to detect subtle visual regressions that directly affect user experience. Therefore, reliable and scalable visual test automation techniques are essential. This study extends existing research by empirically evaluating pixel-level visual testing across multiple execution environments. The results demonstrate that automated visual testing is feasible and stable in an Android headless emulator environment. Furthermore, the cross-platform nature of mobile applications, including different development frameworks, does not negatively impact the effectiveness of the approach. The findings also confirm that the execution platform-whether a real device, a local emulator, or a cloud-based environment-does not significantly influence test reliability. Finally, the implementation based on the Pixelmatch library shows stable and consistent performance across all experimental conditions. These results highlight the practical applicability and robustness of the proposed visual testing methodology for Android applications.

Keywords

Android Android Application Testing Mobile Test Automation WebdriverIO Appium Pixelmatch.

References

A. Bilal, H. T. Mirza, I. Hussain, and A. Ahmad, “Investigating Influence of Google Play Application Titles on Success,” Big Data Research, vol. 36, 2024, [Online]. Available: https://doi.org/10.1016/j.bdr.2024.100443.
A. Niroshan, S. Seneviratne, and A. Seneviratne, “An Empirical Study of Code Obfuscation Practices in the Google Play Store,” arXiv preprint, arXiv:2502.04636, 2025.
O. Cherkashyn, “Application Test Automation in Headless Android Emulator,” Proceedings of International Conference on Applied Innovation in IT, vol. 13, no. 5, pp. 445-452, 2025, [Online]. Available: https://doi.org/10.25673/123064.
Y. Wu, J. Ling, X. Luo, T. Yang, M. Zhao, C. He, and Y. Wu, “Vision-Based Mobile App GUI Testing: A Survey,” ACM Computing Surveys, vol. 58, no. 6, pp. 1-46, Oct. 2025, [Online]. Available: https://doi.org/10.1145/3773027.
R. Coppola, L. Ardito, and M. Torchiano, “Fragility of layout-based and visual GUI test scripts: An assessment study on a hybrid mobile application,” in Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation (A-TEST ’19), New York, NY, USA, Aug. 2019, pp. 28-34, [Online]. Available: https://doi.org/10.1145/3340433.3342824.
S. Adiatma and A. Darmayantie, “Implementation and Comparative Analysis of Test Automation Framework Performance for Functional Testing of Web-Based Applications using the Distance to the Ideal Alternative (DIA) Method,” Widya Teknik, vol. 22, no. 1, 2023, [Online]. Available: https://doi.org/10.33508/wt.v22i1.5027.
S. Godboley, D. Dalei, R. Sadam, and D. P. Mohapatra, “Agile GUI Testing by computing novel Mobile App Coverage Using Appium Tool,” in Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, pp. 1026-1029, 2023, [Online]. Available: https://doi.org/10.1145/3555776.3577806.
H. Sun, A. Rosà, D. Bonetta, and W. Binder, “Automatically Assessing and Extending Code Coverage for NPM Packages,” in 2021 IEEE/ACM International Conference on Automation of Software Test (AST 2021), pp. 40-49, 2021, [Online]. Available: https://doi.org/10.1109/AST52587.2021.00013.
J. Yang, C. E. Jimenez, A. L. Zhang, K. Lieret, J. Yang, X. Wu, O. Press, N. Muennighoff, G. Synnaeve, K. R. Narasimhan, D. Yang, S. I. Wang, and O. Press, “SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?,” in The Thirteenth International Conference on Learning Representations (ICLR), 2025, [Online]. Available: https://doi.org/10.48550/arXiv.2410.03859
L. Zamprogno, B. Hall, R. Holmes, and J. M. Atlee, “Dynamic Human-in-the-Loop Assertion Generation,” IEEE Transactions on Software Engineering, vol. 49, no. 4, pp. 2337-2351, Apr. 2023, [Online]. Available: https://doi.org/10.1109/TSE.2022.3194038.
L. Ardito, R. Coppola, M. Morisio, and M. Torchiano, “Espresso vs. EyeAutomate: An Experiment for the Comparison of Two Generations of Android GUI Testing,” in Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering (EASE ’19), pp. 13-22, 2019, [Online]. Available: https://doi.org/10.1145/3319008.3319022.
D. Kraus, J. Roessler, and M. Sulzmann, “Visual Testing of GUIs by Abstraction,” arXiv preprint, arXiv:2007.10419, 2020, [Online]. Available: https://arxiv.org/abs/2007.10419.
Y. Li, Z. Yang, Y. Guo, and X. Chen, “Humanoid: A deep learning-based approach to automated black-box Android app testing,” in Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE ’19), pp. 1070-1073, 2019, [Online]. Available: https://doi.org/10.1109/ASE.2019.00104.
Z. Liu, C. Li, C. Chen, J. Wang, B. Wu, Y. Wang, J. Hu, and Q. Wang, “VisionDroid: Vision driven Automated Mobile GUI Testing via Multimodal Large Language Model,” arXiv preprint, arXiv:2407.03037v1, 2024, [Online]. Available: https://arxiv.org/abs/2407.03037v1.
D. Ran, Z. Li, C. Liu, W. Wang, W. Meng, X. Wu, H. Jin, J. Cui, X. Tang, and T. Xie, “Automated Visual Testing for Mobile Apps in an Industrial Setting,” in Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP ’22), pp. 55-64, 2022, [Online]. Available: https://doi.org/10.1145/3510457.3513027.
R. Mahmood, N. Mirzaei, and S. Malek, “EvoDroid: Segmented Evolutionary Testing of Android Apps,” in Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE ’14), pp. 599-609, 2014, [Online]. Available: https://doi.org/10.1145/2635868.2635896.
Y. Lan, Y. Lu, Z. Li, M. Pan, W. Yang, T. Zhang, and X. Li, “Deeply Reinforcing Android GUI Testing with Deep Reinforcement Learning,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE ’24), art. no. 71, pp. 1-13, Feb. 2024, [Online]. Available: https://doi.org/10.1145/3597503.3623344.
T. Gu, C. Sun, X. Ma, C. Cao, C. Xu, Y. Yao, Q. Zhang, J. Lu, and Z. Su, “Practical GUI Testing of Android Applications via Model Abstraction and Refinement,” in Proceedings of the 41st International Conference on Software Engineering (ICSE ’19), pp. 269-280, 2019, [Online]. Available: https://doi.org/10.1109/ICSE.2019.00042.
D. S. Dupakuntla Naga, “Cross-Platform Mobile Testing with Appium: A Framework for High-Accuracy Validation in Healthcare,” International Research Journal of Engineering and Technology (IRJET), vol. 12, no. 10, pp. 450-462, Oct. 2025, [Online].
A. J. Swart and P. E. Hertzog, “Quantifying the Percentage of Shading on a PV Module and Its Subsequent Impact on Its Output Power,” European Chemical Bulletin, vol. 12, special issue 3, pp. 6627-6635, Jul. 2023, [Online]. Available: https://doi.org/10.31838/ecb/2023.12.s3.747.