Empirical Performance of E2E Frameworks in React-Vue SPAs Using DIA
DOI:
https://doi.org/10.59395/ijadis.v7i1.1528Keywords:
Automation Testing, End-to-End Testing, Performance Evaluation, Playwright, Selenium WebDriver, Cypress, Single Page ApplicationAbstract
Modern web applications increasingly adopt Single-Page Application (SPA) architectures to enhance the user experience through client-side rendering and dynamic content loading. However, these characteristics introduce significant challenges for automated end-to-end (E2E) testing, including asynchronous DOM manipulation, complex state management, and timing synchronization issues. This study presents a comprehensive empirical comparison of three prominent E2E testing frameworks—Selenium WebDriver, Cypress, and Playwright—across React and Vue-based SPAs. Using a quantitative experimental approach, 25 standardized test cases were executed 15 times each across Chrome, Firefox, and Edge, for a total of 270 testing sessions. Performance evaluation focused on four key metrics: execution time, success rate, CPU usage, and memory consumption. Results demonstrate that Playwright achieved the fastest execution time (56.25 seconds on React-Chrome), while Selenium exhibited superior resource efficiency with the lowest memory consumption (196.59 MB on Vue-Chrome). The Distance to Ideal Alternative (DIA) multi-criteria decision analysis method identified Playwright-Chrome as optimal for React applications (DIA score: 0.886715) and Selenium-Chrome for Vue applications (DIA score: 0.908237), indicating that framework selection should be context-dependent based on application characteristics and deployment requirements. This research supports the conclusion that no universal "best" testing framework exists, underscoring the importance of evidence-based, application-specific tool selection in software quality assurance.
Downloads
References
[1] A. Bai, Million.js: A Fast Compiler-Augmented Virtual DOM for the Web, Proc. ACM Symp. Appl. Comput., pp. 18131820, 2023, doi: 10.1145/3555776.3577683. DOI: https://doi.org/10.1145/3555776.3577683
[2] M. Siahaan et al., Comparative Analysis Study of Front-End JavaScript Frameworks Performance Using Lighthouse Tool, J. Mantik, vol. 6, no. 3, pp. 26854236, 2021.
[3] Octa Selsa Is Anggraeni, Lilik Sugiarto, and Tinuk Agustin, Studi Komparatif Performa Framework Javascript Modern dalam Pengembangan Aplikasi Web, Modem J. Inform. dan Sains Teknol., vol. 2, no. 4, pp. 162177, 2024, doi: 10.62951/modem.v2i4.239. DOI: https://doi.org/10.62951/modem.v2i4.239
[4] A. M. Syahputra, Analisis perbandingan performansi framework Selenium dan Cypress dalam pengujian otomatisasi perangkat lunak pada aplikasi berbasis website, J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 8, no. 3, pp. 39593966, 2024.
[5] H. A. Thooriqoh, T. N. Annisa, and U. L. Yuhana, Selenium Framework for Web Automation Testing: A Systematic Literature ReviewKerangka Selenium untuk Pengujian Otomatis Berbasis Web: Tinjauan Literatur Sistematis, J. Ilm. Teknol. Inf., vol. 19, no. 2, pp. 6576, 2021, [Online]. Available: http://dx.doi.org/10.12962/j24068535.v19i2.a1021 DOI: https://doi.org/10.12962/j24068535.v19i2.a1021
[6] M. Leotta, B. Garcia, F. Ricca, and J. Whitehead, Challenges of End-to-End Testing with Selenium WebDriver and How to Face Them: A Survey, Proc. - 2023 IEEE 16th Int. Conf. Softw. Testing, Verif. Validation, ICST 2023, pp. 339350, 2023, doi: 10.1109/ICST57152.2023.00039. DOI: https://doi.org/10.1109/ICST57152.2023.00039
[7] M. Vieira, V. M. de Lima, W. Viana, M. Bonfim, and P. Rego, Enhancing Continuous Integration Workflows: End-to-End Testing Automation with Cypress, vol. 2, no. Iceis, pp. 160167, 2025, doi: 10.5220/0013230200003929. DOI: https://doi.org/10.5220/0013230200003929
[8] S. B. Nettur, S. Karpurapu, U. Nettur, and L. S. Gajja, Cypress Copilot: Development of an AI Assistant for Boosting Productivity and Transforming Web Application Testing, IEEE Access, vol. 13, no. November 2024, pp. 32153229, 2025, doi: 10.1109/ACCESS.2024.3521407. DOI: https://doi.org/10.1109/ACCESS.2024.3521407
[9] S. Talakola, Automated end to end testing with Playwright for React applications, Int. J. Emerg. Res. Eng. Technol., vol. 5, no. 1, pp. 3847, 2024, doi: 10.63282/3050-922x.ijeret-v5i1p106. DOI: https://doi.org/10.63282/3050-922X.IJERET-V5I1P106
[10] S. P. Wijaya, M. A. Al Abdilah, M. Farhan, and A. Firmansyah, ANALISIS PERBANDINGAN EFISIENSI DAN PERFORMA PENGUJIAN SELENIUM, CYPRESS, DAN PLAYWRIGHT SEBAGAI ALAT AUTOMATION TESTING UNTUK APLIKASI WEB, J. Sains dan Teknol., vol. 8, no. 10, pp. 91100, 2025, doi: 10.2238/rq4d6a48.
[11] M. Ayli, Y. Bakouny, N. Jalloul, and R. Kilany, Enhancing the Resiliency of Automated Web Tests with Natural Language, ACM Int. Conf. Proceeding Ser., no. Ml, pp. 6369, 2024, doi: 10.1145/3700523.3700536. DOI: https://doi.org/10.1145/3700523.3700536
[12] S. Balsam and D. Mishra, Web application testingChallenges and opportunities, J. Syst. Softw., vol. 219, no. August 2024, p. 112186, 2025, doi: 10.1016/j.jss.2024.112186. DOI: https://doi.org/10.1016/j.jss.2024.112186
[13] S. Adiatma and A. Darmayantie, Implementation and Comparative Analysis of Test Automation Framework Performance for Functional Testing of Web-Based Applications using the Distance to the Ideal Alternative (DIA) Method, Widya Tek., vol. 22, no. 2, pp. 3641, 2023, doi: 10.33508/wt.v22i1.5027. DOI: https://doi.org/10.33508/wt.v22i1.5027
[14] M. Cinelli, M. Kadziski, G. Miebs, M. Gonzalez, and R. Sowiski, Recommending multiple criteria decision analysis methods with a new taxonomy-based decision support system, Eur. J. Oper. Res., vol. 302, no. 2, pp. 633651, 2022, doi: 10.1016/j.ejor.2022.01.011. DOI: https://doi.org/10.1016/j.ejor.2022.01.011
[15] C. Merina, N. Anggraini, and N. Hakiem, A comparative analysis of test automation frameworks performance for functional testing in android-based applications using the distance to the ideal alternative method, Proc. 3rd Int. Conf. Informatics Comput. ICIC 2018, pp. 16, 2018, doi: 10.1109/IAC.2018.8780548. DOI: https://doi.org/10.1109/IAC.2018.8780548
[16] Y. P. Cardenas, Application of multi-criteria decision analysis to the selection of software measures, Comput. y Sist., vol. 22, no. 1, pp. 203213, 2018, doi: 10.13053/CyS-22-1-2765. DOI: https://doi.org/10.13053/cys-22-1-2765
[17] G. A. Prasetyo and N. Setiani, Perbandingan Kinerja Framework Automation UI Testing Menggunakan The Distance to The Ideal Alternative, JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 10, no. 1, pp. 224237, 2025, doi: 10.29100/jipi.v10i1.5760. DOI: https://doi.org/10.29100/jipi.v10i1.5760
[18] ISO; IEC; IEEE, Software and systems engineering Software testing Part 4: Test techniques, Geneva, 2022. [Online]. Available: https://www.iso.org/obp/ui/en/#iso:std:79430:en
[19] B. B. Agnieszka Antonczak, Analysis of end-to-end test automation tools based on the examples of Selenium WebDriver and Playwright, Position Pap. 19th Conf. Comput. Sci. Intell. Syst., vol. 40, no. FedCSIS Conference Proceedings, pp. 18, 2024, doi: DOI: 10.15439/2024F3747. DOI: https://doi.org/10.15439/2024F3747
[20] N. L. P. Melyawati, I. M. D. P. Asana, N. N. S. Putri, K. J. Atmaja, and I. G. I. Sudipa, Comparison of Automation Testing On Card Printer Project Using Playwright And Selenium Tools, J. Comput. Networks, Archit. High Perform. Comput., vol. 6, no. 3, pp. 13091320, 2024, doi: 10.47709/cnahpc.v6i3.4362. DOI: https://doi.org/10.47709/cnahpc.v6i3.4362
[21] N. Arifin, N. Agustin, and T. Anggoro, Evaluation Execution Time Features of Simpati Web-Based Monitoring and Evaluation Application Using Automation Testing, J. Tek. Inform., vol. 5, no. 4, pp. 283289, 2024, doi: 10.52436/1.jutif.2024.5.4.2158. DOI: https://doi.org/10.52436/1.jutif.2024.5.4.2158
[22] H. Rojas, R. Renteria, and V. Martinez, Understanding Performance Efficiency in ISO / IEC 25000: A Systematic Literature Review, vol. 9, no. November, pp. 24052417, 2025. DOI: https://doi.org/10.62527/joiv.9.6.3074
[23] Y. Wang, A. M. Menzies, M. Mntyl, and M. Nagappan, Test automation maturity improves product quality, arXiv preprint arXiv:2202.04068, 2022.
[24] A. P. Putra, D. Sartika, and D. Lianda, Decision Support System For The Best Junior High School Using The Distance To The Ideal Alternative ( Dia ) Method At The Education And Culture Office Of The City Of Bengkulu Sistem Pendukung Keputusan Sekolah Terbaik Tingkat Menengah Pertama Menggunakan Metode Distance To The Ideal Alternative ( Dia ) Pada Dinas Pendidikan Dan Kebudayaan Kota Bengkulu, vol. 3, no. 2, pp. 159166, 2024. DOI: https://doi.org/10.37676/jmcs.v3i2.6618
[25] F. R. B. A. N. R. Setiabudi, COMPARATIVE ANALYSIS OF AUTOMATION FUNCTIONAL TESTING, vol. 21, no. 1, pp. 914, 2024.
[26] F. S. D. Silva, M. P. S. Lima, D. Corujo, S. Member, A. J. V. Neto, and S. Member, A Comprehensive Step-Wise Survey of Multiple Attribute Decision-Making Mobility Approaches, vol. 12, no. July, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3436074
[27] B. Sani and S. Jan, EAI Endorsed Transactions Empirical Analysis of Widely Used Website Automated Testing Tools, vol. 3, pp. 111, doi: 10.4108/airo.7285. DOI: https://doi.org/10.4108/airo.7285
[28] S. Di Meglio, L. L. L. Starace, V. Pontillo, R. Opdebeeck, C. De Roover, and S. Di Martino, E2EGit: A Dataset of End-to-End Web Tests in Open Source Projects, Proc. - 2025 IEEE/ACM 22nd Int. Conf. Min. Softw. Repos. MSR 2025, pp. 836840, 2025, doi: 10.1109/MSR66628.2025.00121. DOI: https://doi.org/10.1109/MSR66628.2025.00121
[29] G. C. B. Miranda, A Comparative Analysis of Mobile UI Testing Frameworks in Continuous Integration Environments, ACM Int. Conf. Mob. Softw. Eng. Syst. (MOBILESoft 2023), pp. 102109, 2023, doi: 10.1145/3624032.3624047. DOI: https://doi.org/10.1145/3624032.3624047
[30] A. Tymoshchuk, The Evolution of Test Automation: From Selenium to Playwright . A Comparison of Automation Tools: Selenium vs . Playwright vs . Cypress, vol. 4523, pp. 3745.
[31] M. Mo and B. Paczyk, A comparative analysis of web application test automation tools Analiza porwnawcza narzdzi do automatyzacji testw aplikacji internetowych, vol. 35, no. January, pp. 159165, 2025.
[32] P. Gosik and M. Miosz, Comparative analysis of Cypress and Playwright frameworks in end-to-end testing for applications based on Angular Analiza porwnawcza frameworkw Cypress i Playwright w testach end-to-end dla aplikacji opartych na Angularze, vol. 36, no. May, pp. 320327, 2025. DOI: https://doi.org/10.35784/jcsi.7662
[33] T. M. R. Lagermann, K. S. Carter, S. Mei, G. Ho, and K. Eder, Framework Matters: Energy Efficiency of UI Automation Testing Frameworks, vol. 1, no. 1. arXiv.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Abdillah Rezeki, Setyo Wahyu Saputro, Triando Hamonangan Saragih, Radityo Adi Nugroho, Friska Abadi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Cite
Share
Plum Analytics