Effectiveness Fine-Tuned Multilingual BERT Model for Sentiments Classification Toward Bali's Cultural Attractions

Nengah Widya Utami; Amna Binti  Saad; Made Adi Paramartha  Putra; I Gede Juliana Eka  Putra

doi:10.59395/ijadis.v7i1.1483

Authors

Nengah Widya Utami Faculty of Information Technology and Design, Primakara University, Indonesia
Amna Binti Saad Malaysian Institute of Information Technology, University Kuala Lumpur, Malaysia
Made Adi Paramartha Putra Faculty of Information Technology and Design, Primakara University, Indonesia
I Gede Juliana Eka Putra Faculty of Information Technology and Design, Primakara University, Indonesia

DOI:

https://doi.org/10.59395/ijadis.v7i1.1483

Keywords:

Multilingual BERT, Sentiments Classification, Machine Learning , Cultural Tourism , Tourist Review

Abstract

This study examines the performance of a fine-tuned Multilingual BERT (mBERT) model for sentiment analysis of tourist reviews on Balinese cultural attractions. A multilingual dataset comprising 7,878 user-generated reviews from Google Maps and TripAdvisor was utilized to capture diverse linguistic expressions and visitor perspectives. The research methodology includes: (1) problem formulation and literature review; (2) dataset collection, preprocessing, and tokenization; (3) model training using mBERT as the baseline; (4) fine-tuning for domain adaptation; and (5) comparative evaluation with other Transformer models (XLM-Roberta and Distil-mBERT) and classical algorithms including Logistic Regression, Support Vector Machine, and Naïve Bayes. The results demonstrate a substantial improvement after fine-tuning. The baseline mBERT achieved 85.45% accuracy, while the fine-tuned model reached 92.13% accuracy with an AUC of 0.909, confirming the effectiveness of domain-specific adaptation. Although XLM-Roberta obtained slightly higher performance (93.15% accuracy, AUC 0.946), the fine-tuned mBERT showed stable and competitive results, making it the primary model of this study. Comparisons with classical methods further indicate that Transformer-based approaches provide more balanced and reliable sentiment classification. Sentiment distribution analysis reveals that tourist perceptions are predominantly positive, particularly regarding cultural authenticity and the quality of performances such as the Kecak and Fire Dance. Negative sentiments mainly relate to operational aspects, including crowd management, seating arrangements, and ticketing processes. Overall, this study provides empirical evidence that fine-tuned mBERT can effectively support data-driven evaluation of tourist experiences and deliver actionable insights for improving service quality and sustainability of Bali’s cultural tourism

916 624

Downloads

Download data is not yet available.

References

[1] N. L. P. S. Widhiastuty and I. K. Murdana, “The Impacts of Bali cultural Tourism to the Local Balinese Community,” LACULTOUR: Journal of Language and Cultural Tourism, vol. 3, no. 1, pp. 45–51, Jun. 2024, doi: 10.52352/lacultour.v3i1.1360. DOI: https://doi.org/10.52352/lacultour.v3i1.1360

[2] S. Yufei and D. Lei, “Research on aspect-level sentiment analysis of tourism review text based on joint learning,” in 2023 IEEE International Conference on Control, Electronics and Computer Technology, ICCECT 2023, Institute of Electrical and Electronics Engineers Inc., 2023, pp. 1182–1186. doi: 10.1109/ICCECT57938.2023.10141071. DOI: https://doi.org/10.1109/ICCECT57938.2023.10141071

[3] F. Hastarita Rachman and B. Setya Rintyarna, “Sentiment Analysis of Madura Tourism in New Normal Era using Text Blob and KNN with Hyperparameter Tuning,” 2022. DOI: https://doi.org/10.1109/ISMODE53584.2022.9742894

[4] N. Widya Utami, N. Purnama, I. Made, and A. Prayoga, “Sentiment Analysis System Of Bali Tourism Using Naive Bayes Algorithm And Web Framework,” Informatika dan Sains, vol. 14, p. 2024, 2024, doi: 10.54209/infosains.v14i03.

[5] Y. A. Singgalen, “Improved Sentiment Classification Using Multilingual BERT with Enhanced Performance Evaluation for Hotel Guest Review Analysis,” Journal of Computer System and Informatics (JoSYC), vol. 6, no. 2, pp. 508–520, 2025, doi: 10.47065/josyc.v6i2.6870. DOI: https://doi.org/10.47065/josyc.v6i2.6870

[6] R. RICCIARDI and M. Manisera, “A Multilingual BERT-based classification of reviews for enhanced visitors’ experience analysis,” Jul. 31, 2024. doi: 10.21203/rs.3.rs-4495603/v1. DOI: https://doi.org/10.21203/rs.3.rs-4495603/v1

[7] N. Akhila, J. P. Sanjanasri, and K. P. Soman, “Comparative Study of Bert Models and Roberta in Transformer based Question Answering,” in 2023 3rd International Conference on Intelligent Technologies, CONIT 2023, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/CONIT59222.2023.10205622. DOI: https://doi.org/10.1109/CONIT59222.2023.10205622

[8] M. Rizky Yanuar and S. Shiramatsu, “Aspect Extraction for Tourist Spot Review in Indonesian Language using BERT.”

[9] S. Islam, M. Jahidul Islam, M. Mahadi Hasan, S. M. Shahnewaz Mahmud Ayon, and S. Shabnam Hasan, “Bengali Social Media Post Sentiment Analysis using Deep Learning and BERT Model,” in 2022 IEEE Symposium on Industrial Electronics and Applications, ISIEA 2022, Institute of Electrical and Electronics Engineers Inc., 2022. doi: 10.1109/ISIEA54517.2022.9873680. DOI: https://doi.org/10.1109/ISIEA54517.2022.9873680

[10] J. Xue and Y. Li, “Intelligent Tourism System Based on a BERT Model for the Cultural Relics Domain,” in 2025 4th International Symposium on Computer Applications and Information Technology, ISCAIT 2025, Institute of Electrical and Electronics Engineers Inc., 2025, pp. 195–199. doi: 10.1109/ISCAIT64916.2025.11010753. DOI: https://doi.org/10.1109/ISCAIT64916.2025.11010753

[11] A. Chinnalagu, Comparative Analysis of Fine-Tuned LLM, BERT and DL Models for Customer Sentiment Analysis. IEEE, 2024. DOI: https://doi.org/10.1109/SMART63812.2024.10882546

[12] M. J. Shayegan and Y. Dastan, “A Multilingual Sentiment Analysis Model in Tourism,” 2024. [Online]. Available: http://jac.ut.ac.ir

[13] R. Kumar. Dwivedi and A. Kr. Saxena, Vietnamese Question Answering System from Multilingual BERT Models to Monolingual BERT Model. Prof. Rakesh Kumar Dwivedi, Faculty of Engineering & Computing Sciences, Teerthanker Mahaveer University, 2020.

[14] N. Khurana and V. Prabhu, “Sentiment Analysis Using Transformer Models (BERT, ALBERT, RoBERTa, and DeBERTa) on a SMILE Twitter Dataset.” [Online]. Available: http://t.co/GYKpsOb0W6

[15] N. W. Utami, “Dataset Review Bali’s Cultural Attractions (Multilingual),” Mendeley Data.

[16] N. T. M. Trang and M. Shcherbakov, “Vietnamese question answering system from multilingual BERT models to monolingual BERT Model,” in Proceedings of the 2020 9th International Conference on System Modeling and Advancement in Research Trends, SMART 2020, Institute of Electrical and Electronics Engineers Inc., Dec. 2020, pp. 201–205. doi: 10.1109/SMART50582.2020.9337155. DOI: https://doi.org/10.1109/SMART50582.2020.9337155

[17] T. Kasana and B. Kaur, “Evaluating Student Feedback: A Comparative Study of Traditional ML and DistilBERT Model,” in 2025 5th International Conference on Intelligent Technologies, CONIT 2025, Institute of Electrical and Electronics Engineers Inc., 2025. doi: 10.1109/CONIT65521.2025.11167038. DOI: https://doi.org/10.1109/CONIT65521.2025.11167038

[18] S. Imtiaz and E. K. Hashi, “Analyzing the Performance of Sentiment Analysis Using BERT, DistilBERT, RoBERTa and GPT-2,” in 2024 6th International Conference on Sustainable Technologies for Industry 5.0, STI 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/STI64222.2024.10951049. DOI: https://doi.org/10.1109/STI64222.2024.10951049

[19] S. Riyadi, A. M. Masyhur, and A. D. Andriyani, “Deep Learning for Hate Speech Intensity Analysis: DistilBERT Classification Algorithm,” in 2024 IEEE Symposium on Industrial Electronics and Applications, ISIEA 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ISIEA61920.2024.10607351. DOI: https://doi.org/10.1109/ISIEA61920.2024.10607351

[20] M. S. Arefin, M. Serajun Nabi, M. Touhami, Z. U. Rehman, M. Faizal Ahmad Fauzi, and H. Bin Abdul Karim, “Aspect-Based Sentiment Analysis for Product Reviews using DistilBERT,” Institute of Electrical and Electronics Engineers (IEEE), Dec. 2025, pp. 1–6. doi: 10.1109/mecon67253.2025.11276893. DOI: https://doi.org/10.1109/MECON67253.2025.11276893

[21] S. R. Varghese, S. Juliet, and N. S. Athish, “Social Media Text Analysis for Disaster Management Using DistilBERT Model,” in Proceedings of 2024 International Conference on Science, Technology, Engineering and Management, ICSTEM 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ICSTEM61137.2024.10560620. DOI: https://doi.org/10.1109/ICSTEM61137.2024.10560620

[22] J. M. Ruel Espiritu, R. J. E. Policarpio, A. C. B. Sabado, V. M. B. Sabado, and H. T. Sueno, “Classifying Fake News in Filipino Language using Improved DistilBERT Model,” in Proceedings - 11th International Conference on Information Technology, Computer and Electrical Engineering, ICITACEE 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 320–325. doi: 10.1109/ICITACEE62763.2024.10762768. DOI: https://doi.org/10.1109/ICITACEE62763.2024.10762768