Indonesian Sign Language (BISINDO) Classification Using Xception Transfer Learning Architecture

Meisya Vira  Amelia; Wahyu Syaifullah Jauharis  Saputra; Kartika Maulida  Hindrayani; Prismahardi Aji  Riyantoko

doi:10.59395/ijadis.v6i2.1392

Authors

Meisya Vira Amelia Universitas Pembangunan Nasional "Veteran" Jawa Timur
Wahyu Syaifullah Jauharis Saputra Universitas Pembangunan Nasional "Veteran" Jawa Timur
Kartika Maulida Hindrayani Universitas Pembangunan Nasional "Veteran" Jawa Timur
Prismahardi Aji Riyantoko Department of Information and Communication Systems, Okayama University, Japan

DOI:

https://doi.org/10.59395/ijadis.v6i2.1392

Keywords:

BISINDO, Fine Tuning, Sign Language, Transfer Learning, Xception, Indonesian Sign Language, CNN, deep learning

Abstract

Human communication generally relied on speech. However, this was not applicable to the deaf people, who depended on sign language for daily interactions. Unfortunately, not everyone had the ability to understand sign language. In higher education environments, the lack of individuals proficient in sign language often created inequality in the learning process for deaf students. This limitation could be addressed by fostering a more inclusive environment, one of which was through the implementation of a sign language translation system. Therefore, this study aimed to develop a machine learning model capable of detecting and translating Indonesian Sign Language (BISINDO) alphabet gestures. The model was built using the Xception transfer learning method from Convolutional Neural Networks (CNN). The dataset consisted of 26 BISINDO alphabet gestures with a total of 650 images. The model was evaluated using K-Fold cross-validation and achieved an F1-score of 94% during testing.

1751 665

Downloads

Download data is not yet available.

References

[1] T. M. Milyane and Dkk, Pengantar Ilmu Komunikasi, vol. 5, no. 3. 2022. [Online]. Available: https://repository.penerbitwidina.com/media/publications/557082-pengantar-ilmu-komunikasi-22ec77af.pdf

[2] BPS, “Penduduk Menurut Wilayah dan Tingkat Kesulitan Mendengar,” Badan Pusat Statistik. Accessed: Sep. 13, 2024. [Online]. Available: https://sensus.bps.go.id/topik/tabular/sp2022/145/0/0

[3] BPS, “Jumlah Penduduk Kesulitan Berbicara,” Badan Pusat Statistik. Accessed: Sep. 13, 2024. [Online]. Available: https://sensus.bps.go.id/topik/tabular/sp2022/146/1/0

[4] LMD, “Sebaran Mahasiswa Disabilitas,” Layanan Mahasiswa Disabilitas. Accessed: Sep. 13, 2024. [Online]. Available: https://ptinklusif.kemdiktisaintek.go.id/s/5/sebaran-mahasiswa-disabilitas

[5] Effranzudeta, Yesinovitasari, Herdi, and Dinafitriani, “Journal of Special Education Lectura Urgensi Bahasa Isyarat di Lingkungan Universitas,” vol. 2, no. 1, pp. 65–72, 2024.

[6] D. A. Prasetya and T. P. Anggriawan, “Implementation of Information Technology to Enhance Institutional Presence,” in Nusantara Science and Technology Proceedings, 2025, pp. 242–245. DOI: https://doi.org/10.11594/nstp.2025.4737

[7] W. S. J. Saputra, M. H. Fardana, and M. A. R. Valentino, “Implementasi AI Pendeteksi Pola Gerak Tangan pada Game ‘Pong Ball’ dengan Menggunakan Algoritma Convolutional Neural Network,” J. Manajamen Inform. Jayakarta, vol. 2, no. 3, p. 235, 2022, doi: 10.52362/jmijayakarta.v2i3.834. DOI: https://doi.org/10.52362/jmijayakarta.v2i3.834

[8] D. A. Prasetya, A. P. Sari, P. A. Riyantoko, and T. M. Fahrudin, “The Effect of Information Quality and Service Quality on User Satisfaction of the Government of Kabupaten Malang,” TIERS Inf. Technol. J., vol. 4, no. 1, pp. 32–42, 2023, doi: 10.38043/tiers.v4i1.4328. DOI: https://doi.org/10.38043/tiers.v4i1.4328

[9] T. M. Fahrudin, P. A. Riyantoko, K. M. Hindrayani, and E. M. Safitri, “An Introduction To Machine Learning Games And Its Application For Kids In Fun Project,” Int. J. Comput. Netw. Secur. Inf. Syst., vol. 2, no. 1, pp. 26–30, 2020, [Online]. Available: https://machinelearningforkids.co.uk

[10] M. Idhom, D. A. Prasetya, P. A. Riyantoko, T. M. Fahrudin, and A. P. Sari, “Pneumonia Classification Utilizing VGG-16 Architecture and Convolutional Neural Network Algorithm for Imbalanced Datasets,” TIERS Inf. Technol. J., vol. 4, no. 1, pp. 73–82, 2023, doi: 10.38043/tiers.v4i1.4380. DOI: https://doi.org/10.38043/tiers.v4i1.4380

[11] S. Shania, M. Farid Naufal, V. Riandaru Prasetyo, and M. S. Bin Azmi, “Translator of Indonesian Sign Language Video using Convolutional Neural Network with Transfer Learning,” Indones. J. Inf. Syst., vol. 5, no. 1, pp. 17–27, 2022, doi: 10.24002/ijis.v5i1.5865. DOI: https://doi.org/10.24002/ijis.v5i1.5865

[12] I. D. A. Rachmawati, R. Yunanda, M. F. Hidayat, and P. Wicaksono, “Deep Transfer Learning for Sign Language Image Classification: A Bisindo Dataset Study,” Eng. Math. Comput. Sci. J., vol. 5, no. 3, pp. 175–180, 2023, doi: 10.21512/emacsjournal.v5i3.10621. DOI: https://doi.org/10.21512/emacsjournal.v5i3.10621

[13] M. Sari and E. R. Jamzuri, “Hand Sign Recognition of Indonesian Sign Language System ( SIBI ) Using,” vol. 5, no. 158, pp. 258–265, 2026. DOI: https://doi.org/10.29207/resti.v9i2.6156

[14] F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,” Proc. IEEE Conf. Comput. Vis. pattern Recognit., vol. 7, no. 3, pp. 1251–1258, 2017, doi: https://doi.org/10.48550/arXiv.1610.02357.

[15] A. Noer, “Bahasa Isyarat Indonesia (BISINDO) Alphabets.” [Online]. Available: https://www.kaggle.com/datasets/achmadnoer/alfabet-bisindo

[16] A. Muhaimin, W. Wibowo, and P. A. Riyantoko, “Multi-label Classification Using Vector Generalized Additive Model via Cross-Validation,” J. Inf. Commun. Technol., vol. 22, no. 4, pp. 657–673, 2023, doi: https://doi.org/10.32890/jict2023.22.4.5. DOI: https://doi.org/10.32890/jict2023.22.4.5

[17] M. Vira, “Abjad Bahasa Isyarat Indonesia (BISINDO),” Kaggle. [Online]. Available: https://www.kaggle.com/datasets/meisyavira/abjad-bahasa-isyarat-indonesia-bisindo

[18] R. Balestriero, I. Misra, and Y. LeCun, “A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments,” 2022, [Online]. Available: http://arxiv.org/abs/2202.08325

[19] S. Yang, W. Xiao, M. Zhang, S. Guo, J. Zhao, and F. Shen, “Image Data Augmentation for Deep Learning: A Survey,” 2022, doi: 10.13328/j.cnki.jos.007263.

[20] D. A. Prasetya, A. P. Sari, M. Idhom, and A. Lisanthoni, “Optimizing Clustering Analysis to Identify High-Potential Markets for Indonesian Tuber Exports,” Indones. J. Electron. Electromed. Eng. Med. Informatics, vol. 7, no. 1, pp. 113–122, 2025. DOI: https://doi.org/10.35882/skzqbd57

[21] R. Tuntun, K. Kusrini, and K. Kusnawi, “Analisis Perbandingan Kinerja Algoritma Klasifikasi dengan Menggunakan Metode K-Fold Cross Validation,” J. Media Inform. Budidarma, vol. 6, no. 4, p. 2111, 2022, doi: 10.30865/mib.v6i4.4681. DOI: https://doi.org/10.30865/mib.v6i4.4681

[22] J. Xu, Y. Zhang, and D. Miao, “Three-way confusion matrix for classification: A measure driven view,” Inf. Sci. (Ny)., vol. 507, pp. 772–794, 2020, doi: 10.1016/j.ins.2019.06.064. DOI: https://doi.org/10.1016/j.ins.2019.06.064

[23] C. Tejada, G. Espinoza, and D. Subauste, “Use of Xception Architecture for the Classification of Skin Lesions,” J. Syst. Cybern. Informatics, vol. 22, no. 3, pp. 20–25, 2024, doi: https://doi.org/10.54808/JSCI.22.03.20. DOI: https://doi.org/10.54808/JSCI.22.03.20