Multi-Output Classification of Cognitive Levels and Topics in Indonesian Questions using Deep Learning and Transformers

Orvalamarva Orvalamarva; Oktariani Nurul  Pratiwi; Hanif  Fakhrurroja

doi:10.59395/ijadis.v7i1.1492

Authors

Orvalamarva Orvalamarva Information System Study Program, School of Industrial Engineering, Telkom University, Bandung, Indonesia
Oktariani Nurul Pratiwi Information System Study Program, School of Industrial Engineering, Telkom University, Bandung, Indonesia
Hanif Fakhrurroja Information System Study Program, School of Industrial Engineering, Telkom University, Bandung, Indonesia

DOI:

https://doi.org/10.59395/ijadis.v7i1.1492

Keywords:

Educational assessment , IndoBERT , Multi-output text classification , Revised Bloom’s Taxonomy

Abstract

Managing large-scale digital question banks struggles with manual metadata labeling, especially when identifying material topics and cognitive levels based on the Revised Bloom's Taxonomy. Current automated approaches usually treat these two attributes as separate tasks, which adds to the system's complexity and computational load. This study introduces a multi-output classification method using a shared encoder architecture with two task-specific heads to predict topics and cognitive levels simultaneously. We performed experiments on 685 Indonesian junior high science questions, covering 15 topic labels and four cognitive levels (C1–C4), with an imbalanced distribution in which lower cognitive levels accounted for more than 75% of the dataset. To handle this imbalance, we applied Focal Loss to taxonomy classification, and class weighting was used in the comparison model. A comparative study involved CNN, BiLSTM, DistilBERT, and IndoBERT. Our results demonstrate that IndoBERT delivered the best performance, with F1-macro scores of 0.78 for topics and 0.71 for cognitive levels and showed better performance in minority classes compared to standard cross-entropy-based models. These findings suggest that an integrated multi-output approach can boost the efficiency and accuracy of question labeling and offers potential for integration into Computer-Based Test systems and e-assessment platforms in real time.

1068 525

Downloads

Download data is not yet available.

References

[1] I. Gligorea, M. Cioca, R. Oancea, A. T. Gorski, H. Gorski, and P. Tudorache, Adaptive Learning Using Artificial Intelligence in e-Learning: A Literature Review, Dec. 01, 2023, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/educsci13121216. DOI: https://doi.org/10.3390/educsci13121216

[2] I. Papadogiannis, M. Wallace, and G. Karountzou, Educational Data Mining: A Foundational Overview, Encyclopedia, vol. 4, no. 4, pp. 16441664, Dec. 2024, doi: 10.3390/encyclopedia4040108. DOI: https://doi.org/10.3390/encyclopedia4040108

[3] A. Lubbe, E. Marais, and D. Kruger, Cultivating independent thinkers: The triad of artificial intelligence, Blooms taxonomy and critical thinking in assessment pedagogy, Educ Inf Technol (Dordr), vol. 30, no. 12, pp. 1758917622, Aug. 2025, doi: 10.1007/s10639-025-13476-x. DOI: https://doi.org/10.1007/s10639-025-13476-x

[4] M. Pota, M. Esposito, G. De Pietro, and H. Fujita, Best practices of convolutional neural networks for question classification, Applied Sciences (Switzerland), vol. 10, no. 14, Jul. 2020, doi: 10.3390/app10144710. DOI: https://doi.org/10.3390/app10144710

[5] S. Mondal, M. Barman, and A. Nag, Question classification task based on deep learning models with self-attention mechanism, Multimed Tools Appl, 2024, doi: 10.1007/s11042-024-19239-z. DOI: https://doi.org/10.1007/s11042-024-19239-z

[6] M. O. Gani, R. K. Ayyasamy, A. Sangodiah, and Y. T. Fui, Blooms Taxonomy-based exam question classification: The outcome of CNN and optimal pre-trained word embedding technique, Educ Inf Technol (Dordr), vol. 28, no. 12, pp. 1589315914, Dec. 2023, doi: 10.1007/s10639-023-11842-1. DOI: https://doi.org/10.1007/s10639-023-11842-1

[7] C. Eang and S. Lee, Improving the Accuracy and Effectiveness of Text Classification Based on the Integration of the Bert Model and a Recurrent Neural Network (RNN_Bert_Based), Applied Sciences (Switzerland), vol. 14, no. 18, Sep. 2024, doi: 10.3390/app14188388. DOI: https://doi.org/10.3390/app14188388

[8] T. Miao and D. Xu, KWM-B: Key-Information Weighting Methods at Multiple Scale for Automated Essay Scoring with BERT, Electronics (Switzerland), vol. 14, no. 1, Jan. 2025, doi: 10.3390/electronics14010155. DOI: https://doi.org/10.3390/electronics14010155

[9] X. Chen, D. Zou, H. Xie, G. Cheng, Z. Li, and F. Lee Wang, Automatic Classification of Online Learner Reviews Via Fine-Tuned BERTs, International Review of Research in Open and Distributed Learning, vol. 26, 2025. DOI: https://doi.org/10.19173/irrodl.v26i1.8068

[10] Z. Zhang, W. Yu, M. Yu, Z. Guo, and M. Jiang, A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods, in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, May 2023, pp. 943956. DOI: https://doi.org/10.18653/v1/2023.eacl-main.66

[11] M. D. Laddha, V. T. Lokare, A. W. Kiwelekar, and L. D. Netak, Classifications of the summative assessment for revised Blooms Taxonomy by using deep learning, International Journal of Engineering Trends and Technology, vol. 69, no. 3, pp. 211218, Mar. 2021, doi: 10.14445/22315381/IJETT-V69I3P232. DOI: https://doi.org/10.14445/22315381/IJETT-V69I3P232

[12] A. Indrawati, H. Subagyo, A. Sihombing, Wagiyah, and S. Afandi, Analyzing The Impact Of Resampling Method for Imbalanced Data Text In Indonesian Scientific Articles Categorization, BACA: Jurnal Dokumentasi dan Informasi, vol. 42, no. 2, pp. 133141, Dec. 2020, [Online]. Available: https://jurnalbaca.pdii.lipi.go.id/index.php/baca/article/view/563 DOI: https://doi.org/10.14203/j.baca.v41i2.702

[13] C. Schrer, F. Kruse, and J. M. Gmez, A systematic literature review on applying CRISP-DM process model, in Procedia Computer Science, Elsevier B.V., 2021, pp. 526534. doi: 10.1016/j.procs.2021.01.199. DOI: https://doi.org/10.1016/j.procs.2021.01.199

[14] Rousyati, D. Pratmanto, A. Eko Widodo, K. Fatmawati, and R. Diva Saputra, Analisa Sentimen Persepsi Masyarakat Terhadap Aplikasi Bea Cukai Mobile Menggunakan Algoritma Naive Bayes Dan K-Nearest Neighbors, Jurnal Sains dan Manajemen, vol. 12, no. 2, 2024. DOI: https://doi.org/10.31294/evolusi.v12i2.23576

[15] L. W. Anderson and D. R. Krathwohl, A taxonomy for learning, teaching, and assessing: Abridged Edition. Allyn and Bacon, 2001. Accessed: Jan. 03, 2026. [Online]. Available: https://www.jsu.edu/online/faculty/blooms-revised-taxonomy-verb-list.pdf

[16] P. M. Newton, A. Da Silva, and L. G. Peters, A Pragmatic Master List of Action Verbs for Blooms Taxonomy, Front Educ (Lausanne), vol. 5, Jul. 2020, doi: 10.3389/feduc.2020.00107. DOI: https://doi.org/10.3389/feduc.2020.00107

[17] Y. Kim, Convolutional Neural Networks for Sentence Classification, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Oct. 2014, pp. 17461751. [Online]. Available: http://nlp.stanford.edu/sentiment/ DOI: https://doi.org/10.3115/v1/D14-1181

[18] B. Yang, Y. Wang, and Y. Zhan, Lithium Battery State-of-Charge Estimation Based on a Bayesian Optimization Bidirectional Long Short-Term Memory Neural Network, Energies (Basel), vol. 15, no. 13, Jul. 2022, doi: 10.3390/en15134670. DOI: https://doi.org/10.3390/en15134670

[19] A. Vaswani et al., Attention Is All You Need, in 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 2017.

[20] M. D. Maulana and C. S. K. Aditya, Perbandingan IndoBERT dan Bi-LSTM Dalam Mendeteksi Pelanggaran Undang-Undang ITE, Sintech Journal, vol. 8, no. 1, Apr. 2025.

[21] H. M. Ahmad and A. Rahimi, Dynamic Focal Loss for Imbalanced Learning, SN Comput Sci, vol. 6, no. 6, Aug. 2025, doi: 10.1007/s42979-025-04198-5. DOI: https://doi.org/10.1007/s42979-025-04198-5

[22] S. Riyanto, I. S. Sitanggang, T. Djatna, and T. D. Atikah, Comparative Analysis using Various Performance Metrics in Imbalanced Data for Multi-class Text Classification, International Journal of Advanced Computer Science and Applications, vol. 14, no. 6, pp. 10821090, 2023. DOI: https://doi.org/10.14569/IJACSA.2023.01406116

[23] W. Khan, A. Daud, K. Khan, S. Muhammad, and R. Haq, Exploring the frontiers of deep learning and natural language processing: A comprehensive overview of key challenges and emerging trends, Natural Language Processing Journal, vol. 4, p. 100026, Sep. 2023, doi: 10.1016/j.nlp.2023.100026. DOI: https://doi.org/10.1016/j.nlp.2023.100026

[24] R. Katirci and H. elik, Learning Rate Sensitivity In Transformer Models: A Case Study In Neural Machine Translation, in Sivas International Conference On Scientific And Innovation Research-IV, Sivas, May 2025, pp. 719729.

[25] J. Lovn-Melgarejo, L. Soulier, K. Pinel-Sauvagnat, and L. Tamine, Studying Catastrophic Forgetting in Neural Ranking Models, 2021, pp. 375390. doi: 10.1007/978-3-030-72113-8_25. DOI: https://doi.org/10.1007/978-3-030-72113-8_25

[26] B. Hemanth Sai, S. Mukherjee, and S. R. Dubey, Adaptive adam-based optimizers using second-order weight decoupling and gradient-aware weight decay for vision transformer, Mach Vis Appl, vol. 36, no. 3, May 2025, doi: 10.1007/s00138-025-01686-9. DOI: https://doi.org/10.1007/s00138-025-01686-9

[27] F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP, in Proceedings of the 28th International Conference on Computational Linguistics, Barcelona: Online, Dec. 2020, pp. 757770. [Online]. Available: https://huggingface.co/ DOI: https://doi.org/10.18653/v1/2020.coling-main.66

[28] B. Jang, M. Kim, G. Harerimana, S. U. Kang, and J. W. Kim, Bi-LSTM model to increase accuracy in text classification: Combining word2vec CNN and attention mechanism, Applied Sciences (Switzerland), vol. 10, no. 17, Sep. 2020, doi: 10.3390/app10175841. DOI: https://doi.org/10.3390/app10175841

[29] L. Cao, X. Liu, and H. Shen, Adaptable Focal Loss for Imbalanced Text Classification, Springer Nature Switzerland, pp. 466475, 2022, doi: 10.1007/978-3-030-96772-7. DOI: https://doi.org/10.1007/978-3-030-96772-7_43

[30] C. J. Stanny, Reevaluating Blooms Taxonomy: What measurable verbs can and cannot say about student learning, Educ Sci (Basel), vol. 6, no. 4, Dec. 2016, doi: 10.3390/educsci6040037. DOI: https://doi.org/10.3390/educsci6040037

[31] N. Patwardhan, S. Marrone, and C. Sansone, Transformers in the Real World: A Survey on NLP Applications, Apr. 01, 2023, MDPI. doi: 10.3390/info14040242. DOI: https://doi.org/10.3390/info14040242

[32] M. Sharma, A Comparative Analysis of Natural Language Processing Models: BERT and LSTM in Enhancing Business Communication, Int J Innov Sci Res Technol, pp. 13601365, Apr. 2025, doi: 10.38124/ijisrt/25apr943. DOI: https://doi.org/10.38124/ijisrt/25apr943