Performance Evaluation of AdamW, RMSProp, and Nadam Optimizers on EfficientNetB2 Model for Image Data Classification

Fanita Damayanti; Sugiyarto  Surono; Aris  Thobirin

doi:10.59395/ijadis.v7i1.1482

Authors

Fanita Damayanti Mathematics, Universitas Ahmad Dahlan, Indonesia
Sugiyarto Surono Mathematics, Universitas Ahmad Dahlan, Indonesia
Aris Thobirin Mathematics, Universitas Ahmad Dahlan, Indonesia

DOI:

https://doi.org/10.59395/ijadis.v7i1.1482

Keywords:

EfficientNetB2 , Optimizer , RMSprop , Histopathological Image , Classification

Abstract

This study examines the effect of different optimization algorithms on the performance of the EfficientNetB2 model in classifying lung and colon histopathology images. Three commonly used optimizers AdamW, RMSprop, and Nadam were analyzed to compare their influence on convergence trends, classification accuracy, and overall learning consistency. Using a five-class dataset covering benign and malignant tissue samples, the experimental results show that all three optimizers are able to deliver reliable predictions, although with varying performance characteristics. RMSprop emerges as the most effective optimizer, achieving the highest accuracy across all evaluation stages, with 99.05% during training, 99.16% on validation, and 98.72% on testing, along with the lowest loss values. This indicates that RMSprop facilitates faster and more stable convergence compared to the other two methods. AdamW also demonstrates strong predictive performance but shows limitations when distinguishing cancer types with closely similar morphological structures. Nadam attains high accuracy in early stages yet exhibits lower initial stability than RMSprop. Overall, pairing EfficientNetB2 with RMSprop provides the most optimal configuration for this classification task. These results offer valuable insights for designing better training strategies and strengthening the effectiveness of medical imaging based computer aided diagnostic systems.

1288 633

Downloads

Download data is not yet available.

References

[1] Y. Wu et al., “Recent Advances of Deep Learning for Computational Histopathology: Principles and Applications,” Mar. 01, 2022, MDPI. doi: 10.3390/cancers14051199. DOI: https://doi.org/10.3390/cancers14051199

[2] I. D. Mienye, T. G. Swart, G. Obaido, M. Jordan, and P. Ilono, “Deep Convolutional Neural Networks in Medical Image Analysis: A Review,” Mar. 01, 2025, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/info16030195. DOI: https://doi.org/10.20944/preprints202408.1288.v1

[3] W. Yue, S. Liu, and Y. Li, “Eff-PCNet: An Efficient Pure CNN Network for Medical Image Classification,” Applied Sciences (Switzerland), vol. 13, no. 16, Aug. 2023, doi: 10.3390/app13169226. DOI: https://doi.org/10.3390/app13169226

[4] K. Khotimah, S. Surono, and A. Thobirin, “Optimizing EfficientNet for imbalanced medical image classification using grey wolf optimization,” Computer Science and Information Technologies, vol. 6, no. 2, pp. 112–121, 2025, doi: 10.11591/csit.v6i2.pp112-121. DOI: https://doi.org/10.11591/csit.v6i2.p112-121

[5] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Sep. 2020, [Online]. Available: http://arxiv.org/abs/1905.11946

[6] M. W. Purbandanu, R. Yanuarta, and A. Kurniawan, “Optimization of Skin Cancer Detection to Improve Accuracy with the Application of Efficient Convolutional Neural Network and EfficientNetB2 Models,” Journal of Intelligent Computing & Health Informatics, vol. 5, no. 2, Apr. 2024, doi: 10.26714/jichi.v5i2.14338. DOI: https://doi.org/10.26714/jichi.v5i2.14338

[7] M. Latha, P. S. Kumar, R. R. Chandrika, T. R. Mahesh, V. V. Kumar, and S. Guluwadi, “Revolutionizing breast ultrasound diagnostics with EfficientNet-B7 and Explainable AI,” BMC Med Imaging, vol. 24, no. 1, Dec. 2024, doi: 10.1186/s12880-024-01404-3. DOI: https://doi.org/10.1186/s12880-024-01404-3

[8] K. Natarajan et al., “A novel method for the detection and classification of multiple diseases using transfer learning-based deep learning techniques with improved performance,” Neural Comput Appl, vol. 36, no. 30, pp. 18979–18997, Oct. 2024, doi: 10.1007/s00521-024-09900-x. DOI: https://doi.org/10.1007/s00521-024-09900-x

[9] M. Alruwaili and M. Mohamed, “An Integrated Deep Learning Model with EfficientNet and ResNet for Accurate Multi-Class Skin Disease Classification,” Diagnostics, vol. 15, no. 5, Mar. 2025, doi: 10.3390/diagnostics15050551. DOI: https://doi.org/10.3390/diagnostics15050551

[10] H. Sun et al., “An Improved Medical Image Classification Algorithm Based on Adam Optimizer,” Mathematics, vol. 12, no. 16, Aug. 2024, doi: 10.3390/math12162509. DOI: https://doi.org/10.3390/math12162509

[11] A. M. J. M. D. Zubair Rahman, R. Mythili, K. Chokkanathan, T. R. Mahesh, K. Vanitha, and T. E. Yimer, “Enhancing image-based diagnosis of gastrointestinal tract diseases through deep learning with EfficientNet and advanced data augmentation techniques,” BMC Med Imaging, vol. 24, no. 1, Dec. 2024, doi: 10.1186/s12880-024-01479-y. DOI: https://doi.org/10.1186/s12880-024-01479-y

[12] G. Latif, J. Alghazo, M. A. Khan, G. Ben Brahim, K. Fawagreh, and N. Mohammad, “Deep convolutional neural network (CNN) model optimization techniques—Review for medical imaging,” 2024, American Institute of Mathematical Sciences. doi: 10.3934/math.2024998. DOI: https://doi.org/10.3934/math.2024998

[13] X. Wang and L. Aitchison, “How to set AdamW’s weight decay as you scale model and dataset size,” Jun. 2025, [Online]. Available: http://arxiv.org/abs/2405.13698

[14] N. Shi, D. Li, M. Hong, and R. Sun, “RMSPROP CONVERGES WITH PROPER HYPER-PARAMETER.”

[15] S. Mulla and N. F. Shaikh, “Over comparative study of text summarization techniques based on graph neural networks,” Web Intelligence, vol. 22, no. 2, pp. 231–248, Apr. 2024, doi: 10.3233/WEB-230014. DOI: https://doi.org/10.3233/WEB-230014

[16] Z. Zhuang, M. Liu, A. Cutkosky, and F. Orabona, “Understanding AdamW through Proximal Methods and Scale-Freeness,” Jan. 2022, [Online]. Available: http://arxiv.org/abs/2202.00089

[17] Q. Zhang, Y. Zhou, and S. Zou, “Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance,” Mar. 2025, [Online]. Available: http://arxiv.org/abs/2404.01436

[18] R. Abdulkadirov, P. Lyakhov, and N. Nagornov, “Survey of Optimization Algorithms in Modern Neural Networks,” Jun. 01, 2023, MDPI. doi: 10.3390/math11112466.

[19] S. Anjum et al., “Lung Cancer Classification in Histopathology Images Using Multiresolution Efficient Nets,” Comput Intell Neurosci, vol. 2023, no. 1, Jan. 2023, doi: 10.1155/2023/7282944. DOI: https://doi.org/10.1155/2023/7282944

[20] N. Guetta, A. Shabtai, I. Singh, S. Momiyama, and Y. Elovici, “Dodging Attack Using Carefully Crafted Natural Makeup,” Sep. 2021, [Online]. Available: http://arxiv.org/abs/2109.06467

[21] H. Naveed, S. Anwar, M. Hayat, K. Javed, and A. Mian, “Survey: Image Mixing and Deleting for Data Augmentation,” Feb. 2023, [Online]. Available: http://arxiv.org/abs/2106.07085 DOI: https://doi.org/10.1016/j.engappai.2023.107791

[22] L. Alzubaidi et al., “Review of deep learning: concepts, CNN architectures, challenges, applications, future directions,” J Big Data, vol. 8, no. 1, Dec. 2021, doi: 10.1186/s40537-021-00444-8. DOI: https://doi.org/10.1186/s40537-021-00444-8

[23] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Sep. 2020, [Online]. Available: http://arxiv.org/abs/1905.11946

[24] M. Tan and Q. V. Le, “EfficientNetV2: Smaller Models and Faster Training,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2104.00298

[25] J. Su, S. Anderson, and L. Mihaylova, “Holistic Self-Distillation with the Squeeze and Excitation Network for Fine-grained Plant Pathology Classification,” in 2023 26th International Conference on Information Fusion, FUSION 2023, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.23919/FUSION52260.2023.10224184. DOI: https://doi.org/10.23919/FUSION52260.2023.10224184

[26] L. Deng, H. Suo, and D. Li, “Deepfake Video Detection Based on EfficientNet-V2 Network,” Comput Intell Neurosci, vol. 2022, 2022, doi: 10.1155/2022/3441549. DOI: https://doi.org/10.1155/2022/3441549

[27] H. Daneshmand, A. Joudaki, E. Zurich, and F. Bach, “Batch Normalization Orthogonalizes Representations in Deep Random Networks.” [Online]. Available: https://github.com/hadidaneshmand/batchnorm21.git

[28] S. Rajaraman, G. Zamzmi, and S. K. Antani, “Novel loss functions for ensemble-based medical image classification,” PLoS One, vol. 16, no. 12 December, Dec. 2021, doi: 10.1371/journal.pone.0261307. DOI: https://doi.org/10.1371/journal.pone.0261307

[29] P. Zhou, X. Xie, Z. Lin, K.-C. Toh, and S. Yan, “Win: Weight-Decay-Integrated Nesterov Acceleration for Faster Network Training,” 2024. [Online]. Available: https://github.com/sail-sg/win.

[30] R. Abdulkadirov, P. Lyakhov, and N. Nagornov, “Survey of Optimization Algorithms in Modern Neural Networks,” Jun. 01, 2023, MDPI. doi: 10.3390/math11112466. DOI: https://doi.org/10.20944/preprints202304.0648.v1

[31] J. Tu, A. Yasin, and N. S. Mansor, “NadamClip: A Novel Optimization Algorithm for Improving Prediction Accuracy and Training Stability,” Processes, vol. 13, no. 7, Jul. 2025, doi: 10.3390/pr13072145. DOI: https://doi.org/10.3390/pr13072145

[32] S. Farhadpour, T. A. Warner, and A. E. Maxwell, “Selecting and Interpreting Multiclass Loss and Accuracy Assessment Metrics for Classifications with Class Imbalance: Guidance and Best Practices,” Remote Sens (Basel), vol. 16, no. 3, Feb. 2024, doi: 10.3390/rs16030533. DOI: https://doi.org/10.3390/rs16030533

[33] N. Widya Utami, M. A. P. Putra, I. G. J. E. Putra, and G. A. Sampedro, “A CNN-Based Information System for Balinese Dance Classification with Hyperparameter Optimization,” International Journal of Advances in Data and Information Systems, vol. 6, no. 2, pp. 376–390, Aug. 2025, doi: 10.59395/ijadis.v6i2.1402. DOI: https://doi.org/10.59395/ijadis.v6i2.1402