Hybrid CNN and LLM for Image-Based Classification of Plant Leaf Diseases
DOI:
https://doi.org/10.59395/ijadis.v7i2.1559Keywords:
EfficientNetV2-M, GPT-4o, PlantVillage Dataset, Hybrid CNN and LLM, Plant Disease ClassificationAbstract
Plant diseases have continued to threaten agricultural productivity, while manual inspection methods have remained inefficient and prone to subjectivity. This study proposed and assessed a hybrid framework integrating a Convolutional Neural Network (CNN) with a Large Language Model (LLM) to perform image-based plant leaf disease classification accompanied by interpretable diagnostic explanations. EfficientNetV2-M was employed as the visual backbone and trained on 11 selected classes of apple, grape, and potato leaf images derived from the PlantVillage dataset. A structured data splitting strategy was applied to ensure reliable model validation and unbiased testing. The classification capability of the CNN component was examined through standard multi-class evaluation indicators, including class-wise predictive consistency and error distribution analysis. Experimental results indicated that the model delivered highly consistent predictions, reaching a peak test accuracy of 99.79%, reflecting its robustness in distinguishing visually similar disease patterns. To overcome the black-box limitation, prediction outputs were transformed into structured prompts and processed by GPT-4o to generate contextual explanations. The generated narratives systematically described observable symptoms, highlighted distinguishing characteristics, and suggested initial management actions. Overall, the proposed hybrid system demonstrated that combining high-performance visual recognition with language-based reasoning enhanced both diagnostic reliability and interpretability in digital agriculture applications.
Downloads
References
[1] C. Pal, S. Karmakar, I. Mukherjee, and P. P. Chakrabarti, A lightweight and explainable CNN model for empowering plant disease diagnosis, Sci. Rep., vol. 15, no. 1, pp. 114, 2025, doi: 10.1038/s41598-025-94083-1. DOI: https://doi.org/10.1038/s41598-025-94083-1
[2] D. Shah, V. Trivedi, V. Sheth, A. Shah, and U. Chauhan, ResTS: Residual Deep interpretable architecture for plant disease detection, Inf. Process. Agric., vol. 9, no. 2, pp. 212223, 2022, doi: 10.1016/j.inpa.2021.06.001. DOI: https://doi.org/10.1016/j.inpa.2021.06.001
[3] J. Lu, L. Tan, and H. Jiang, Review on convolutional neural network (CNN) applied to plant leaf disease classification, Agric., vol. 11, no. 8, pp. 118, 2021, doi: 10.3390/agriculture11080707. DOI: https://doi.org/10.3390/agriculture11080707
[4] J. Chen, J. Chen, D. Zhang, Y. Sun, and Y. A. Nanehkaran, Using deep transfer learning for image-based plant disease identi fi cation, Comput. Electron. Agric., vol. 173, no. November 2019, p. 105393, 2020, doi: 10.1016/j.compag.2020.105393. DOI: https://doi.org/10.1016/j.compag.2020.105393
[5] I. Pacal et al., A systematic review of deep learning techniques for plant diseases, Artif. Intell. Rev., vol. 57, no. 11, p. 304, 2024, doi: 10.1007/s10462-024-10944-7. DOI: https://doi.org/10.1007/s10462-024-10944-7
[6] K. Hong, Y. Zhou, and H. Han, The pipelines of deep learning-based plant image processing, Quant. Plant Biol., vol. 6, p. e23, 2025, doi: 10.1017/qpb.2025.10018. DOI: https://doi.org/10.1017/qpb.2025.10018
[7] T. D. Salka, M. B. Hanafi, S. M. S. A. A. Rahman, D. B. M. Zulperi, and Z. Omar, Plant leaf disease detection and classification using convolution neural networks model: a review, Artif. Intell. Rev., vol. 58, no. 10, 2025, doi: 10.1007/s10462-025-11234-6. DOI: https://doi.org/10.1007/s10462-025-11234-6
[8] J. Xiong, D. Yu, S. Liu, L. Shu, X. Wang, and Z. Liu, A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning, pp. 119, 2021, doi: 10.3390/electronics10010081. DOI: https://doi.org/10.3390/electronics10010081
[9] W. Shafik, A. Tufail, C. De Silva Liyanage, and R. A. A. H. M. Apong, Using transfer learning-based plant disease classification and detection for sustainable agriculture, BMC Plant Biol., vol. 24, no. 1, pp. 119, 2024, doi: 10.1186/s12870-024-04825-y. DOI: https://doi.org/10.1186/s12870-024-04825-y
[10] J. Pan et al., ChatLeafDisease: a chain-of-thought prompting approach for crop disease classification using large language models, Plant Phenomics, vol. 7, no. 3, p. 100094, 2025, doi: https://doi.org/10.1016/j.plaphe.2025.100094. DOI: https://doi.org/10.1016/j.plaphe.2025.100094
[11] E. Amri, Y. Gulzar, A. Yeafi, S. Jendoubi, F. Dhawi, and M. S. Mir, Advancing automatic plant classification system in Saudi Arabia: introducing a novel dataset and ensemble deep learning approach, Model. Earth Syst. Environ., vol. 10, no. 2, pp. 26932709, 2024, doi: 10.1007/s40808-023-01918-9. DOI: https://doi.org/10.1007/s40808-023-01918-9
[12] H. K. Kondaveeti and C. G. Simhadri, Evaluation of deep learning models using explainable AI with qualitative and quantitative analysis for rice leaf disease detection, Sci. Rep., vol. 15, no. 1, p. 31850, 2025, doi: 10.1038/s41598-025-14306-3. DOI: https://doi.org/10.1038/s41598-025-14306-3
[13] V. Hassija et al., Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence, Cognit. Comput., vol. 16, no. 1, pp. 4574, 2024, doi: 10.1007/s12559-023-10179-8. DOI: https://doi.org/10.1007/s12559-023-10179-8
[14] Y. Wang et al., A large language model for multimodal identification of crop diseases and pests, Sci. Rep., vol. 15, no. 1, 2025, doi: 10.1038/s41598-025-01908-0. DOI: https://doi.org/10.1038/s41598-025-01908-0
[15] H. Li, H. Wu, Q. Li, and C. Zhao, A review on enhancing agricultural intelligence with large language models, Artif. Intell. Agric., vol. 15, no. 4, pp. 671685, 2025, doi: https://doi.org/10.1016/j.aiia.2025.05.006. DOI: https://doi.org/10.1016/j.aiia.2025.05.006
[16] Y. Arun and G. S. Viknesh, Leaf Classification for Plant Recognition Using EfficientNet Architecture, Proc. IEEE 2022 4th Int. Conf. Adv. Electron. Comput. Commun. ICAECC 2022, vol. 10, no. 11, pp. 6872, 2022, doi: 10.1109/ICAECC54045.2022.9716637. DOI: https://doi.org/10.1109/ICAECC54045.2022.9716637
[17] J. Li et al., Foundation models in smart agriculture: Basics, opportunities, and challenges, Comput. Electron. Agric., vol. 222, 2024, doi: 10.1016/j.compag.2024.109032. DOI: https://doi.org/10.1016/j.compag.2024.109032
[18] M. Y. Shams, S. A. Gamel, and F. M. Talaat, Enhancing crop recommendation systems with explainable artificial intelligence: a study on agricultural decision-making, Neural Comput. Appl., vol. 36, no. 11, pp. 56955714, 2024, doi: 10.1007/s00521-023-09391-2. DOI: https://doi.org/10.1007/s00521-023-09391-2
[19] D. K. Saha, M. R. Ahmed, T. D. Nath, R. I. Boby, M. J. Hossen, and M. F. Mridha, Fusing explainable deep learning ensembles and LLM recommendations for real-time plant leaf disease diagnosis, Intell. Syst. with Appl., vol. 28, p. 200596, 2025, doi: https://doi.org/10.1016/j.iswa.2025.200596. DOI: https://doi.org/10.1016/j.iswa.2025.200596
[20] S. P. Mohanty, D. P. Hughes, and M. Salath, Using deep learning for image-based plant disease detection, Front. Plant Sci., vol. 7, no. September, pp. 110, 2016, doi: 10.3389/fpls.2016.01419. DOI: https://doi.org/10.3389/fpls.2016.01419
[21] S. Murugesan, J. Chinnadurai, S. Srinivasan, S. K. Mathivanan, R. R. Chandan, and U. Moorthy, Robust multiclass classification of crop leaf diseases using hybrid deep learning and Grad-CAM interpretability, Sci. Rep., vol. 15, no. 1, p. 29955, 2025, doi: 10.1038/s41598-025-14847-7. DOI: https://doi.org/10.1038/s41598-025-14847-7
[22] J. H. Sinamenye, A. Chatterjee, and R. Shrestha, Potato plant disease detection: leveraging hybrid deep learning models, BMC Plant Biol., vol. 25, no. 1, p. 647, 2025, doi: 10.1186/s12870-025-06679-4. DOI: https://doi.org/10.1186/s12870-025-06679-4
[23] S. Lee and S. Lee, Efficient Data Augmentation Methods for Crop Disease Recognition in Sustainable Environmental Systems, Big Data Cogn. Comput., vol. 9, no. 1, 2025, doi: 10.3390/bdcc9010008. DOI: https://doi.org/10.3390/bdcc9010008
[24] S. Timilsina, S. Sharma, and S. Kondo, Advancements in maize leaf disease detection, segmentation and classification: A review, Biosyst. Eng., vol. 255, p. 104162, 2025, doi: https://doi.org/10.1016/j.biosystemseng.2025.104162. DOI: https://doi.org/10.1016/j.biosystemseng.2025.104162
[25] M. Tan and Q. Le, EfficientNetV2: Smaller Models and Faster Training, in Proceedings of the 38th International Conference on Machine Learning, M. Meila and T. Zhang, Eds., in Proceedings of Machine Learning Research, vol. 139. PMLR, 2021, pp. 1009610106. [Online]. Available: https://proceedings.mlr.press/v139/tan21a.html
[26] S. Kumar et al., Spatial attention-guided pre-trained networks for accurate identification of crop diseases, Sci. Rep., vol. 15, no. 1, pp. 129, 2025, doi: 10.1038/s41598-025-08004-3. DOI: https://doi.org/10.1038/s41598-025-08004-3
[27] S. Fuji, B. Nugroho, and I. G. Susrama, Implementation Of Hybrid EfficientNet V2 And Vision Transformer for Apple Leaf Diseases Classification, vol. 3, no. 1, pp. 111, 2025, doi: doi.org/10.32985/ijeces.16.8.5. DOI: https://doi.org/10.33005/itij.v3i1.42
[28] R. S. S. Devi, V. R. V. Kumar, and P. Sivakumar, EfficientNetV2 Model for Plant Disease Classification and Pest Recognition, 2023. doi: 10.32604/csse.2023.032231. DOI: https://doi.org/10.32604/csse.2023.032231
[29] P. Ramachandran, B. Zoph, and Q. V Le, Searching for Activation Functions, ArXiv, vol. abs/1710.0, 2018, [Online]. Available: https://api.semanticscholar.org/CorpusID:10919244
[30] C.-C. Lee, H.-H. (Hank) Chau, H.-L. Wang, Y.-F. Chuang, and Y. Chau, Mild cognitive impairment prediction based on multi-stream convolutional neural networks, BMC Bioinformatics, vol. 22, no. 5, p. 638, 2024, doi: 10.1186/s12859-024-05911-6. DOI: https://doi.org/10.1186/s12859-024-05911-6
[31] G. Tariku, I. Ghiglieno, G. Gilioli, F. Gentilin, S. Armiraglio, and I. Serina, Automated Identification and Classification of Plant Species in Heterogeneous Plant Areas Using Unmanned Aerial Vehicle-Collected RGB Images and Transfer Learning, Drones, vol. 7, no. 10, 2023, doi: 10.3390/drones7100599. DOI: https://doi.org/10.3390/drones7100599
[32] J. F. Hernndez Santa Cruz, An ensemble approach for multi-stage transfer learning models for COVID-19 detection from chest CT scans, Intell. Med., vol. 5, p. 100027, 2021, doi: https://doi.org/10.1016/j.ibmed.2021.100027. DOI: https://doi.org/10.1016/j.ibmed.2021.100027
[33] L. Yang, Theoretical Analysis of Adam Optimizer in the Presence of Gradient Skewness, Int. J. Appl. Sci., vol. 7, no. 2 SE-Articles, p. p27, 2024, doi: 10.30560/ijas.v7n2p27. DOI: https://doi.org/10.30560/ijas.v7n2p27
[34] M. N. Razali, N. Arbaiy, P.-C. Lin, and S. Ismail, Optimizing Multiclass Classification Using Convolutional Neural Networks with Class Weights and Early Stopping for Imbalanced Datasets, Electronics, vol. 14, no. 4, 2025, doi: 10.3390/electronics14040705. DOI: https://doi.org/10.3390/electronics14040705
[35] W. Liu, G. Wu, H. Wang, and F. Ren, Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition, Sensors, vol. 25, no. 13, 2025, doi: 10.3390/s25134096. DOI: https://doi.org/10.3390/s25134096
[36] T. Savage, A. Nayak, R. Gallo, E. Rangan, and J. H. Chen, Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine, npj Digit. Med., vol. 7, no. 1, p. 20, 2024, doi: 10.1038/s41746-024-01010-1. DOI: https://doi.org/10.1038/s41746-024-01010-1
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frenky Riski Gilang Pratama, Sugiarto Surono, Aris Thobirin
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Cite
Share
Plum Analytics