Enhancing Chili Leaf Disease Classification Using Multimodal Visual-Thermal Image Fusion

Abstract

Chili peppers are an important horticultural crop, and their productivity can be severely affected by leaf diseases, thereby reducing harvest quality and economic value. Deep learning-based image classification, particularly using Convolutional Neural Networks (CNNs), has demonstrated great potential for detecting plant diseases. However, most existing studies rely on RGB images, which are limited to visually apparent symptoms and are susceptible to variations in lighting, imaging angles, and visual similarities between disease classes. To address these limitations, this study introduces a supervised RGB-thermal multimodal fusion workflow for classifying chili leaf diseases. The proposed workflow first evaluates three baseline CNN architectures EfficientNet-B0, ResNet50, and DenseNet121 using RGB images to determine the most suitable base model. Based on the evaluation results, DenseNet121 was selected as the backbone for the proposed multimodal model. The model employs a dual-branch DenseNet121 architecture to extract RGB and thermal features separately, which are then integrated via feature-level concatenation for final classification. This study utilizes a dataset comprising 280 paired, labeled chili leaf samples across four classes: leaf spot, leaf curl, yellowing, and healthy. The results show that the supervised RGB-only model achieved an accuracy of 0.9416, while the proposed multimodal RGB-thermal fusion model improved the accuracy to 0.9583. These findings indicate that thermal images provide complementary information to RGB images and can improve chili leaf disease classification performance.