HMSMT: Hierarchical Multi-Stage Multi-Task Deep Learning for Plant Leaf Disease Diagnosis

Abstract

Plant leaf diseases represent a major threat to agricultural productivity, requiring accurate and robust automated diagnosis methods for precision farming. Existing deep learning approaches are often limited by single-task designs, insufficient lesion localization capabilities, and severity estimation mechanisms that are not effectively integrated with disease classification. To address these challenges, this paper proposes a Hierarchical Multi-Stage Multi-Task (HMSMT) deep learning framework that jointly performs disease classification, pixel-level lesion segmentation, and disease severity estimation within a unified architecture. The proposed framework employs an EfficientNet-B3 backbone and a dual-branch feature fusion architecture to integrate complementary semantic and spatial disease representations. A fusion-enhanced segmentation head is employed for pixel-level lesion localization, while a dedicated non-linear regression head estimates disease severity from the shared fused representation, allowing severity prediction to benefit from both semantic disease information and spatial representations. All task-specific objectives are jointly optimized using a multi-task learning strategy. Experiments conducted on selected and label-harmonized subsets of the PlantVillage and PlantDoc datasets achieve classification accuracies of 98.40% and 94.95%, respectively, with lesion segmentation Dice scores of 99.21% and 97.84%, and severity estimation MAE values of 2.41% and 2.81%, respectively. Ablation experiments demonstrate the contribution of the dual-branch fusion architecture and show that the proposed non-linear severity regression head outperforms its linear counterpart. Furthermore, evaluation on the controlled PlantVillage and real-field PlantDoc datasets demonstrates the ability of the framework to maintain strong performance under domain shift. Repeated-run experiments further show relatively stable performance across different random initializations, providing additional evidence of the reliability of the reported results. These findings highlight the potential of the proposed HMSMT framework for integrated and reliable plant disease diagnosis under both controlled and challenging real-world conditions.