Semi-Supervised Multimodal Individual Tree Crown Delineation via Crown Vertical Semantic Distillation With Selective Coupling

Abstract

Trees play a pivotal role in industrial processes by providing raw materials, such as timber and paper, while simultaneously contributing to sustainable practices through carbon dioxide sequestration and biodiversity support. Accurate monitoring of individual tree is crucial for managing forest resources and assessing environmental health. While unmanned aerial vehicle imagery provides an effective data source for this task, deep learning-based segmentation methods are often hindered by the prohibitive cost of generating the required instance-level annotations, particularly in dense forest scenes. To overcome this limitation, this article introduces a novel semi-supervised framework for individual tree segmentation. Recognizing that RGB imagery alone cannot capture vertical canopy structure, we integrate LiDAR-derived point cloud data through a multimodal fusion strategy that enhances the perceptual capabilities of the model. Specifically, our method leverages a canopy height model (CHM) to complement the rich textural details of RGB images. We propose an adaptive and efficient two-stage fusion architecture featuring a novel crown vertical semantic distillation with selective coupling module to integrate these heterogeneous data sources. Furthermore, we introduce a mask-guided training mechanism for tree crown to resolve the issue of inconsistent prediction of tree crown, which is common in transformer-based segmentation models. Extensive experiments conducted on four diverse datasets validate the effectiveness of the proposed method. The proposed method consistently outperforms existing semi-supervised methods based solely on RGB or CHM, as well as other multimodal semi-supervised instance segmentation methods. These results demonstrate its strong potential for accurate and annotation-efficient tree crown delineation.