A Deep Learning Model for Tree Species Instance Segmentation and Classification Using Ground-Level RGB Imagery and Automated Annotations

Abstract

Abstract. Accurate tree species identification is essential for effective forest management, biodiversity monitoring, and resource estimation. While automated methods relying on aerial and canopy-level remote sensing have become prevalent, they often struggle in dense, multi-layered forest stands, where critical lower-stem and bark features are obscured. To address this limitation, we present a Deep Learning (DL) framework for tree species classification utilizing ground-level RGB imagery. Because manual annotation of terrestrial images in forest environments is labor-intensive and complicated by occlusions, we introduce a new “in-situ” forest image dataset alongside an automated labeling pipeline. This pipeline generates training annotations by projecting tree-species data derived from Mobile Laser Scanning (MLS) onto 2D images based on photogrammetric reconstruction. The proposed DL model leverages these automatically labeled images to effectively recognize tree species based on structural and bark characteristics. The model achieves overall F1-scores of 0.78 and 0.75 for object detection and instance segmentation, respectively. Ultimately, our approach complements existing methods for detecting tree positions and diameters, facilitating the creation of a holistic, cost-effective, and scalable forest inventory dataset.