RTMDet-AS: an improved real-time instance segmentation model for tomato harvesting based on attention mechanism and SIoU loss

Abstract

Automated robot harvesting is becoming more and more important for solving labor shortages and improving productivity in modern agriculture. However, due to the complex background, dense fruit clusters, and limited computing resources of edge devices, achieving reliable real-time visual perception in unstructured greenhouse environments remains a daunting challenge. This paper presents RTMDet-AS, a robust instance segmentation network tailored to address the challenges of cluttered backgrounds, dense fruit clusters, and the strict efficiencyaccuracy trade-off required for edge computing in unstructured greenhouses. Our architecture is based on the enhanced channel attention (ECA) network with the dual-pooling mechanism to make the feature responses adaptive, to suppress the irrelevant background information, and enhance the texture of fruits. SCYLLA-IoU (SIoU) loss was also introduced to replace Generalized IoU (GIoU) loss, because it uses angle-aware penalties to make regression oscillation in dense clusters confined and accelerate convergence. Furthermore, online data augmentation is introduced to help the model transfer knowledge from a static dataset to a dynamic harvesting task and bridge the domain gap. Moreover, experiments show that RTMDet-AS has Mask AP of 65.9% and Box AP of 67.3%, outperforming the industrial models’ standard YOLOv5s-seg by 2.9% and 1.2% respectively. With a 96.7 FPS inference rate and fewer computational efforts, the proposed model is a good model for real-time perception in agricultural robotics.