Improved YOLOv8n-seg for instance segmentation of mango fruits and peduncles in natural orchard environments

Abstract

To improve the instance segmentation accuracy of mango fruits and peduncles in complex mountainous orchard scenes and provide visual decision support for robotic harvesting, this study proposes a model-driven perception and picking-point localization method. Specifically, an RGB-D mango dataset was constructed under natural orchard conditions, covering strong illumination, shadows, backlighting, fruit overlap, branch and leaf occlusion, and peduncle crossing. An improved lightweight instance segmentation model named SHS-YOLOv8n-seg was then developed based on YOLOv8n-seg. StarNet_s1 was introduced as the backbone to enhance feature extraction under complex backgrounds. Furthermore, a high-frequency and spatial perception feature pyramid network was adopted to strengthen multi-scale feature fusion and improve the representation of slender peduncles. The SPPF module was used to expand the receptive field, and the parameter-free SimAM attention mechanism was introduced to enhance target responses while suppressing background interference. In the single-run comparison, the proposed model achieved Precision, Recall, mAP@50, and mAP@50:95 values of 89.62%, 88.17%, 90.19%, and 67.94%, respectively. Compared with the baseline YOLOv8n-seg model, these values increased by 2.42, 2.90, 2.75, and 2.57 percentage points, respectively. Moreover, fruit–peduncle matching, geometric constraints, RGB-D depth information, and PCA-based local direction estimation were combined to infer the picking point and recover its 3D coordinates from the segmentation masks. In the evaluation of 57 RGB-D images, the picking point position accuracy reached 98.2%, while the local peduncle direction accuracy reached 91.2%. Overall, the proposed method can accurately segment mango fruits and peduncles in complex natural environments and convert the segmentation results into picking point positions, 3D coordinates, and local direction information, thereby providing theoretical and technical support for intelligent mango harvesting robots.

Plant phenotyping relevance

マンゴー果実・果梗の画像セグメンテーションと3D形状情報の抽出手法を開発・評価しており、単なる収穫対象の位置検出を超えて、再利用可能な植物器官の形態情報を取得する方法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.