Improved YOLOv8 for multi-colored apple fruit instance segmentation and 3D localization

Abstract

Robotic apple harvesting requires precise instance segmentation and 3D localization, especially for multi-colored apples under complex orchard conditions with occlusions and variable lighting. Current deep learning methods lack robustness and accuracy for such scenarios, limiting automation. This study proposes improved YOLOv8-based models to enhance segmentation of multi-colored apples, combined with a high-precision 3D localization pipeline to advance practical robotic harvesting. To address these issues, this study collected apple images in three colors from two locations, creating a dataset of 5171 images. Four enhanced YOLOv8-based models—RA-YOLO, GA-YOLO, YA-YOLO, and MCA-YOLO—were proposed for segmenting red, green, yellow, and mixed multi-colored apples. RA-YOLO integrates the GD mechanism and EMBConv structure based on EfficientNet's MBConv. GA-YOLO replaces standard convolutions with dynamic serpentine convolution and adds the P6 layer for large object detection. YA-YOLO utilizes deformable convolution (DCNv2) and introduces the new attention mechanism MPCA. MCA-YOLO combines the P6 layer, DCNv2, and EMBConv structure, merging the strengths of other models. RA-YOLO, GA-YOLO, and YA-YOLO achieved mAP values of 95.2 %, 96.4 %, and 95.4 %, respectively, for single-colored apple instance segmentation, surpassing baseline models and those in existing literature. MCA-YOLO achieved mAP values of 95.6 %, 96.6 %, and 94.6 % for single-colored apples and 95.6 % for mixed multi-colored apples. Ablation experiments validated the necessity of each module. Finally, a high-precision 3D localization and shaping pipeline was developed, achieving an average localization error of 2.636 mm and a shaping error of 0.768 mm, enabling millimeter-level localization and sub-millimeter-level shaping for apple harvesting optimization.

Plant phenotyping relevance

リンゴ果実のインスタンス分割、3D位置推定、形状推定を開発・検証しており、収穫対象の単なる検出を超えて果実形状という植物器官形質を定量化する手法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.