YOLOv9s-multi: orientation-based fruit selection for robotic apple thinning

Abstract

Accurate detection and orientation estimation of immature apples are crucial for effective thinning decisions in robotic apple thinning. Existing research either relies on computationally expensive RGB-D approaches with 3D geometric fitting to estimate orientation and size or requires multiple separate models for thinning decision, limiting their real-time performance on robotic platforms. To address these issues, a multi-task model, YOLOv9s-Multi, is proposed. First, this model integrates segmentation and keypoint heads to perform instance segmentation of immature apples and detect their calyx keypoint positions. Second, the detection head employs an efficient and lightweight Depthwise Convolution module (DWConv) to reduce model parameters while accurately capturing spatial features across channels. Finally, the orientation is derived from the segmentation centroid and calyx keypoint, enabling pixel-based fruit selection for thinning decision-making. This model is evaluated on a self-developed dataset that divides immature apples based on developmental stage: Flower-Retained Stage (FR-Stage) and Fruit-Visible Stage (FV-Stage). Results show that instance segmentation and keypoint AP@0.5 for FV-Stage are 89.3% and 86.4%, respectively, while for FR-Stage they are 71.6% and 79.8%. The model further achieves prediction accuracies of 92.80% (FV-Stage) and 72.59% (FR-Stage) within an acceptable error of 30 °. The pixel-based fruit selection method achieves 74.00% and 70.31% selection accuracy on the test and an additional measurement dataset, respectively. Compared with the baseline YOLOv9s-seg, the number of parameters is reduced by 11.4%. In contrast to 3D fitting methods, our approach provides lower computational complexity, faster inference speed, and higher accuracy. These results demonstrate that the proposed model can efficiently estimate the orientation of immature apples and perform fruit selection in close-range scenes and complex lighting environments, which are challenging for depth cameras to handle. The code and datasets are publicly available on GitHub: https://github.com/DIANSLEE/YOLOv9s-Multi.

Plant phenotyping relevance

未熟リンゴのセグメンテーション、萼点検出、重心との関係から果実の向きという器官形質を推定する画像解析手法が研究の中心であり、精度評価とデータセット検証も行っているため。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.