An image-based system for locating pruning points in apple trees using instance segmentation and RGB-D images

Abstract

Intelligent pruning is an effective way to improve the efficiency of fruit tree pruning and reduce production costs, where the positioning of fruit tree pruning points is the key. In this study, a pruning point localisation method based on deep learning and red-green-blue-depth (RGB-D) is proposed for dormant tall spindle apple trees, which can quickly and accurately identify the pruning points on the primary branches. Firstly, red-green-blue (RGB) images and depth images of apple trees were acquired by using the Realsense D435i depth camera, and the SOLOv2 instance segmentation model was used to segment the trunks, branches, and supports in RGB images. Secondly, the manual pruning rules were adapted to improve the pruning methods; then using OpenCV image processing method, the junction points of branches and trunk and potential pruning points were gained, and the world coordinates of the points were obtained according to the coordinate transformation to calculate the length of branch diameter and spacing. Finally, the pruning points were determined according to the pruning rules. The results show that SOLOv2 has better segmentation effect compared with Mask R–CNN and Cascade Mask R–CNN. The mean absolute error between the estimated and manually measured values of branch diameter and spacing are 1.10 mm and 16.06 mm, and the recognition accuracy of pruning points is 87.2%, with a recognition time of about 3.8 s for each image. It is shown that the method can measure branch diameter and spacing and quickly locate pruning points with high reliability and accuracy, and the study provides a basis for the development of apple tree pruning robots.

Plant phenotyping relevance

RGB-D画像と画像処理・深層学習による枝径・枝間隔の推定および剪定点抽出が研究の中心であり、植物形態形質の取得手法を開発・検証しているため。

Code and data availability

公開本文の所在を確認できませんでした。非公開または購読が必要な可能性があります。

No evidence-backed public reproduction asset is currently recorded.