Citrus pose estimation under complex orchard environment for robotic harvesting

Abstract

The growth poses of citrus on trees are diverse. To ensure minimal loss during citrus harvesting, accurately estimating the pose of citrus is particularly important. To solve this problem, this research developed a real-time citrus pose estimation system based on neural networks and point cloud processing algorithms. Specifically, this method uses neural networks to identify citrus. After constructing the citrus point cloud, it is input into the Random Sample Consensus with Levenberg-Marquardt (RANSAC-LM) point cloud processing algorithm to obtain the citrus coordinates. Combined with citrus growth information, the pose is output. By analyzing the distribution of citrus poses, citrus poses convenient for end- effector harvesting are defined. To enhance the camera's ability to obtain information about citrus, a camera observation model is constructed to dynamically adjust the camera position. Through experiments, the appropriate deep learning target detection framework YOLO V5 is selected for citrus object detection. The precision (P), recall rate (R), and mean average precision (mAP) are 92.3 %, 79.1 %, and 88.5 % respectively. This network can handle detection tasks in real orchard environments. The original Random Sample Consensus (RANSAC) is improved by using the Levenberg-Marquardt (LM) nonlinear optimization method. Experimental results show that RANSAC-LM reduces the citrus center coordinate precision error from (0.2, 0.2, 2.3) mm to (0.1, 0.2, 1.4) mm, reduces the accuracy Spherical Error Probable (SEP) from 2.77 to 1.61, and finally reduces the citrus pose error from 5.72° to 2.43°. The efficiency of the proposed citrus pose estimation algorithm is 0.24 s. Deployed on a citrus picking robot, it verifies the feasibility of the algorithm and provides a new solution for the pose estimation problem of citrus harvesting robots.

Plant phenotyping relevance

柑橘果実の座標・生育姿勢という器官形質を点群処理とニューラルネットワークで推定し、精度を実験検証している。収穫対象の単なる検出を超えた姿勢計測法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.