In-field tea shoot detection and 3D localization using an RGB-D camera

Abstract

Tea shoot detection and localization are highly challenging tasks because of varying illumination, inevitable occlusion, tiny targets, and dense growth. To achieve the automatic plucking of tea shoots in a tea garden, a reliable algorithm based on red, green, blue-depth (RGB-D) camera images was developed to detect and locate tea shoots in fields for tea harvesting robots. In this study, labeling criteria were first established for the images collected for multiple periods and varieties in the tea garden. Then, a “you only look once” (YOLO) network was used to detect tea shoot (one bud with one leaf) regions on RGB images collected by an RGB-D camera. Additionally, the detection precision for tea shoots was 93.1% and the recall rate was 89.3%. To achieve the three-dimensional (3D) localization of the plucking position, 3D point clouds of the detected target regions were acquired by fusing the depth image and RGB image captured by an RGB-D camera. Then, noise was removed using point cloud pre-processing and the point cloud of the tea shoots was obtained using Euclidean clustering processing and a target point cloud extraction algorithm. Finally, the 3D plucking position of the tea shoots was determined by combining the tea growth characteristics, point cloud features, and sleeve plucking scheme, which solved the problem that the plucking point may be invisible in fields. To verify the effectiveness of the proposed algorithm, tea shoot localization and plucking experiments were conducted in the tea garden. The plucking success rate for tea shoots was 83.18% and the average localization time for each target was about 24 ms. All the results demonstrate that the proposed method could be used for robotic tea plucking.

Plant phenotyping relevance

RGB-D画像、物体検出、点群処理により茶芽の検出・3D位置推定法を開発・検証しており、植物器官の形態・位置という表現型取得が中心的です。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.