Deep learning approach for detecting tomato flowers and buds in greenhouses on 3P2R gantry robot.

Abstract

In recent years, significant advancements have been made in the field of smart greenhouses, particularly in the application of computer vision and robotics for pollinating flowers. Robotic pollination offers several benefits, including reduced labor requirements and preservation of costly pollen through artificial tomato pollination. However, previous studies have primarily focused on the labeling and detection of tomato flowers alone. Therefore, the objective of this study was to develop a comprehensive methodology for simultaneously labeling, training, and detecting tomato flowers specifically tailored for robotic pollination. To achieve this, transfer learning techniques were employed using well-known models, namely YOLOv5 and the recently introduced YOLOv8, for tomato flower detection. The performance of both models was evaluated using the same image dataset, and a comparison was made based on their Average Precision (AP) scores to determine the superior model. The results indicated that YOLOv8 achieved a higher mean AP (mAP) of 92.6% in tomato flower and bud detection, outperforming YOLOv5 with 91.2%. Notably, YOLOv8 also demonstrated an inference speed of 0.7 ms when considering an image size of 1920 × 1080 pixels resized to 640 × 640 pixels during detection. The image dataset was acquired during both morning and evening periods to minimize the impact of lighting conditions on the detection model. These findings highlight the potential of YOLOv8 for real-time detection of tomato flowers and buds, enabling further estimation of flower blooming peaks and facilitating robotic pollination. In the context of robotic pollination, the study also focuses on the deployment of the proposed detection model on the 3P2R gantry robot. The study introduces a kinematic model and a modified circuit for the gantry robot. The position-based visual servoing method is employed to approach the detected flower during the pollination process. The effectiveness of the proposed visual servoing approach is validated in both un-clustered and clustered plant environments in the laboratory setting. Additionally, this study provides valuable theoretical and practical insights for specialists in the field of greenhouse systems, particularly in the design of flower detection algorithms using computer vision and its deployment in robotic systems used in greenhouses.

Plant phenotyping relevance

トマトの花・蕾という植物器官の状態を画像から検出する手法を開発・比較評価し、ロボットへの実装まで扱っており、単なる生物実験の routine 測定ではなく、表現型取得法が中心である。

Code and data availability

The paper describes a custom tomato flower/bud image dataset (1250 images dataset A, 300 from Google Images) and YOLOv5/YOLOv8 models, but no block contains a public deposit, repository, or availability statement for the dataset, annotations, code, or trained models. The only linked material is the article's own 'Supp​

No evidence-backed public reproduction asset is currently recorded.