Detection and Reconstruction of Passion Fruit Branches via CNN and Bidirectional Sector Search.

Abstract

Accurate detection and reconstruction of branches aid the accuracy of harvesting robots and extraction of plant phenotypic information. However, the complex orchard background and twisting growing branches of vine fruit trees make this challenging. To solve these problems, this study adopted a Mask Region-based convolutional neural network (Mask R-CNN) architecture incorporating deformable convolution to segment branches in complex backgrounds. Based on the growth posture, a branch reconstruction algorithm with bidirectional sector search was proposed to adaptively reconstruct the segmented branches obtained by an improved model. The average precision, average recall, and F1 scores of the improved Mask R-CNN model for passion fruit branch detection were found to be 64.30%, 76.51%, and 69.88%, respectively, and the average running time on the test dataset was 0.75 s per image, which is better than the compared model. We randomly selected 40 images from the test dataset to evaluate the branch reconstruction. The branch reconstruction accuracy, average error, average relative error of reconstructed diameter, and mean intersection-over-union (mIoU) were 88.83%, 1.98 px, 7.98, and 83.44%, respectively. The average reconstruction time for a single image was 0.38 s. This would promise the proposed method to detect and reconstruct plant branches under complex orchard backgrounds.

Plant phenotyping relevance

CNNによる枝の検出・再構成と直径推定を中心に、植物形態情報の抽出手法を開発・評価しているため、ロボット収穫支援に加えて再利用可能な表現型計測法に該当する。

Code and data availability

The paper's Data Availability statement explicitly provides all training/test data (passion fruit branch images and annotations) freely via the authors' public GitHub repository, which matches an allowed URL.