Using deep learning for pruning region detection and plant organ segmentation in dormant spur-pruned grapevines

Abstract

Even though mechanization has dramatically decreased labor requirements, vineyard management costs are still affected by selective operations such as winter pruning. Robotic solutions are becoming more common in agriculture, however, few studies have focused on grapevines. This work aims at fine-tuning and testing two different deep neural networks for: (i) detecting pruning regions (PRs), and (ii) performing organ segmentation of spur-pruned dormant grapevines. The Faster R-CNN network was fine-tuned using 1215 RGB images collected in different vineyards and annotated through bounding boxes. The network was tested on 232 RGB images, PRs were categorized by wood type (W), orientation (Or) and visibility (V), and performance metrics were calculated. PR detection was dramatically affected by visibility. Highest detection was associated with visible intermediate complex spurs in Merlot (0.97), while most represented coplanar simple spurs allowed a 74% detection rate. The Mask R-CNN network was trained for grapevine organs (GOs) segmentation by using 119 RGB images annotated by distinguishing 5 classes (cordon, arm, spur, cane and node). The network was tested on 60 RGB images of light pruned (LP), shoot-thinned (ST) and unthinned control (C) grapevines. Nodes were the best segmented GOs (0.88) and general recall was higher for ST (0.85) compared to C (0.80) confirming the role of canopy management in improving performances of hi-tech solutions based on artificial intelligence. The two fine-tuned and tested networks are part of a larger control framework that is under development for autonomous winter pruning of grapevines. Supplementary information The online version contains supplementary material available at 10.1007/s11119-023-10006-y.

Plant phenotyping relevance

深層学習によるブドウ樹の剪定領域検出と器官セグメンテーションを開発・評価しており、植物器官状態の画像ベース取得が中心である。

Code and data availability

The paper's annotated grapevine organ segmentation dataset (images with polygon/bounding-box annotations for cordon, arm, spur, cane, node) is publicly deposited on Zenodo. The pruning region detection dataset is not public and must be requested from the corresponding author. No author analysis code is available (code: