Automated Detection and Segmentation of Grain Spikes in Greenhouse Images Using Shallow and Deep Learning Neural Networks: A Comparison of Six Methods

Abstract

Abstract Image-based plant phenotyping is the major approach to quantitative assessment of important plant properties. For automated analysis of a large amount of image data from high-throughput greenhouse measurements, efficient techniques for image segmentation are required. However, conventional approaches to whole plant and plant organ segmentation are hampered by high variability of plant and background illumination, and naturally occurring changes in geometry and colors of growing plants. Consequently, application of advanced machine learning techniques for automated image segmentation is required. Here, we investigate six advanced neural network (NN) methods for detection and segmentation of grain spikes in RGB images including three detection deep NNs (SSD, Faster-RCNN, YOLOv3/v4), two deep (U-Net, DeepLabv3+) and one shallow segmentation NNs. Our experimental results show superior performance of deep learning NNs that achieve in average more than 90% accuracy by detection and segmentation of wheat as well as barley and rye spikes. However, different methods demonstrate different performance on matured, emergent and occluded spikes. In addition to comprehensive comparison of six NN methods, a GUI-based tool (SpikeApp) provided with this work demonstrates the application of detection and segmentation NNs to fully automated spike phenotyping. Further improvements of evaluated NN approaches are discussed.

Plant phenotyping relevance

穀粒穂の画像検出・セグメンテーション手法を比較評価し、SpikeAppによる自動フェノタイピングを実証しており、植物表現型取得手法が中心である。

Code and data availability

The paper's data availability statement explicitly provides demo software (SpikeApp with pre-trained U-Net, YOLOv3, and shallow ANN models) and example spike images for public download from the authors' IPK page. The DeepLabv3+ GitHub link is a cited third-party library, and psi.cz is the imaging facility, not a paper-