Deep Learning Methods Using Imagery from a Smartphone for Recognizing Sorghum Panicles and Counting Grains at a Plant Level

Abstract

High-throughput phenotyping is the bottleneck for advancing field trait characterization and yield improvement in major field crops. Specifically for sorghum ( Sorghum bicolor L.), rapid plant-level yield estimation is highly dependent on characterizing the number of grains within a panicle. In this context, the integration of computer vision and artificial intelligence algorithms with traditional field phenotyping can be a critical solution to reduce labor costs and time. Therefore, this study aims to improve sorghum panicle detection and grain number estimation from smartphone-capture images under field conditions. A preharvest benchmark dataset was collected at field scale (2023 season, Kansas, USA), with 648 images of sorghum panicles retrieved via smartphone device, and grain number counted. Each sorghum panicle image was manually labeled, and the images were augmented. Two models were trained using the Detectron2 and Yolov8 frameworks for detection and segmentation, with an average precision of 75% and 89%, respectively. For the grain number, 3 models were trained: MCNN (multiscale convolutional neural network), TCNN-Seed (two-column CNN-Seed), and Sorghum-Net (developed in this study). The Sorghum-Net model showed a mean absolute percentage error of 17%, surpassing the other models. Lastly, a simple equation was presented to relate the count from the model (using images from only one side of the panicle) to the field-derived observed number of grains per sorghum panicle. The resulting framework obtained an estimation of grain number with a 17% error. The proposed framework lays the foundation for the development of a more robust application to estimate sorghum yield using images from a smartphone at the plant level.

Plant phenotyping relevance

スマートフォン画像からソルガム穂の検出・分割と粒数推定を開発・検証しており、植物形質取得手法が研究の中心である。

Code and data availability

The paper's authors explicitly state that the code used to train, test, and analyze the data is publicly available on GitHub. The phenotype image datasets are only available upon request.