Soybean yield estimation and lodging discrimination based on lightweight UAV and point cloud deep learning

Abstract

The unmanned aerial vehicle (UAV) platform has emerged as a powerful tool in soybean (Glycine max (L.) Merr.) breeding phenotype research due to its high throughput and adaptability. However, previous studies have predominantly relied on statistical features like vegetation indices and textures, overlooking the crucial structural information embedded in the data. Feature fusion has often been confined to a one-dimensional exponential form, which can decouple spatial and spectral information and neglect their interactions at the data level. In this study, we leverage our team's cross-circling oblique (CCO) route photography and Structure-from-Motion with Multi-View Stereo (SfM-MVS) techniques to reconstruct the three-dimensional (3D) structure of soybean canopies. Newly point cloud deep learning models SoyNet and SoyNet-Res were further created with two novel data-level fusion that integrate spatial structure and color information. Our results reveal that incorporating RGB color and vegetation index (VI) spectral information with spatial structure information, leads to a significant reduction in root mean square error (RMSE) for yield estimation (22.55 ​kg ​ha -1 ) and an improvement in F1-score for five-class lodging discrimination (0.06) at S7 growth stage. The SoyNet-Res model employing multi-task learning exhibits better accuracy in both yield estimation (RMSE: 349.45 ​kg ​ha -1 ) when compared to the H2O-AutoML. Furthermore, our findings indicate that multi-task deep learning outperforms single-task learning in lodging discrimination, achieving an accuracy top-2 of 0.87 and accuracy top-3 of 0.97 for five-class. In conclusion, the point cloud deep learning method exhibits tremendous potential in learning multi-phenotype tasks, laying the foundation for optimizing soybean breeding programs.

Plant phenotyping relevance

UAV・SfM-MVSによるダイズ群落の3D構造再構成と、収量推定・倒伏判別のための専用深層学習モデル開発が研究の中心であり、再利用可能な表現型取得・推定手法に該当する。

Code and data availability

The article's Data availability statement explicitly says the code and data used in the study (soybean UAV point cloud phenotyping, SoyNet/SoyNet-Res models, yield/lodging analysis) are publicly downloadable from the authors' GitLab repository.