Multimodal fusion of UAV-based computer vision and plant water content dynamics for high-throughput soybean maturity classification

Abstract

Soybean is the most important oilseed and forage crop globally. Advancements in high-throughput phenotyping technologies are critical for accelerating genetic improvement in modern breeding research. However, conventional methods for assessing soybean maturity remain labor intensive. This study developed high-throughput phenotyping algorithms based on unmanned aerial vehicle (UAV) multispectral imagery combined with machine learning to monitor the maturity process of 30 soybean cultivars in large-scale breeding trials. UAV images and plant water content (PWC) data were collected to classify soybean maturity into four distinct phases: immaturity (i.e. the period before R5 stage), late pod filling (i.e. R5 to R6), physiological maturity (i.e. R7), and harvesting maturity (i.e. R8). We evaluated the performance of three classification approaches: (1) a computer vision model utilizing UAV-derived color features, (2) a PWC-based model retrieving PWC dynamics using UAV-derived feature, and (3) a multimodal fusion model integrating computer vision and PWC dynamics. Computer vision model effectively distinguished immature and mature plants but showed limitations in resolving specific maturity phases due to genetic variation in canopy color among cultivars (training set accuracy: 0.71; validation set accuracy: 0.69). The sensitive UAV-derived features were applied to establish the prediction model of PWC using convolutional neural network, which achieved the highest R 2 (training set: R 2 = 0.95; validation set: R 2 = 0.86) between the predicted and measured PWC. The PWC-based algorithm outperformed the computer vision approach, achieving higher classification accuracy (training set: 0.78; validation set: 0.79). Strong correlations between PWC and pod water content, stem water content, and leaf water content underscored the physiological relevance of PWC in tracking maturation dynamics. Further improvement in classification accuracy was achieved with the multimodal fusion model (training set: 0.84; validation set: 0.83), which combined the information of computer vision and PWC dynamics. It was also confirmed that the multimodal fusion model achieved the lowest misclassification rate in the validation analysis across diverse soybean cultivars. These findings emphasize the potential of integrating UAV-based computer vision and PWC features to improve the accuracy and efficiency of soybean maturity classification. The proposed multimodal approach offers a robust framework for phenotypic selection and trait evaluation, providing valuable insights for soybean breeding programs.

Plant phenotyping relevance

UAV画像、植物水分含量推定、機械学習を統合し、ダイズ成熟期という植物形質を高スループットに分類する手法を開発・検証しており、フェノタイピング手法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.