Multimodal data fusion and attention-based deep learning for estimating winter wheat chlorophyll content

Abstract

Accurate estimation of leaf chlorophyll content is essential for monitoring crop growth and supporting precision agricultural management. The soil and plant analyzer development (SPAD) instrument readings represent the relative chlorophyll content (RCC) in leaves, a key indicator of photosynthetic capacity and physiological status in wheat. This study proposes a multimodal data fusion approach integrating unmanned aerial vehicle (UAV)-derived vegetation indices (VI) and texture features (TF) from multispectral imagery with short-term environmental time-series data collected from in-field meteorological and soil sensors to estimate winter wheat RCC. A self-attention deep neural network (SA-DNN) was developed to capture complex nonlinear relationships among multimodal inputs. Employing a multi-stage progressive feature selection strategy that combines Pearson and Spearman correlations, minimum redundancy maximum relevance (mRMR), and least absolute shrinkage and selection operator (LASSO), eight optimal features were ultimately selected from VI and TF, together with indicators of short-term variability in environmental factors (EF). These features included OSAVI, NDRE, R-Mea, R-SEM, AH-std-7, DPT-std-7, SD-std-7, and pH-std-7. The SA-DNN model achieved the best estimation performance (R² = 0.913, RMSE = 3.945), significantly outperforming traditional machine learning models such as XGBoost, random forest (RF), support vector regression (SVR), Adaboost, and partial least squares regression (PLSR). SHapley Additive exPlanations (SHAP) analysis further quantified the contributions of individual features, revealing that short-term (7-day) fluctuations in EF, particularly sunshine duration and air humidity, played a dominant role in regulating variations in RCC. Overall, this study demonstrates that the synergistic integration of multimodal data within an attention-based deep learning framework significantly augments the precision of winter wheat RCC estimation, providing a powerful tool for real-time crop growth monitoring and the optimization of precision agricultural management.

Plant phenotyping relevance

UAV画像・環境センサーデータから小麦葉のクロロフィル含量を推定する深層学習手法を開発し、複数モデルとの性能比較で検証しており、植物表現型取得が研究の中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.