Application of multimodal data fusion and explainable AI for classifying water stress in sweet potatoes.

Abstract

Sweet potato (Ipomoea batatas L.) exhibits strong resilience in nutrient-poor soils and contains high levels of dietary fiber and antioxidant compounds. It also is highly tolerant to water stress, which has also contributed to its global distribution, particularly in regions prone to climatic variability. However, frequent abnormal climatic events have recently caused declines in both the quality and yield of sweet potatoes. To address this, machine learning (ML) and deep learning (DL) models based on a Vision Transformer-Convolutional Neural Network (ViT-CNN) were developed to classify water stress levels in sweet potato. RGB-thermal imagery captured from low-altitude platforms and various growth indicators were used to develop the classifier. The K-Nearest Neighbors (KNN) model outperformed other ML models in classifying water stress levels at all growth stages. The DL model simplified the original five-level water stress classification into three levels. This enhanced its sensitivity to extreme stress conditions, improve model performance, and increased its applicability to practical agricultural management strategies. To enhance practical applicability under open-field conditions, several environmental variables were newly defined to calculate the crop water stress index (CWSI). Furthermore, an integrated system was developed using gradient-weighted class activation mapping (Grad-CAM), explainable artificial intelligence (XAI), and a graphical user interface (GUI) to support intuitive interpretation and actionable decision-making. The system will be expanded into an online and fixed-camera platform to enhance its applicability to smart farming in diverse field crops.

Plant phenotyping relevance

RGB・熱画像と生育指標を用いてサツマイモの水ストレス状態を分類するモデルを開発し、CWSI、XAI、GUIを統合したシステムを構築しており、植物状態の取得・推定手法が研究の中心である。

Code and data availability

The article describes RGB–thermal imagery, growth indicator measurements, ML/DL models, and a GUI system, but contains no data availability statement, repository deposit, or public URL for any dataset, image set, code, or trained model. allowed_urls is empty, so no paper-specific public asset can be linked.

No evidence-backed public reproduction asset is currently recorded.