Robust Fall Army Worm detection in maize using multimodal RGB and thermal image fusion.

Abstract

Effective pest and disease detection plays a crucial role in minimizing crop losses and improving decision-making in precision agriculture. Among the most destructive pests affecting maize crops globally is the Fall Army Worm (FAW), known for its rapid spread and high impact on yield. Existing detection practices often rely on manual scouting, which can be inefficient, labour intensive and prone to human error. This study proposes a novel deep learning based framework for the automatic classification of FAW infested and healthy maize crops by integrating RGB and thermal image modalities. The core objective is to enhance detection accuracy through multimodal image fusion. A hybrid DNN-ViT model is introduced, combining two complimentary pipelines: (i) feature-level fusion, where CNN extracted features from RGB and thermal images are fused and classified using a Deep Neural Network (DNN) and (ii) image-level fusion, where a 6 channel RGB-thermal image is directly processed using a modified Vision Transformer (ViT). Experimental results demonstrate that the fused model achieved superior performance with an accuracy of 0.98, precision, recall and F1-score of 0.98 and AUC-ROC of 0.98 on the test set, outperforming models trained on RGB-only, thermal-only and unfused data. The ablation study confirms the effectiveness of multimodal fusion, with the no-fusion model showing significantly lower performance (accuracy-0.60 and AUC-ROC-0.67). This work highlights the benefits of integrating complementary data sources for robust crop health monitoring. Future research will explore enhanced fusion strategies, environmental robustness and field level deployment to validate the model's practical applicability.

Plant phenotyping relevance

RGB・熱画像融合によるFAW被害・健全状態の画像判定モデルを開発し、融合方式や性能を比較検証しているため、植物の健康状態を取得する方法が中心である。

Code and data availability

The paper's paired RGB/thermal maize FAW image dataset is publicly deposited on Figshare (part of a peer-reviewed data publication), and the authors' custom Python analysis code is released as a public supplementary file (Supplementary Code.zip) with explicit availability language. The Figshare URL matches an allowed,