A generative AI-Driven framework integrating CNN-VLM-LLM for intelligent crop disease diagnosis and control strategy generation

Abstract

This study proposes a generative AI-driven framework integrating CNN, VLM and LLM, aiming to provide intelligent solutions for the diagnosis of crop diseases and the generation of control strategies. The framework comprises four core modules: a data enhancement and style transfer module based on CycleGAN, a CNN-based disease detection module, a VLM-based visual semantic description module, and an LLM-based control strategy generation module. In the data enhancement module, CycleGAN is utilized to perform style transfer on the original dataset. This process makes the image features more realistic under natural conditions and improves the model's generalization ability in real-world agricultural production environments. In the disease detection module, the Yolo-CDDet model is developed, which adopts a cascaded feature learning architecture. This architecture consists of a deformable convolution backbone network, a global-local feature pyramid pooling neck network, and a decoupled prediction structure detection head, enabling precise identification and classification of disease regions. In the visual semantic description module, the MultiTask-CLIP model is constructed, featuring a multi-task classification head. The model outputs text descriptions with fixed feature combinations, providing detailed visual evidence for subsequent control strategy formulation. In the control strategy generation module, the Falcon-40B large language model serves as the core component. By leveraging web crawlers to collect open-source professional agricultural literature and applying the LoRA fine-tuning method to optimize model parameters, the model is optimized. It generates scientifically grounded and practical control recommendations tailored to specific disease characteristics. Experimental results demonstrate that the Yolo-CDDet model achieves superior performance on both the original dataset and the dataset enhanced by style transfer. The model exhibits high Recall, Average Precision, and other excellent metrics. The MultiTask-CLIP model outperforms competing models across multiple evaluation criteria, particularly excelling in CIDEr scores. Additionally, the control strategy generation mechanism based on Falcon-40B surpasses baseline models in terms of Recall, Precision, ROUGE-L, and other quantitative analysis indicators, producing high-quality control strategy texts. This study offers a novel and effective approach for the intelligent diagnosis and integrated management of crop diseases.

Plant phenotyping relevance

作物病害領域で、病斑領域の画像検出・分類モデルを開発し、データ拡張や複数モデルとの性能評価を行っているため、植物の病害状態を観測・推定する方法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.