A dual-branch model combining convolution and vision transformer for crop disease classification

Abstract

Computer vision holds tremendous potential in crop disease classification, but the complex texture and shape characteristics of crop diseases make disease classification challenging. To address these issues, this paper proposes a dual-branch model for crop disease classification, which combines Convolutional Neural Network (CNN) with Vision Transformer (ViT). Here, the convolutional branch is utilized to capture the local features while the Transformer branch is utilized to handle global features. A learnable parameter is used to achieve a linear weighted fusion of these two types of features. An Aggregated Local Perceptive Feed Forward Layer (ALP-FFN) is introduced to enhance the model’s representation capability by introducing locality into the Transformer encoder. Furthermore, this paper constructs a lightweight Transformer block using ALP-FFN and a linear self-attention mechanism to reduce the model’s parameters and computational cost. The proposed model achieves an exceptional classification accuracy of 99.71% on the PlantVillage dataset with only 4.9M parameters and 0.62G FLOPs, surpassing the state-of-the-art TNT-S model (accuracy: 99.11%, parameters: 23.31M, FLOPs: 4.85G) by 0.6%. On the Potato Leaf dataset, the model attains 98.78% classification accuracy, outperforming the advanced ResNet-18 model (accuracy: 98.05%, parameters: 11.18M, FLOPs: 1.82G) by 0.73%. The model proposed in this paper effectively combines the advantages of CNN and ViT while maintaining a lightweight design, providing an effective method for the precise identification of crop diseases.

Plant phenotyping relevance

植物の病徴を画像から分類するCNN・ViT融合モデルを開発し、複数データセットで精度・計算量を比較検証しているため、病害状態の画像ベース表現型推定が中心である。

Code and data availability

植物フェノタイピング解析を再現する公開資産であることを、入力本文と直接リンクから確認できなかったため保留しました。

No evidence-backed public reproduction asset is currently recorded.