A lightweight method for apple disease segmentation using multimodal transformer and sensor fusion

Abstract

To address the challenges of multimodal data fusion, low deployment efficiency, and inadequate recognition robustness in complex environments for fruit tree disease segmentation and severity classification, a multimodal parallel transformer-based framework was proposed for apple disease recognition and grading. This method integrates image data with multi-dimensional environmental sensor information. An image segmentation preprocessing module was incorporated to enhance lesion region representation, while a cross-scale attention mechanism and a frame-wise diffusion module were introduced to improve robustness under challenging backgrounds. Additionally, pruning, quantization, and knowledge distillation techniques were employed to enable lightweight deployment. Experimental results demonstrated that the full model achieved outstanding performance on apple disease recognition tasks, reaching a precision of 0.98, recall of 0.93, F1-score of 0.95, and accuracy of 0.96, surpassing several state-of-the-art methods including Mask R-CNN, SegFormer, and Swin Transformer. After compression, the model size was reduced to 76.4 MB, and computational complexity decreased to 6.1 G, enabling real-time inference speeds of 25.2 FPS and 39.6 FPS on Jetson Xavier and Orin platforms, respectively. Ablation studies confirmed the performance contributions of the segmentation preprocessing, sensor fusion, and diffusion modules, demonstrating the potential of the proposed framework for deployment in resource-constrained agricultural scenarios.

Plant phenotyping relevance

リンゴ病斑の画像セグメンテーションと重症度分類を中核とする軽量マルチモーダル手法を開発し、性能比較・アブレーション・実装性能評価まで行っているため、植物フェノタイピング手法として含める。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.