Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning.

Abstract

Citrus Huanglongbing (HLB), also named citrus greening disease, occurs worldwide and is known as a citrus cancer without an effective treatment. The symptoms of HLB are similar to those of nutritional deficiency or other disease. The methods based on single-source information, such as RGB images or hyperspectral data, are not able to achieve great detection performance. In this study, a multi-modal feature fusion network, combining a RGB image network and hyperspectral band extraction network, was proposed to recognize HLB from four categories (HLB, suspected HLB, Zn-deficient, and healthy). Three contributions including a dimension-reduction scheme for hyperspectral data based on a soft attention mechanism, a feature fusion proposal based on a bilinear fusion method, and auxiliary classifiers to extract more useful information are introduced in this manuscript. The multi-modal feature fusion network can effectively classify the above four types of citrus leaves and is better than single-modal classifiers. In experiments, the highest accuracy of multi-modal network recognition was 97.89% when the amount of data was not very abundant (1,325 images of the four aforementioned types and 1,325 pieces of hyperspectral data), while the single-modal network with RGB images only achieved 87.98% recognition and the single-modal network using hyperspectral information only 89%. Results show that the proposed multi-modal network implementing the concept of multi-source information fusion provides a better way to detect citrus HLB and citrus deficiency.

Plant phenotyping relevance

RGB画像とハイパースペクトル情報から柑橘葉の病害・栄養欠乏状態を推定するマルチモーダル手法を提案・評価しており、植物状態の取得と分類が研究の中心である。

Code and data availability

The paper's RGB/hyperspectral citrus leaf dataset and multi-modal network code are not publicly deposited; the data availability statement only promises raw data from the authors upon request, with no public URL or repository identifier in the supplied blocks.

No evidence-backed public reproduction asset is currently recorded.