Flexible spatial-frequency feature fusion for UAV-based semantic segmentation in rice phenotyping

Abstract

• FSFF improves UAV-based semantic segmentation for rice phenotyping. • LFFE builds the frequency awareness and mitigates the panicle-leaf similarity. • ASCE restores phase-aware local context and addresses the mutual occlusion. • The proposed method was evaluated in practical applications of rice breeding. UAV-based semantic segmentation offers new insights to accelerate breeding better varieties in rice breeding applications. However, the morphological similarity and mutual occlusion between the panicles and leaves still pose severe challenges for efficient rice phenotyping. To address these problems, this paper proposed a flexible spatial-frequency feature fusion (FSFF) method for high-throughput UAV-based semantic segmentation. The FSFF method consists of three key components: Learnable frequency feature extraction (LFFE), Adaptive spatial context enhancement (ASCE), and hierarchical feature fusion (HFF). LFFE is employed to build the foundation of frequency awareness, addressing the challenges from the morphological similarity between the panicles and leaves; ASCE is introduced to enhance boundary information and mitigate the negative effects of mutual occlusion. After that, the LFFE and ASCE modules are integrated in the HFF mode through a series of transformations. Ablation study was conducted to confirm the effectiveness of the proposed modules, and visualized explanation for performance improvement was explored by transforming the learned kernels to frequency spectrums. Later, the FSFF method was compared with the mainstream semantic segmentation approaches. Experimental results demonstrate that the FSFF method outperformed other counterparts in mIoU (+2.12%), pixel accuracy (+0.8%), and SSIM (+1.09%) with the best inference speed (0.7311 ms/image). Finally, the FSFF method was evaluated on the public dataset and practical rice breeding applications. The experimental results prove the generalization and potential of FSFF method in rice phenotyping, which may build a foundation to accelerate breeding cycles and ensure food security. Relevant codes will be available at https://github.com/ZZZ-bbb/FSFF/tree/master .

Plant phenotyping relevance

UAV画像によるイネの穂・葉の形態を対象としたセマンティックセグメンテーション手法を開発し、アブレーション、比較評価、公開データセットおよび育種実環境で検証しているため、フェノタイピング手法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.