Canopy-attention-YOLOv4-based immature/mature apple fruit detection on dense-foliage tree architectures for early crop load estimation

Abstract

Accurate detection of both immature and mature apples in orchard environments is essential for early crop load management. A near real-time method is proposed in this study for detecting green (early-stage), green–red-mixed (mid-stage; red varieties), or red apples (harvest-stage; red varieties). Both the number of fruits and fruit size were estimated for the entire tree with a single image captured by a low-cost smartphone using two different imaging methods (oblique and panorama modes). An attention mechanism module called the convolutional block attention module (CBAM) was added to the generic YOLOv4 detector to improve the detection accuracy by only focusing on the target canopies. Furthermore, an adaptive layer and larger-scale feature map were included in the modified network structure, enabling it to adapt to various characteristics of fruits and canopies during the entire growing season, such as different fruit colors and sizes, dense-foliage conditions, and severe occlusions. To verify the effectiveness of the proposed method, we compared our improved model, canopy-attention-YOLOv4 (or CA-YOLOv4), with other commonly adopted models available in the literature, such as the original YOLOv4, Faster R-CNN, and single-shot multibox detector (SSD). Two commonly planted apple varieties, “Envy” and “Scifresh”, were used in the study. The results showed that the proposed CA-YOLOv4 detector performed the best among all the algorithms, with up to ∼3% improvement in terms of fruit counting over the original YOLOv4. With the “Envy” variety, fruit detection accuracies were 86.2%, 87.5%, and 92.6% for the early-, mid-, and harvest stages, respectively, whereas the same were 71.0%, 83.6%, and 86.3% for the “Scifresh” variety, which has denser canopy foliage. Both imaging methods proposed in this study only needed one/single shot targeting the entire fruiting tree, which can be highly efficient for real-world applications in crop load management. Finally, CA-YOLOv4 estimated fruit sizes were compared to manual measurements, achieving up to R² values of 0.68 in fruit height and 0.66 in fruit width estimations.

Plant phenotyping relevance

果実の検出・計数・サイズ推定を行う画像ベースの植物表現型取得法を開発・比較検証しており、方法が研究の中心である。

Code and data availability

公開本文の所在を確認できませんでした。非公開または購読が必要な可能性があります。

No evidence-backed public reproduction asset is currently recorded.