Upgrading swin-B transformer-based model for accurately identifying ripe strawberries by coupling task-aligned one-stage object detection mechanism

Abstract

With the wave of agricultural modernization, deep learning technology has brought revolutionary changes to the vision system of strawberry picking robots. Yet, the morphological diversity of strawberries, small and dense targets, and high overlap scenes make the detection and ripeness classification of strawberries a great challenge. To solve these problems, we introduce a new task-aligned one-stage object detection (TOOD) mechanism. Firstly, we incorporate the Swin-B (Swin-Base) transformer module to enhance the feature extraction performance in the backbone network. Secondly, we replace the original feature pyramid network (FPN) with CARAFE-FPN, which utilizes advanced upsampling methods to enhance detection at different scales. A multi-scale training (MST) approach is applied to capture the small targets effectively. Additionally, the Augmentations library is utilized for dataset augmentation to enhance the model’s generalization. Lastly, we refine the task alignment learning head and propose simple anchor alignment metric (S-aam) to reduce the impact of parameters on network performance for finding the optimal solutions. We collected a complex strawberry image dataset of more than 90,000 instances to test the method effectiveness in detecting strawberry ripeness. The results show that our model achieves 74.1% average precision (AP), 93.9% AP₅₀, and 84.1% AP₇₅, respectively. Our model shows the superior detection performance compared to most of models with fewer parameters and lower FLOPs. In addition, our model obtained the highest accuracy in detecting small strawberry targets. To prove the generalization of the model, we also verify it on COCO dataset, and the results show that the performance has been enhanced by 0.7% compared to the baseline. In summary, our proposed methods can be used to accurately identify ripe strawberries, which has the potential to be applied in strawberry picking robot system.

Plant phenotyping relevance

イチゴ画像から成熟度という植物状態を推定する物体検出手法を開発し、専用データセットで性能評価しているため、植物フェノタイピング手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.