Strawberry harvest date prediction using multi-feature fusion deep learning in plant factory

Abstract

Strawberries have high consumer demand due to their palatability and nutritional benefits. Commercial strawberry production in plant factories with artificial lighting (PFALs) is gaining popularity as a viable strategy for improving economic viability through high-quality fruit production. Accurate information of the optimal harvest date is crucial for optimizing harvesting decisions. While numerous studies utilize deep learning to assess strawberry ripeness, they typically only categorize generalized ripeness levels instead of predicting specific harvest dates, leaving a gap with the practical needs of growers. In this study, we proposed a two-stage multi-feature fusion model for strawberry harvest date prediction and integrated it with a web application to facilitate practical production management in PFALs. The model consists of a fruit segmentation network and a ripeness prediction network. A time-series image dataset of single fruits was constructed to continuously track the ripening process of strawberries, and a five-stage division of strawberry ripeness stages depending on optimal harvest dates was defined. A U-Net based segmentation network with post-processing was developed to automatically extract only the target fruits, which showed a reliable performance with a mIoU of 0.977. A multi-feature fusion network called Triple-Branch Attention Fusion (TBAF) was built to predict the ripeness categories with information on optimal harvest dates. The results showed that the TBAF model with fusion of color, attention-enhanced, and low-level shape features exhibited the highest performance compared to baseline models, with an overall accuracy of 0.859 and an F1 score of 0.859. In addition, a user-friendly web application was developed with the deployment of deep learning models and inspection video processing workflow to support strawberry harvesting in PFALs. Overall, this study demonstrated a prototype approach utilizing deep learning to provide essential information for grower’s decision making in practical strawberry production.

Plant phenotyping relevance

イチゴ果実の画像から成熟段階・最適収穫日を推定するセグメンテーションおよび深層学習ワークフローを開発しており、植物状態の取得・推定手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.