Foundation model-based apple ripeness and size estimation for selective harvesting

Abstract

Harvesting is a critical task in the tree fruit industry, demanding extensive manual labor and substantial costs, and exposing workers to potential hazards. Recent advances in automated harvesting offer a promising solution by enabling efficient, cost-effective, and ergonomic fruit picking within tight harvesting windows. However, existing harvesting technologies often indiscriminately harvest all visible and accessible fruits, including those that are unripe or undersized. This study introduces a novel foundation-model-based framework for efficient apple ripeness and size estimation. Specifically, we curated two public RGBD-based Fuji apple image datasets, integrating expanded annotations for ripeness (“Ripe” vs. “Unripe”) based on fruit color and image capture dates. The resulting comprehensive dataset, Fuji-Ripeness-Size Dataset, includes 4,027 images and 16,257 annotated apples with ripeness and size labels. To the best of our knowledge, this is the first published dataset on apples with ripeness and size annotations. Leveraging Grounding-DINO, a foundation-model-based object detector, we achieved robust apple detection and ripeness estimation, with mean Average Precision being 72.8, outperforming other state-of-the-art models in the evaluation on our dataset. Additionally, we developed six size estimation algorithms, made a comprehensive comparison using box-plots, and identified the best algorithm with lowest error and variation. The Fuji-Ripeness-Size Dataset and the apple detection and size estimation algorithms are made publicly available¹1The code and dataset is available at https://github.com/zhukeyi-stan/Fuji_Ripeness_And_Size_Estimation., which provides valuable benchmarks for future studies in automated and selective harvesting.

Plant phenotyping relevance

リンゴの熟度・サイズという植物器官の形質を画像から推定する手法を開発・比較し、データセットとベンチマークも提供しているため、フェノタイピング手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.