Real-time multi-attribute quality grading of tabletop strawberries under occlusions for harvesting robots

Abstract

• Modular RGB–D pipeline for real-time multi-attribute grading of strawberries. • ResNet-101 multi-head classifier predicted maturity, disease, and deformity jointly. • CycleGAN restored strawberry shape and texture from partially occluded views. • QH and MADM strategies converted multi-attribute predictions to grading decisions. • Field tests validated end-to-end perception-to-placement pipeline in farms. Strawberry grading was critical for meeting market standards, yet manual post-harvest sorting remained labor-intensive, damage-prone, and inconsistent, while most vision-based methods targeted single attributes and lacked integrated in-field operation. We presented a real-time RGB–D grading framework for tabletop-cultivated strawberries in greenhouse environments. The system combined instance segmentation (YOLOv11–Seg), occlusion-aware completion via CycleGAN, and a three-branch multi-task network based on ResNet-101 to simultaneously infer maturity, disease status (including asymptomatic fruit), and deformity. Pose normalization and depth alignment supported calibrated volume-to-mass regression for non-destructive weight estimation. To improve robustness to rare morphology, 500 deformed-fruit samples were synthesized with a Gemini-based generative model to mitigate class imbalance. Built upon this perception stack, we implemented two grading strategies: a deterministic Quality Hierarchy (QH) for real-time robotic routing and a weighted multi-attribute decision-making (MADM) scheme for flexible batch evaluation. The multi-attribute classifier achieved 93.65% overall accuracy (disease 94.09%, maturity 94.07%, shape 94.42%) with an inference latency of 33.65 ms (29.7 FPS). Depth-assisted mass estimation yielded average errors of 8.11% for complete fruits and 10.47% under occlusion after completion; in field deployment on marketable fruits routed to size grading, it achieved MAE/RMSE of 1.85/2.20 g with R 2 = 0.9384 (MAPE 8.48%) and a three-bin size-grade accuracy of 90.91% (100/110). In single-target harvesting mode on an RTX 4060 GPU (640x480, 30 fps), end-to-end latency was 87.56 ms per harvested target in complete mode and 197.56 ms when completion was invoked (CycleGAN 110.00 ms/instance). These results demonstrated a practical, non-destructive, occlusion-aware, and deployable multi-attribute grading solution for intelligent strawberry harvesting in real greenhouse scenarios.

Plant phenotyping relevance

RGB-D画像解析でイチゴの成熟度、病徴、形状、重量を推定し、性能検証とロボット実装まで行う中心的なフェノタイピング手法研究。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.