Apple diameter prediction during mechanical picking based on flexible force-sensing and CNN-BiLSTM-Attention method

Abstract

Fruit diameter grading is essential for commercialization, packaging, and market sales, directly impacting the value and competitiveness of fruits. Traditional diameter grading is typically performed after harvesting, relying on manual or mechanical methods. This process introduces extra steps and increases the risk of fruit damage during transit, which can reduce economic efficiency. To address this issue, this study introduces a real-time apple diameter grading method utilizing a flexible force-sensing gripper and the CNN-BiLSTM-Attention deep learning network, enabling synchronized intelligent identification and grading of apple diameter during robotic mechanical picking. First, ionogel-based triboelectric nanogenerators (IG-TENG) were developed and mounted on the surface of a three-finger Fin-Ray flexible picking end effector. A contact force monitoring system was established using a modular apple model, and the force sensor was calibrated. This setup allowed for accurate measurement of the contact force between the fingers and the apple. Using a multi-layer perceptron (MLP) to integrate mechanical response data, robotic hand motor stroke, and apple posture information, an apple contact force model was created to accurately predict the actual gripping force under various grasping conditions. Finally, a CNN-BiLSTM-Attention diameter prediction model was designed to deliver real-time, precise fruit diameter estimates. Orchard experiments demonstrated that the apple diameter grading method, combining force sensing with deep learning, achieved a mean absolute error (MAE) of 2.13 mm and a grading accuracy of 92 %, supporting non-destructive gripping and accurate grading. This research addresses the limitations of traditional diameter grading methods, streamlines harvesting steps, enhances efficiency, and offers a cost-effective and reliable solution for non-destructive fruit diameter grading.

Plant phenotyping relevance

リンゴ径という植物器官形質を、力覚センサーと深層学習でリアルタイム推定・等級化する手法の開発、校正、検証が研究の中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.