3D reconstruction and segmentation of grape bunches for robotic berry thinning

Abstract

• End-to-end pipeline from neural reconstruction to physical grape berry manipulation. • Efficient point clouds generation using NeRF and the metric scale derived directly from robot kinematics. • RANSAC sphere fitting achieves 92.1% berry detection precision without annotated training data. • Stem-aligned 6-DoF pose optimization improves end-to-end grip success by 17.2%. Table grape thinning requires selective removal of 20–40% of berries from dense clusters. In practice, workers decide which berries to remove by considering both the approximate berry count and local 3D spatial characteristics such as crowding and relative positioning. Automating this task is challenging because conventional 2D image-based approaches suffer from occlusion-related counting errors and lack explicit 3D spatial information necessary for reliable manipulation. We propose a robot-integrated vision pipeline that reconstructs grape bunch structure from posed multi-view RGB images. Neural Radiance Fields (NeRF) is used to learn a volumetric scene representation, from which a dense, low-noise point cloud is extracted via depth back-projection, and RANSAC-based geometric fitting models individual berries and stems, enabling berry-level segmentation and orientation estimation for manipulation planning. The perception pipeline uses an eye-in-hand RealSense D405 camera mounted on a Fanuc CRX-5iA collaborative robot. Camera poses are derived from the robot kinematic chain, allowing the reconstructed point cloud and detected berry centers to be expressed directly in the metric robot base frame without external scale recovery. In robot-mounted RealSense D405 experiments on 10 grape bunches, RANSAC sphere fitting achieved a counting MAE of 0.50 berries, RMSE of 0.71 berries, and mean center localization error of 2.71 mm. On a 52-bunch benchmark, RANSAC sphere fitting outperforms the learning-based SoftGroup++ method for 3D berry instance segmentation (92.1% vs 82.8% average precision) without requiring annotated training data. In 35 manipulation trials, the system achieved an 85.7% pre-grasp reachability rate and an 83.3% conditional target success rate demonstrating an end-to-end pipeline from neural reconstruction to manipulation-ready berry poses.

Plant phenotyping relevance

ブドウ房の3D再構成、ベリー分割・計数・位置推定を中核とするロボット統合型フェノタイピング手法であり、技術性能も定量評価している。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.