Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology

Abstract

Automatic vision-based picking in orchards and fields is a highly challenging task. The orchard banana central stock, which is large in size, low in color contrast, and falls within a complex background, was taken as the subject in this research. A measurement framework based on multi-vision technology was established, and a set of general methods were utilized to improve the comprehensive performance of multi-view-geometry-based vision modules in orchard picking tasks. Multiple cameras at different angles were deployed to maximize the perception range. The global geometric parameters of the cameras were calibrated and a robust semantic segmentation network was trained to achieve effective image pre-processing. A novel adaptive stereo matching strategy was designed to ensure that the robot reliably completes 3D triangulation at various depths as it moves across the target area. Global calibration errors were corrected via a high-accuracy point cloud stitching algorithm. Experimental results indicated that the proposed adaptive stereo matching strategy was accurate to different sampling depths and showed stable performance, and the proposed point cloud stitching algorithm accurately stitched multi-view point clouds. This work provides theoretical and practical references for the 3D sensing of banana central stocks in complex environments. The proposed technique was designed for adaptability of the multi-vision system for field perception, so it can be easily transferred to similar applications such as the 3D reconstruction of agricultural targets, 3D positioning of fruit clusters, and 3D robotic arm obstacle avoidance.

Plant phenotyping relevance

バナナ株の3次元形状を取得・再構成するマルチビジョン計測法が研究の中心であり、単なる収穫対象の位置検出を超えた植物器官の形態計測手法に該当する。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.