Viewpoint Planning for Fruit Size and Position Estimation

Abstract

Modern agricultural applications require knowledge about the position and size of fruits on plants. However, occlusions from leaves typically make obtaining this information difficult. We present a novel viewpoint planning approach that builds up an octree of plants with labeled regions of interest (ROIs), i.e., fruits. Our method uses this octree to sample viewpoint candidates that increase the information around the fruit regions and evaluates them using a heuristic utility function that takes into account the expected information gain. Our system automatically switches between ROI targeted sampling and exploration sampling, which considers general frontier voxels, depending on the estimated utility. When the plants have been sufficiently covered with the RGB-D sensor, our system clusters the ROI voxels and estimates the position and size of the detected fruits. We evaluated our approach in simulated scenarios and compared the resulting fruit estimations with the ground truth. The results demonstrate that our combined approach outperforms a sampling method that does not explicitly consider the ROIs to generate viewpoints in terms of the number of discovered ROI cells. Furthermore, we show the real-world applicability by testing our framework on a robotic arm equipped with an RGB-D camera installed on an automated pipe-rail trolley in a capsicum glasshouse.

Plant phenotyping relevance

果実の位置・サイズという植物器官形質をRGB-Dセンサで取得・推定する視点計画法を開発し、シミュレーションと実環境で検証しており、フェノタイピング手法が中心である。

Code and data availability

The paper's viewpoint-planning system source code and the simulated capsicum plant environments used in the experiments are publicly available on GitHub. OctoMap is a generic third-party library, not a paper-specific asset.