Attention-driven next-best-view planning for efficient reconstruction of plants and targeted plant parts

Abstract

Robots in tomato greenhouses need to perceive the plant and plant parts accurately to automate monitoring, harvesting, and de-leafing tasks. Existing perception systems struggle with the high levels of occlusion in plants and often result in poor perception accuracy. One reason for this is because they use fixed cameras or predefined camera movements. Next-best-view (NBV) planning presents an alternate approach, in which the camera viewpoints are reasoned and strategically planned such that the perception accuracy is improved. However, existing NBV-planning algorithms are agnostic to the task-at-hand and give equal importance to all the plant parts. This strategy is inefficient for greenhouse tasks that require targeted perception of specific plant parts, such as the perception of leaf nodes for de-leafing. To improve targeted perception in complex greenhouse environments, NBV planning algorithms need an attention mechanism to focus on the task-relevant plant parts. In this paper, the role of attention in improving targeted perception using an attention-driven NBV planning strategy was investigated. Through simulation experiments using plants with high levels of occlusion and structural complexity, it was shown that focusing attention on task-relevant plant parts can significantly improve the speed and accuracy of 3D reconstruction. Further, with real-world experiments, it was shown that these benefits extend to complex greenhouse conditions with natural variation and occlusion, natural illumination, sensor noise, and uncertainty in camera poses. The results clearly indicate that using attention-driven NBV planning in greenhouses can significantly improve the efficiency of perception and enhance the performance of robotic systems in greenhouse crop production.

Plant phenotyping relevance

植物および植物部位の3D再構成を対象に、注意機構付き次善視点計画を開発し、シミュレーションと実環境で精度・速度を検証しているため、植物フェノタイピング手法が中心である。

Code and data availability

The supplied blocks describe simulation and real-world greenhouse NBV-planning experiments (tomato plant models, RGB-D data of seven plants, view planners), but contain no data or code availability statement, no public repository, and no author-hosted URL for the paper's datasets, plant models, or analysis code. The G(

No evidence-backed public reproduction asset is currently recorded.