Robotic self-supervised local view planning for efficient and scalable 3D plant reconstruction across varying plant sizes

Abstract

Accurate perception in complex agricultural environments is challenging due to significant plant occlusion, primarily from leaves, which hinder data collection and increase uncertainty in robotic operations. Deep-learning-based Next-Best-View (DL-NBV) methods address this by using neural networks to predict information gain (IG) for potential camera views and actively repositioning the camera to maximize data collection with minimal views. However, training DL-NBV models requires extensive IG-labeled data. A self-supervised learning-based NBV method, SSL-Global-NBV, enables robots to collect training data autonomously and improve themselves for global NBV planning. Despite its advantages, SSL-Global-NBV has two key limitations: (1) it requires a fixed number of views, limiting scalability across different plant sizes, and (2) it selects views globally, resulting in inefficient view transitions across the entire view space, reducing trajectory efficiency. To overcome these limitations, this paper introduces SSL-Local-NBV, which incorporates local view planning for scalable and efficient view selection. To prevent redundant visits to the same views, a View Trajectory Network (VTN) was proposed to memorize the view trajectory information of visited views. Comprehensive evaluations in simulation and real-world plant reconstruction demonstrated that SSL-Local-NBV reduced trajectory distance by 56%–70% per reconstruction cycle, achieving 267%–300% higher trajectory efficiency than global NBV methods. Compared to SSL-Global-NBV, SSL-Local-NBV improved plant reconstruction efficiency by 5.2% across varying plant sizes, demonstrating greater scalability. For real plants, SSL-Local-NBV achieved over 80% reconstruction, confirming its feasibility in practical applications. Notably, SSL-Local-NBV fully automated training through self-supervised learning, enabling continuous and lifelong robotic learning.

Plant phenotyping relevance

植物の3D再構成を効率化するロボット視点計画手法を開発・評価しており、植物形態の取得が中心的な技術貢献である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.