Low-cost monocular RGB-based 3D structural mapping for horticultural plants via semantic scene completion

Abstract

Precision agriculture increasingly relies on detailed structural information, such as canopy height and canopy volume, to enhance crop health monitoring and operational safety. However, existing methods based on costly LiDAR or RGB-D sensors are often impractical for large-scale deployment in dynamic and unstructured horticultural environments. Furthermore, conventional 2D segmentation and SLAM-based pipelines typically generate sparse, geometrically inconsistent semantic maps which are insufficient for actionable structural analysis in agricultural applications. To overcome these limitations, we propose a monocular 3D structural mapping framework tailored for horticultural plants via semantic scene completion. At inference, the proposed model takes a single RGB image as input and predicts voxel-wise geometry and semantics, from which task-oriented structural maps, including canopy height, canopy volume, and obstacle-aware traversability layers, are derived. Specifically, we first introduce a Depth-Aware Decoder Module that explicitly recovers depth in the spatial domain and fuses 2D-to-3D features, thereby mitigating depth ambiguity and reducing reliance on accurate pose. Second, an NCS-Guided Geometry Encoder is designed to inject normalized depth into voxel positional embeddings, enabling self-attention to perform global relational modeling within a depth-aware geometric coordinate system. In addition, a Global Encoder is utilized to refine local structural details, while an occupancy head produces the final 3D semantic completion outputs. We construct a horticultural 3D semantic scene dataset using an RGB-D sensor, which serves as a benchmark for evaluating our method, while the deployed model remains RGB-only. Extensive quantitative and qualitative experiments are conducted on both the Semantic-KITTI dataset and our dataset. On our dataset, the method achieves 82.31% occupancy IoU, 84.26% mIoU, and 86.25% precision. Beyond voxel-level evaluation, manual field measurements further show canopy height MAE values of 0.019-0.026 m and canopy volume proxy relative errors of 8.4%-11.4%. These results demonstrate the effectiveness of our approach in real-world agricultural scenarios, providing actionable structural insights for crop monitoring and autonomous robotic operations.

Plant phenotyping relevance

単眼RGB画像から植物の樹冠高・樹冠体積などの構造形質を推定する3Dフェノタイピング手法を開発し、データセット構築と実測検証も行っているため、方法が研究の中心である。

Code and data availability

The supplied blocks describe a horticultural SSC dataset and trained model, but contain no data or code availability statement, no public repository, and no authors' URL for any paper-specific asset. The only URLs present are the article DOI and citations to prior work (AAAI papers, OccFormer), which are not paper-phen

No evidence-backed public reproduction asset is currently recorded.