PlantSegNet: 3D point cloud instance segmentation of nearby plant organs with identical semantics

Abstract

In this study, we introduce PlantSegNet, a novel neural network model for instance segmentation of nearby objects with similar geometric structures. Our work addresses the challenges of instance segmentation of plant point clouds, including the difficulty of annotating and labeling point clouds, the loss of local structural information in neural network components, and the generation of large numbers of incorrect small clusters due to poor choices of the loss function. One of the key contributions of our approach is a digital twin of sorghum, i.e., a procedural sorghum model, which was used to generate point clouds of sorghum fields. This allowed us to create a large-scale, annotated, synthetic dataset of sorghum plants that we used to train our PlantSegNet model. We demonstrated the effectiveness of our method in segmenting instances of sorghum leaves grown in outdoor field settings. To the best of our knowledge, this is the first study to address this specific instance segmentation problem for plants grown in such a setting. We compared our proposed method with other state-of-the-art methods for indoor settings, including SGPN and TreePartNet, on both synthetic and real data. Our results show that PlantSegNet outperforms these methods regarding accuracy, robustness, and efficiency.

Plant phenotyping relevance

植物葉の点群から器官インスタンスを抽出するニューラルネットワークを開発し、合成データセット作成、実データでの比較検証まで行っており、植物表現型取得手法が中心である。

Code and data availability

The authors publicly release their PlantSegNet analysis code (PyTorch models and TreePartNet wrapper) together with their labeled synthetic and real sorghum point cloud datasets, and separately state the datasets (synthetic/real sorghum and Tree Dataset) are publicly available on their GitHub page.