3D plant segmentation: Comparing a 2D-to-3D segmentation method with state-of-the-art 3D segmentation algorithms

Abstract

Plant measurements are crucial to determine which plants grow optimal under certain conditions. These measurements can be done by hand, or automated using cameras, also known as image-based plant phenotyping. These images can be used to create point clouds to measure plant traits in 3D. To extract plant traits, accurate segmentation is crucial. Most point cloud segmentation methods rely on 3D segmentation algorithms. These algorithms are not as advanced and developed as 2D algorithms. In addition, 2D neural networks are pre-trained on large diverse datasets. In our work, it was therefore hypothesised that segmentation of point clouds using projection-based methods can obtain a higher accuracy than voxel or point-based algorithms. To test this hypothesis, a 2D-to-3D reprojection method was developed and compared with three state-of-the-art 3D segmentation algorithms; Swin3D-s, Point Transformer v3 and MinkUNet34C. The 2D-to-3D method segmented images using Mask2Former, reprojected the predictions to the point cloud, and used a majority vote algorithm to merge multiple predictions. All algorithms were trained and tested to segment 3D point clouds into leaves, main stem, side stem, and pole. There was no significant difference between the 2D-to-3D, Swin3D-s and Point Transformer v3 algorithm, indicating that state-of-the-art voxel or point-based methods perform similar than our projection-based method. However, the 2D-to-3D method had a higher performance by including virtual cameras and it had a higher training efficiency. With only five annotated plants, a similar performance was obtained than training Swin3D-s on 25 plants indicating the added value of the developed pipeline.

Plant phenotyping relevance

植物の3D形態計測を目的とした点群セグメンテーション手法を開発し、既存アルゴリズムと比較評価しており、表現型取得ワークフローが研究の中心です。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.