PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage

Abstract

Segmentation of plant point clouds to obtain high-precise morphological traits is essential for plant phenotyping. Although the fast development of deep learning has boosted much research on segmentation of plant point clouds, previous studies mainly focus on the hard voxelization-based or down-sampling-based methods, which are limited to segmenting simple plant organs. Segmentation of complex plant point clouds with a high spatial resolution still remains challenging. In this study, we proposed a deep learning network plant segmentation transformer (PST) to achieve the semantic and instance segmentation of rapeseed plants point clouds acquired by handheld laser scanning (HLS) with the high spatial resolution, which can characterize the tiny siliques as the main traits targeted. PST is composed of: (i) a dynamic voxel feature encoder (DVFE) to aggregate the point features with the raw spatial resolution; (ii) the dual window sets attention blocks to capture the contextual information; and (iii) a dense feature propagation module to obtain the final dense point feature map. The results proved that PST and PST-PointGroup (PG) achieved superior performance in semantic and instance segmentation tasks. For the semantic segmentation, the mean IoU, mean Precision, mean Recall, mean F1-score, and overall accuracy of PST were 93.96%, 97.29%, 96.52%, 96.88%, and 97.07%, achieving an improvement of 7.62%, 3.28%, 4.8%, 4.25%, and 3.88% compared to the second-best state-of-the-art network PAConv. For instance segmentation, PST-PG reached 89.51%, 89.85%, 88.83% and 82.53% in mCov, mWCov, mPerc90, and mRec90, achieving an improvement of 2.93%, 2.21%, 1.99%, and 5.9% compared to the original PG. This study proves that the deep-learning-based point cloud segmentation method has a great potential for resolving dense plant point clouds with complex morphological traits.

Plant phenotyping relevance

ラッペシード植物の高解像度点群から形態形質を抽出するセグメンテーション手法を開発し、性能評価しているため、植物フェノタイピング手法が中心である。

Code and data availability

The paper's key asset is a fully annotated HLS rapeseed plant point cloud dataset (55 samples, semantic and instance labels), but the authors state it is only available upon request, and no public URL or repository is provided. No public code or model release is mentioned.

No evidence-backed public reproduction asset is currently recorded.