Point clouds segmentation of rapeseed siliques based on sparse-dense point clouds mapping

Abstract

In this study, we propose a high-throughput and low-cost automatic detection method based on deep learning to replace the inefficient manual counting of rapeseed siliques. First, a video is captured with a smartphone around the rapeseed plants in the silique stage. Feature point detection and matching based on SIFT operators are applied to the extracted video frames, and sparse point clouds are recovered using epipolar geometry and triangulation principles. The depth map is obtained by calculating the disparity of the matched images, and the dense point cloud is fused. The plant model of the whole rapeseed plant in the silique stage is reconstructed based on the structure-from-motion (SfM) algorithm, and the background is removed by using the passthrough filter. The downsampled 3D point cloud data is processed by the DGCNN network, and the point cloud is divided into two categories: sparse rapeseed canopy siliques and rapeseed stems. The sparse canopy siliques are then segmented from the original whole rapeseed siliques point cloud using the sparse-dense point cloud mapping method, which can effectively save running time and improve efficiency. Finally, Euclidean clustering segmentation is performed on the rapeseed canopy siliques, and the RANSAC algorithm is used to perform line segmentation on the connected siliques after clustering, obtaining the three-dimensional spatial position of each silique and counting the number of siliques. The proposed method was applied to identify 1457 siliques from 12 rapeseed plants, and the experimental results showed a recognition accuracy greater than 97.80%. The proposed method achieved good results in rapeseed silique recognition and provided a useful example for the application of deep learning networks in dense 3D point cloud segmentation.

Plant phenotyping relevance

スマートフォン映像と3D点群、深層学習・幾何処理を用いてナタネの莢を自動検出・計数する手法が研究の中心であり、植物形態形質の取得方法を技術的に検証している。

Code and data availability

The supplied blocks describe the rapeseed silique point cloud dataset, DGCNN training data, and segmentation pipeline, but contain no data availability statement, no public repository deposit, no author-provided URL for code, models, or point cloud data, and no allowed_urls are provided. The only mentioned tool (Cloud-

No evidence-backed public reproduction asset is currently recorded.