FSDNET: A features spreading net with density for 3D segmentation in agriculture

Abstract

The accurate segmentation of fruit phenotypes in the field is of great significance for agricultural automation in the 3D scene. Although the existing fruit segmentation based on 3D point cloud has made great progress, in the complex field environment, due to lighting, leaf occlusion, shooting angle and other problems, the point cloud obtained by depth camera often has the problem of multiple voids and discrete points, which seriously affects the accurate segmentation of fruit phenotype. This paper proposes a embedding subnetwork FSDnet based on density-based feature extraction and feature propagation and embeds it in the novel segmentation networks, which effectively improves the segmentation accuracy of the point cloud phenotype in multi-hole and multi-discrete fruits, including (1) The density-based point cloud feature extraction and feature propagation theory is proposed to alleviate the problem of perception degradation in fruit edge point caused by discrete points and holes caused by imcomplete point cloud in the agriculture scene. (2) A density-adaptive embedding semantic segmentation framework FSDnet is proposed, and embedding the classical point cloud neural network can significantly improve the segmentation accuracy of the fruit phenotypes with multiple holes and discrete points in the traditional network. (3) This paper made a strawberry dataset and tested the designed new neural network on both strawberry and apple filed dataset. After FSDnet is embedded on different novel net, almost all net have been improved. We verified the performance of FSDnet in different density states in agricultural scenarios, mitigated the negative impact of density on segmentation accuracy, proving that it can adapt to different point cloud density in agricultural scenarios in comparison between Gaussian density and other two traditional density schemes, Gaussian density reduces the computational traffic (0.58G) of the network while maintaining similar performance to the other two densities, proving the superiority of assuming a Gaussian density.

Plant phenotyping relevance

3D点群から果実表現型を分割・抽出する手法を開発し、イチゴ・リンゴデータセットで性能検証しているため、植物フェノタイピング手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.