Organ3DNet: A deep network for segmenting organ semantics and instances from dense plant point clouds

Abstract

Worldwide food shortage has put the plant phenotyping research to the spotlight because phenotyping enhances crop yield under limited land use by accelerating the cycle of modern breeding. The prerequisite of organ-level phenotyping is the accurate segmentation of organs from crop point clouds. Though mainstream deep networks have reported satisfactory results on certain species, they usually require sampling each input point cloud to a fixed number before network learning. Most existing networks also recommend each input to contain less than 10,000 points, which may smooth the structural details and lead to deterioration of segmentation of small organs. Moreover, these methods are still struggling to face challenges such as segmenting crops with a large number of organs and scalability to multiple species. In this paper, we propose Organ3DNet—a novel deep-learning-based architecture tailored for organ segmentation on high-precision plant 3D data. By integrating a Sparse 3D Convolutional Network Backbone (S3DCNB) as encoder and a new Transformer Decoder part containing a cascade of Query Refinement Modules (QRM) and Mask Modules (MM), Organ3DNet begins with query points obtained with 3D Edge-preserving Sampling (3DEPS) and gradually refines those queries into masks to effectively represent different organ instances. A high-precision dataset containing 889 samples from five species is also provided in this study. In experiment on this dataset, our Organ3DNet outcompeted four networks including ASIS, JSNet, PlantNet, and PSegNet. On the organ semantic segmentation task, our method surpasses the second best JSNet by 2.10 % on F1 and 3.63 % on IoU; while on the instance segmentation task, Organ3DNet surpasses the second best PSegNet by large margins of 16.46 % on mCov and 13.44 % on mWCov, respectively. Validation tests also show that Organ3DNet performs well on downstream tasks and real agricultural scenarios. The dataset and code associated with Organ3DNet are open to the readers. • An open crop point cloud dataset including 5 different species with organ semantics and instance annotations. • Our Organ3DNet can perform organ-level semantic and instance segmentation tasks directly on dense 3D crops. • Organ3DNet integrates advanced computational modules, and outcompetes several state-of-the-art segmentation networks. • The segmented organs by Organ3DNet can be directly used for precise calculation of organ-level phenotypes.

Plant phenotyping relevance

植物3D点群から器官を分割し、器官レベル形質の計算に利用する深層学習手法とデータセットを開発・検証しており、フェノタイピング手法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.