A graph-based approach for simultaneous semantic and instance segmentation of plant 3D point clouds

Abstract

Accurate simultaneous semantic and instance segmentation of a plant 3D point cloud is critical for automatic plant phenotyping. Classically, each organ of the plant is detected based on the local geometry of the point cloud, but the consistency of the global structure of the plant is rarely assessed. We propose a two-level, graph-based approach for the automatic, fast and accurate segmentation of a plant into each of its organs with structural guarantees. We compute local geometric and spectral features on a neighbourhood graph of the points to distinguish between linear organs (main stem, branches, petioles) and two-dimensional ones (leaf blades) and even 3-dimensional ones (apices). Then a quotient graph connecting each detected macroscopic organ to its neighbors is used both to refine the labelling of the organs and to check the overall consistency of the segmentation. A refinement loop allows to correct segmentation defects. The method is assessed on both synthetic and real 3D point-cloud data sets of Chenopodium album (wild spinach) and Solanum lycopersicum (tomato plant).

Plant phenotyping relevance

植物3D点群から器官を自動分割・識別するグラフベース手法を開発し、合成および実データで評価しており、植物表現型取得の技術が中心である。

Code and data availability

The paper's Chenopodium 3D point cloud dataset (with ground truth annotations) is publicly deposited on Zenodo, and the reconstruction pipeline code is open source on GitHub (romi/plant-3d-vision).