CLCFM3: A 3D Reconstruction Algorithm Based on Photogrammetry for High-Precision Whole Plant Sensing Using All-Around Images

Abstract

This research aims to develop a novel technique to acquire a large amount of high-density, high-precision 3D point cloud data for plant phenotyping using photogrammetry technology. The complexity of plant structures, characterized by overlapping thin parts such as leaves and stems, makes it difficult to reconstruct accurate 3D point clouds. One challenge in this regard is occlusion, where points in the 3D point cloud cannot be obtained due to overlapping parts, preventing accurate point capture. Another is the generation of erroneous points in non-existent locations due to image-matching errors along object outlines. To overcome these challenges, we propose a 3D point cloud reconstruction method named closed-loop coarse-to-fine method with multi-masked matching (CLCFM3). This method repeatedly executes a process that generates point clouds locally to suppress occlusion (multi-matching) and a process that removes noise points using a mask image (masked matching). Furthermore, we propose the closed-loop coarse-to-fine method (CLCFM) to improve the accuracy of structure from motion, which is essential for implementing the proposed point cloud reconstruction method. CLCFM solves loop closure by performing coarse-to-fine camera position estimation. By facilitating the acquisition of high-density, high-precision 3D data on a large number of plant bodies, as is necessary for research activities, this approach is expected to enable comparative analysis of visible phenotypes in the growth process of a wide range of plant species based on 3D information.

Plant phenotyping relevance

植物フェノタイピングのためのフォトグラメトリ画像から高精度3D点群を再構成する手法を開発しており、表現型取得法が研究の中心である。

Code and data availability

The authors explicitly deposit the MMM/CLCFM analysis code and scripts in a public GitHub repository, which also provides download links to the supporting image and 3D point cloud data used in this paper's soybean phenotyping reconstructions. The supplementary material contains only result figures, not datasets. Gene/N