Research on automatic 3D reconstruction of plant phenotype based on Multi-View images

Abstract

Three-dimensional reconstruction plays a crucial role in quantifying crop phenotypes and exploring crop physiological structures. This paper presents a phenotyping platform designed for the 3D reconstruction of complex plants, utilizing multi-view images and introducing a joint evaluation criterion for both the reconstruction algorithm and the platform. Initially, a device composed of Raspberry Pi camera, SSH protocol, USB-TTL, motorized turntable, and shadow booth is built for automated image acquisition and transmission. Then, a dataset containing carex cabbage and kale is created and trained based on U2-net to achieve precise image segmentation. After that, an improved structure from motion algorithm, named IVOP & AKAZE-SFM, and multi-view stereo algorithm are utilized for the fine-scale reconstruction of plants. Next, by combing color filtering and Euclidean clustering, a denoising algorithm is proposed to obtain clean point clouds of plants. Finally, a method for calibrating the size of the plant point cloud based on priori condition is proposed to solve the problem of point cloud deformation in reconstruction. The evaluation of image segmentation model resulted in a precision of 0.91, a recall of 0.972, an IOU of 0.943, and a maxFβ of 0.099. The proposed IVOP&AKAZE-SFM is assessed against mainstream algorithms, the results show that our algorithm has the minimum average track length, minimum average reprojection error and generated the most points. The correlation coefficient (R2) between the extracted traits and measured phenotype, such as plant height and plant width, are 0.999 and 1.000, while the root means squared errors (RMSE) are 0.298 cm and 0.338 cm. Consequently, the platform offers a cost-effective, automated, and integrated solution for fine-scale plant 3D reconstruction.

Plant phenotyping relevance

植物のマルチビュー画像から3D形状を再構成し、草丈・株幅などの表現型を抽出するプラットフォームの開発、評価、検証が中心であるため。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.