Three-dimensional reconstruction and parameters extraction of walnut (Juglans regia L.) branches based on Neural Radiation Fields

Abstract

Three-dimensional (3D) reconstruction is important for obtaining morphological information and making intelligent management decisions for fruit trees. Thus, a method for the 3D reconstruction and parameters extraction of branches based on Neural Radiation Fields (NeRF) was proposed for walnut (Juglans regia L.) trees. This approach combined Structure from Motion (SfM) with NeRF and used multi-view images to reconstruct branches. First, a dataset of multi-view images of walnut trees was built and camera poses were obtained using SfM. Second, WalnutNeRF was optimized by incorporating hash encoding, piecewise sampler and appearance embedding features to address challenges associated with complex outdoor environments and accurately reconstruct branches. A scale recovery method using calibration objects was employed to extract branch parameters. The effectiveness of WalnutNeRF was evaluated by analyzing rendering performance, reconstruction efficiency, point cloud quality, and the accuracy of extracted branch parameters. WalnutNeRF outperformed existing methods in terms of the quality of rendered images and the accuracy of estimated depth, as determined using PSNR, SSIM, LPIPS, and other metrics. WalnutNeRF resulted in a branch reconstruction accuracy of 90.94 %, with a training time that was 9-time faster than that of SfM-MVS. Compared with SfM-MVS, WalnutNeRF decreased reconstruction errors for the main branches, lateral branches, and watershoots by 72 %, 67 %, and 57 %, respectively, and decreased the errors in length by 7.09 %, 4.33 %, and 65.07 %, respectively. Accordingly, WalnutNeRF decreased the reconstruction time, while increasing accuracy, providing robust support for the development of intelligent management applications (e.g., intelligent pruning) for walnut trees.

Plant phenotyping relevance

NeRFとSfMを用いてクルミ枝の3D形状を再構成し、枝パラメータを抽出・精度評価する手法が研究の中心であるため、植物フェノタイピング手法として含める。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.