P3DFusion: A cross-scene and high-fidelity 3D plant reconstruction framework empowered by vision foundation models and 3D Gaussian splatting

Abstract

Efficient acquisition of 3D plant structures is crucial for investigating growth mechanisms and phenotype analysis. Traditional 3D reconstruction methods exhibited significant limitations when faced with complex background interference, leading to low reconstruction efficiency and compromised result integrity. To address these challenges, a cross-scene 3D plant reconstruction framework P3DFusion was proposed with two key technological modules: (1) GSAM2 multi-view image processing method with Vision Foundation Models, which combines Grounding DINO and Segment Anything Model 2 (SAM2) to achieve high-precision plant segmentation under zero-shot conditions; (2) High-fidelity modeling based on 3D Gaussian splatting (3DGS) to generate high-quality, measurable meshes optimized for plant structural analysis. We evaluated P3DFusion using two datasets: Dataset1 (greenhouse-potted plants) and Dataset2 (open-field sugar beets). The P3DFusion exhibited significant improvements in reconstruction efficiency (SfM-Time reductions of 8.5 %/47.9 %, Total processing time reductions of 60.9 %/65.2 %) and quality metrics (SSIM increases of 12.9 %/19.8 %, PSNR increases of 11.8 %/13 % reaching 24.26 dB/24.75 dB, and LPIPS reductions of 70 %/85 %) for Dataset 1 and Dataset 2, respectively, compared to the original 3DGS. The P3DFusion outperforms InstantNGP (PSNR: 22.3 %/32.9 % increase) and COLMAP (PSNR: 231.4 %/266.1 % increase). Phenotype trait extraction from reconstructed models shows strong consistency with ground truth measurements (R² > 0.93). The proposed method not only provides an effective solution for cross-scene 3D plant reconstruction but also establishes a robust technical foundation for advanced plant phenotype research.

Plant phenotyping relevance

植物の3D再構成・セグメンテーションと形質抽出を中核とする手法開発および比較検証であり、植物フェノタイピング手法として明確に該当する。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.