Branching Gaussian Processes with Applications to Spatiotemporal\n Reconstruction of 3D Trees

Abstract

We propose a robust method for estimating dynamic 3D curvilinear branching structure from monocular images. While 3D reconstruction from images has been widely studied, estimating thin structure has received less attention. This problem becomes more challenging in the presence of camera error, scene motion, and a constraint that curves are attached in a branching structure. We propose a new general-purpose prior, a branching Gaussian processes (BGP), that models spatial smoothness and temporal dynamics of curves while enforcing attachment between them. We apply this prior to fit 3D trees directly to image data, using an efficient scheme for approximate inference based on expectation propagation. The BGP prior's Gaussian form allows us to approximately marginalize over 3D trees with a given model structure, enabling principled comparison between tree models with varying complexity. We test our approach on a novel multi-view dataset depicting plants with known 3D structures and topologies undergoing small nonrigid motion. Our method outperforms a state-of-the-art 3D reconstruction method designed for non-moving thin structure. We evaluate under several common measures, and we propose a new measure for reconstructions of branching multi-part 3D scenes under motion.

Plant phenotyping relevance

植物の3D分枝構造を画像から推定する手法を開発し、植物データセットで既存手法との比較評価と新たな再構成指標の提案を行っており、フェノタイピング手法が中心である。

Code and data availability

The paper describes a novel multi-view plant dataset and evaluation code ('Our dataset and evaluation code may be downloaded from ivilab.org.'), which would be paper-specific phenotyping assets. However, that URL is not among the allowed_urls, so no actionable asset can be reported. The only allowed URL, http://www.vmt

No evidence-backed public reproduction asset is currently recorded.