Research on binocular stereo vision phenotyping measurement for leafy vegetable based on 3DGS supervision

Abstract

Accurate non-destructive measurement of phenotypic data for leafy vegetable is critical for effective management of breeding, growth monitoring, and yield estimation. However, the scarcity of large-scale stereo matching datasets in agricultural scenarios and the high cost of depth-sensing devices tailored for small targets pose significant challenges to achieving accurate phenotyping of crops in complex growing environments. This study introduces a novel 3D Gaussian Splatting (3DGS)-supervised stereo matching network to enable accurate depth estimation of leafy vegetable. We enhance Gaussian rendering techniques to generate stereo training data from multi-view images. A multi-scale feature extraction network with deformable convolutions is proposed to adaptively capture diverse features, ranging from low-level details (edges and textures) to structural and semantic information. To address the multi-peak characteristics of disparity in leafy vegetable, a top- k algorithm is employed for disparity regression, significantly improving disparity estimation accuracy. To enhance the adaptability of cost aggregation, we construct a combined cost volume through a concatenation and group-wise correlations and leverage feature attention blocks to refine disparity prediction. Additionally, depthwise separable convolutions and optimized methods of the number of iterative blocks for leafy vegetable small targets are introduced to reduce computational complexity and accelerate training and inference. Experimental results demonstrate the effectiveness of our depth estimation method, achieving an Endpoint Error (EPE) of 2.32 pixels overall and 0.91 pixels specifically in leafy vegetable area compared to disparity ground truth. Furthermore, we developed a 3D point cloud phenotyping measurement method using instance segmentation and 3D reconstruction. Notably, the measured height, number of leaves, and surface area achieve R 2 values of 0.9946, 0.9577, and 0.9775 against manual measurements, respectively. These high-precision results underscore the practical applicability of our method and provide a theoretical foundation for deployment in agricultural production systems.

Plant phenotyping relevance

葉菜の深度推定と3D点群による形質測定法を開発し、手動測定との比較で高さ・葉数・表面積を検証しており、フェノタイピング手法が研究の中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.