Depth4PH: a vision foundation model-based framework for plant height estimation in agricultural scenes

Abstract

Plant height is a key 3D phenotypic trait for assessing crop growth, biomass accumulation, and lodging resistance. To overcome the practical limitations of conventional plant height measurement methods, this study proposes a novel vision foundation model-based framework named Depth for Plant Height (Depth4PH) for plant height estimation in agricultural scenes using low-cost monocular RGB imaging. As part of our contributions, a synthetic–real coupled multimodal dataset was constructed by integrating Blender virtual agricultural scenes (Blender VAS) with real field images. Building upon the existing Depth Anything V2 foundation model, we developed a novel module called Transfer-based Agricultural Metric Depth Anything V2 (TAM-Depth V2) for absolute metric depth estimation through parameter-efficient fine-tuning, depth decoder reconstruction, and joint loss optimization. Furthermore, we designed a novel multi-source prompt-based segmentation framework, MSP-SAM2, to generate positive and negative prompts for zero-shot crop instance segmentation. Finally, a new inverse physical plant height estimation algorithm, RANSAC-Per, was introduced to estimate plant height by combining truncated percentile statistics with local RANSAC micro-plane fitting, thereby reducing the effects of depth noise and field microtopographic variation. The result showed that TAM-Depth V2 achieved stable absolute depth estimation, with an RMSE of 0.1162 m and an AbsRel of 4.25%. Compared to the original box-prompted SAM 2, MSP-SAM2 achieved a 4.4% improvement in mIoU, reaching 91.6% and a recall of 93.2%. On a 350-plant multi-crop test set, Depth4PH achieved R 2 = 0.948, RMSE = 12.23 cm, and MAE = 8.82 cm, and MAPE =10.15%, with crop-specific RMSEs ranging from 3.99 cm (cucumber) to 20.73 cm (maize), significantly outperforming the traditional Global-MinMax baseline (which had an RMSE of 22.62 cm). These results indicate that Depth4PH provides a promising foundational pathway for high-throughput crop phenotyping. With future optimization for edge deployment, it holds significant potential to support high-throughput monitoring in precision agriculture.

Plant phenotyping relevance

植物高の画像取得・深度推定・セグメンテーション・高さ抽出アルゴリズムを一体化した植物表現型計測フレームワークの開発と検証が中心であり、データセット構築と性能評価も含む。

Code and data availability

The paper's phenotyping assets (Blender VAS synthetic RGB-D dataset, the 17,938-image synthetic–real depth dataset, the 350-plant physical height test set, and TAM-Depth V2/MSP-SAM2 models) are not publicly deposited; the authors state data are available only on request. No public code or dataset URL is provided.

No evidence-backed public reproduction asset is currently recorded.