UAV-borne RGB image and LiDAR fusion for reconstruction of 3D simulated hyperspectral data for crop growth parameter estimation

Abstract

Under the dual pressures of food security and sustainable agricultural development, rapid and simultaneous detection of multiple crop growth parameters has become a core technological requirement for optimizing field management and improving resource utilization efficiency. UAVs carrying one or more sensors to collect of different crop growth parameters have achieved remarkable results in the field of single morphological or physiological parameter analysis. However, existing low-cost devices often failed to collect 3D geometric data and high-resolution spectral information simultaneously in field conditions, while the different nature and data structure of point cloud and spectral data brought special challenges to data fusion, restricting the ability of simultaneous multi-parameter resolution. Facing such challenges, in this paper, we design and develop a system that can take into account the simultaneous acquisition of 3D geometric data and high-resolution spectral information in field conditions through multi-sensor fusion and deep learning algorithm innovation. Based on a mature color point cloud data structure, we combine RGB cameras and laser radar sensors to fuse RGB images and point cloud data. We improved a spectral reconstruction network, construct a dedicated chlorophyll response sensitive band dataset for training, and reconstructed hyperspectral images with 36 channels in the 500–850 nm band range from RGB images, which greatly reduces the cost of the spectral information acquisition device. Experiments show that the SAM (Spectral Angle Mapper) value between the reconstructed hyperspectral data and the original hyperspectral data is less than 0.03. Finally, the growth parameters of crops are estimated using spectral and point cloud data. The developed equipment was calibrated and tested, and experimental data were collected under real field conditions for plant height (PH), leaf area index (LAI), and chlorophyll content estimation. The experimental results showed that the system could accurately analyze maize PH and LAI with Rt2 of 0.98 and 0.97, respectively, and that the chlorophyll content analysis capability was at the same level as that of other studies that have used UAV-mounted hyperspectral cameras for leaf chlorophyll content (LCC) detection, and the established estimation model Rt2 reached 0.66. The canopy chlorophyll content (CCC) of maize could be accurately estimated by fusing the data, and the Rt2 reached 0.95.

Plant phenotyping relevance

RGB画像・LiDAR融合と深層学習による3D形状およびハイパースペクトル情報の再構成システムを開発し、圃場で植物形質推定を校正・検証しており、フェノタイピング手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.