RGB-to-synthetic-thermal image translation using generative AI to support crop water stress assessment

Abstract

Thermal imaging is becoming a valuable tool for monitoring plant canopy temperature, which can serve as an indicator of crop water stress. However, specialized thermal sensors are often cost-prohibitive. This study explored strategies for supplementing crop water stress monitoring by generating synthetic thermal images from standard Red-Green-Blue (RGB) imagery captured using an unmanned aerial vehicle system (UAVs) equipped with a Zenmuse XT2 sensor and leveraging deep learning models. UAV-based RGB and thermal images were collected from 32 experimental plots of sweet corn and green beans over three growing seasons from 2020 to 2023. Each crop was subjected to one full and three deficit irrigation treatments, replicated four times. A total of 3,400 UAV images were collected over three seasons. Image processing was done in Pix4D software, and orthomosaic RGB and thermal maps were spatially aligned using ground control points (GCPs). The UAV RGB and thermal map data were split into 80 % and 20 % for training and testing, respectively. Two image-to-image translation generative adversarial network (GAN) deep learning models, specifically Pix2PixGAN and CycleGAN, were used to generate synthetic thermal images from RGB inputs. Image quality evaluation metrics, i.e., correlation coefficients (r), mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM), were used to evaluate the models’ performance. Crop water stress index (CWSI) values were also computed from measured and generated thermal imageries to assess practical applicability. Generated thermal canopy temperature outputs from the Pix2PixGAN model showed a strong correlation with the measured data using a thermal camera (r >0.95). Moreover, Pix2PixGAN resulted in lower MSE (5.63) and higher PSNR (42.98) than CycleGAN (MSE = 7.09, PSNR = 40.56), whereas CycleGAN had a slightly higher SSIM (0.44) than Pix2PixGAN (0.31). CWSI values derived from the generated thermal images reflected the expected gradients of water stress across irrigation treatments, with the highest CSWI observed from deficit irrigation treatments compared to the full irrigation. These results demonstrate that RGB-to-synthetic-thermal image translation using GAN models could be used to support crop water stress assessment and irrigation scheduling.

Plant phenotyping relevance

RGB画像から合成熱画像を生成し、作物キャノピー温度と水ストレス指標を推定するGAN手法の開発・比較・性能評価が中心であり、植物状態の表現型取得に直接関係する。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.