End-to-end pipeline for simultaneous temperature estimation and super resolution of low-cost uncooled infrared camera frames for precision agriculture applications

Abstract

Radiometric infrared (IR) imaging is a valuable technique for remote-sensing applications in precision agriculture, such as irrigation monitoring, crop health assessment, and yield estimation. Low-cost uncooled non-radiometric IR cameras offer new implementations in agricultural monitoring. However, these cameras have inherent drawbacks that limit their usability, such as low spatial resolution, spatially variant nonuniformity, and lack of radiometric calibration. In this article, we present an end-to-end pipeline for temperature estimation and super resolution of frames captured by a low-cost uncooled IR camera. The pipeline consists of two main components: a deep-learning-based temperature-estimation module, and a deep-learning-based super-resolution module. The temperature-estimation module learns to map the raw gray level IR images to radiometric-grade temperature maps while also correcting for nonuniformity. The super-resolution module uses a deep-learning network to enhance the spatial resolution of the IR images by scale factors of ×2 and ×4. We evaluated the performance of the pipeline on both simulated and real-world agricultural datasets composing of roughly 20,000 frames of various crops. For the simulated data, the results were on par with the real-world data with sub-degree accuracy — 0.54∘C mean absolute error (MAE) for ×2 scale factor, and 0.84∘C MAE for ×4 scale factor. For the real data, the proposed pipeline was compared to a high-end radiometric thermal camera, and achieved sub-degree accuracy — 0.81∘C MAE for ×2 scale factor, and 0.81∘C MAE for ×4 scale factor. The results of the real data are on par with the simulated data. We show that our pipeline can compete with high-end thermal cameras in terms of quality and accuracy of the temperature and crop water stress index (CWSI) estimations using affordable hardware, with errors of 1.42% for ×2 and 1.86% for ×4 between the ground truth and the estimated CWSI. The runtime of the pipeline is less than 1sec per frame on a CPU, allowing it to run at video rates. The proposed pipeline can enable various applications in precision agriculture that require high quality thermal information from low-cost IR cameras.

Plant phenotyping relevance

低コスト赤外線カメラから植物温度と作物水ストレス指数を推定する深層学習パイプラインを開発し、実データ・シミュレーションおよび高性能熱画像カメラとの比較で精度を検証しており、植物フェノタイピング手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.