Deep learning models for identifying crop and field attributes from near surface cameras

Abstract

Near surface cameras, such as those in the PhenoCam network, are a common source of ground truth data in modelling and remote sensing studies. Despite having locations across numerous agricultural sites, few studies have used near surface cameras to track the unique phenology of croplands. Due to management activities, crops do not have a natural vegetation cycle which many phenological extraction methods are based on. For example, a field may experience abrupt changes due to harvesting and tillage throughout the year. A single camera can also record several different plants due to crop rotations, fallow fields, and cover crops. Current methods to estimate phenology metrics from image time series compress all image information into a relative greenness metric, which discards a large amount of contextual information. This can include the type of crop present, whether snow or water is present on the field, the crop phenology, or whether a field lacking green plants consists of bare soil, fully senesced plants, or plant residue. Here we developed a modelling workflow to create a daily time series of crop type and phenology, while also accounting for other factors such as obstructed images and snow covered fields. We used a mainstream deep learning image classification model, VGG16. Deep learning classification models do not have a temporal component, so to account for temporal correlation among images our workflow incorporates a hidden markov model in the post-processing. The initial image classification model had out of sample F1 scores of 0.83-0.85, which improved to 0.86-0.91 after all post-processing steps. The resulting time series show the progression of crops from emergence to harvest, and can serve as a daily, local scale dataset of field states and phenological stages for agricultural research.

Plant phenotyping relevance

近接カメラ画像から作物種とフェノロジーを日次推定する深層学習・HMMワークフローを開発しており、植物の状態・生育段階の抽出が研究の中心である。

Code and data availability

The authors state that all analysis code, data, and final model predictions are publicly available in two Zenodo repositories (DOIs 10.5281/zenodo.5618316 and 10.5281/zenodo.5579797), directly reproducing this paper's PhenoCam crop phenology classification and HMM post-processing analysis.