Temporal image sandwiches enable link between functional data analysis and deep learning for single-plant cotton senescence

Abstract

Abstract Abstract. Senescence is a highly ordered biological process involving resource redistribution away from ageing tissues that affects yield and quality in annuals and perennials. Images from 14 unmanned/unoccupied/uncrewed aerial system/vehicle (UAS, UAV and drone) flights captured the senescence window across two experiments while functional principal component analysis effectively reduced the dimensionality of temporal visual senescence ratings (VSRs) and two vegetation indices: the red chromatic coordinate (RCC) index and the transformed normalized difference green and red (TNDGR) index. Convolutional neural networks trained on temporally concatenated, or ‘sandwiched’, UAS images of individual cotton plants (Gossypium hirsutum L.), allowed single-plant analysis. The first functional principal component scores (FPC1) served as the regression target across six CNN models (M1–M6). Model performance was strongest for FPC1 scores from VSRs (R2 = 0.857 and 0.886 for M1 and M4), strong for TNDGR (R2 = 0.743 and 0.745 for M3 and M6), and strong-to-moderate for RCC index (R2 = 0.619 and 0.435 for M2 and M5), with deep learning attention of each model confirmed by activation of plant pixels within saliency maps. Single-plant UAS image analysis across time enabled translatable implementations of high-throughput phenotyping by linking deep learning with functional data analysis. This has applications for fundamental plant biology, monitoring orchards or other spaced plantings, plant breeding, and genetic research.

Plant phenotyping relevance

単一個体の綿花について、時系列UAS画像とCNN・機能的データ解析を用いて老化表現型を推定する方法が研究の中心であり、性能評価も実施している。

Code and data availability

The paper's DATA AVAILABILITY section states that all code for FPCA, ANOVA and CNN regression, plus all files needed to run the scripts including the raw single-plant UAS images, are publicly available in the authors' GitHub repository. This is a paper-specific, public, actionable asset covering both the phenotyping (c