Automated dynamic phenotyping of whole oilseed rape ( Brassica napus ) plants from images collected under controlled conditions.

Abstract

Introduction Recent advancements in sensor technologies have enabled collection of many large, high-resolution plant images datasets that could be used to non-destructively explore the relationships between genetics, environment and management factors on phenotype or the physical traits exhibited by plants. The phenotype data captured in these datasets could then be integrated into models of plant development and crop yield to more accurately predict how plants may grow as a result of changing management practices and climate conditions, better ensuring future food security. However, automated methods capable of reliably and efficiently extracting meaningful measurements of individual plant components (e.g. leaves, flowers, pods) from imagery of whole plants are currently lacking. In this study, we explore interdisciplinary application of MapReader, a computer vision pipeline for annotating and classifying patches of larger images that was originally developed for semantic exploration of historical maps, to time-series images of whole oilseed rape (Brassica napus) plants. Methods Models were trained to classify five plant structures in patches derived from whole plant images (branches, leaves, pods, flower buds and flowers), as well as background patches. Three modelling methods are compared: (i) 6-label multi-class classification, (ii) a chain of binary classifiers approach, and (iii) an approach combining binary classification of plant and background patches, followed by 5-label multi-class classification of plant structures. Results A combined plant/background binarization and 5-label multi-class modelling approach using a ‘resnext50d_4s2x40d’ model architecture for both the binary classification and multi-class classification components was found to produce the most accurate patch classification for whole B. napus plant images (macro-averaged F1-score = 88.50, weighted average F1-score = 97.71). This combined binary and 5-label multi-class classification approach demonstrate similar performance to the top-performing MapReader ‘railspace’ classification model. Discussion This highlights the potential applicability of the MapReader model framework to images data from across scientific and humanities domains, and the flexibility it provides in creating pipelines with different modelling approaches. The pipeline for dynamic plant phenotyping from whole plant images developed in this study could potentially be applied to imagery from varied laboratory conditions, and to images datasets of other plants of both agricultural and conservation concern.

Plant phenotyping relevance

植物全体画像から葉・花・莢などの構造を自動抽出・分類する動的フェノタイピング手法を開発・評価しており、方法が研究の中心である。

Code and data availability

The paper's phenotyping analysis is based on a public RGB image dataset of Brassica napus plants, explicitly deposited by the authors with a public URL. MapReader is a generic pre-existing library and the HuggingFace railspace models are cited prior work, not paper-specific assets.