OSC-CO2: coattention and cosegmentation framework for plant state change with multiple features

Abstract

Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO2) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object’s pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs) to address segmentation accuracy in high-dimensional plant imagery with evolving plant objects. The efficacy of OSC-CO2 is demonstrated using plant growth sequences imaged with infrared, visible, and fluorescence cameras in multiple views using a remote sensing, high-throughput phenotyping platform, and is evaluated using Jaccard index and precision measures. We also introduce CosegPP+, a dataset that is structured and can provide quantitative information on the efficacy of our framework. Results show that OSC-CO2 out performed state-of-the art segmentation and cosegmentation methods by improving segementation accuracy by 3% to 45%.

Plant phenotyping relevance

植物画像から成長状態を抽出する画像セグメンテーション手法を開発し、ハイスループット表現型解析プラットフォームで評価しているため、方法が研究の中心である。

Code and data availability

The paper's authors publicly released both their analysis code (OSC-CO2 framework on GitHub) and the paper-specific plant phenotyping image dataset (CosegPP+, a VSTEM plant imagery dataset from the UNL LemnaTec platform) with explicit availability statements and URLs.