Contrastive multi-view representation learning for multi-camera plant phenotyping: A cotton field study

Abstract

Attempts to deploy computer vision in agricultural tasks often suffer from a shortage of annotated data. One strategy to alleviate the impact of limited data is Self-Supervised Learning (SSL), which involves pre-training a model on a pretext task that utilizes automatically generated annotations. The primary objective of this study is to leverage a multi-camera view dataset of cotton boll images for contrastive learning in order to enable phenotyping tasks with minimal data annotation. This dataset was collected in the field using six camera views. The efficacy of two contrastive learning frameworks (SimCLR and MoCo) in producing representations when positive examples originate from different cameras was investigated, and a comprehensive study of how the camera positions affect performance was conducted. After self-supervised pre-training, linear evaluation and semi-supervised learning experiments were performed on boll detection and plot status downstream tasks. In general, using multiple camera views with SimCLR and MoCo improves cotton boll detection mean average precision by 14% compared to vanilla SimCLR and MoCo. Through careful investigation using synthetic data, it was determined that relative camera poses with an intermediate amount of overlap seem more likely to perform well. Neither MoCo nor SimCLR was consistently superior to the other in this context. The representations embed meaningful features about the cotton plants, such as overall boll density, but also less meaningful ones, such as lighting variations. This technique could potentially accelerate the development of phenotyping algorithms based on data collected from field robots. • A contrastive learning method based on comparing multi-camera views was developed. • The method was tested with images of cotton bolls from a ground robot. • The method outperformed baseline contrastive learning approaches.

Plant phenotyping relevance

マルチカメラ画像とコントラスト学習による植物表現学習・フェノタイピング手法の開発と評価が中心であり、綿花のボール検出性能を検証している。

Code and data availability

The paper's data availability statement points to a public GitHub repository containing the authors' code to reproduce the multi-camera contrastive learning phenotyping experiments. A processed-data Zenodo deposit (10.5281/zenodo.18164649) is also mentioned, but its URL is not among the allowed URLs, so only the code资产