PhenoVision: A framework for automating and delivering research-ready plant phenology data from field images

Abstract

Plant phenology plays a fundamental role in shaping ecosystems, and global change-induced shifts in phenology have cascading impacts on species interactions and ecosystem structure and function. Detailed, high-quality observations of when plants undergo seasonal transitions such as leaf-out, flowering, and fruiting are critical for tracking causes and consequences of phenology shifts, but these data are often sparse and biased globally. These data gaps limit broader generalizations and forecasting improvements in the face of continuing disturbance. One solution to closing such gaps is to document phenology on field images taken by public participants. iNaturalist, in particular, provides global scale research-grade data and is expanding rapidly. Here we utilize over 53 million field images of plants and millions of human annotations from iNaturalist – data spanning all angiosperms and drawn from across the globe – to train a computer vision model (PhenoVision) to detect the presence of fruits and flowers. PhenoVision utilizes a vision transformer architecture pretrained with a masked autoencoder to improve classification success, and it achieves high accuracy for flower (98.5%) and fruit presence (95%). Key to producing research-ready phenology data is post-calibration tuning and validation focused on reducing noise inherent in field photographs, and maximizing the true positive rate. We also develop a standardized set of quality metrics and metadata so that results can be used effectively by the community. Finally, we showcase how this effort vastly increases phenology data coverage, including regions of the globe where data have been limited before. Our end products are tuned models, new data resources, and an application streamlining discovery and use of those data for the broader research and management community. We close by discussing next steps, including automating phenology annotations, adding new phenology targets, e.g., leaf phenology, and further integration with other resources to form a global central database integrating all in-situ plant phenology resources.

Plant phenotyping relevance

植物の野外画像から開花・結実というフェノロジー形質を推定する画像解析モデルを開発し、校正・検証・品質指標も扱っており、フェノタイピング手法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.