Segment Anything for Comprehensive Analysis of Grapevine Cluster Architecture and Berry Properties.

Abstract

Grape cluster architecture and compactness are complex traits influencing disease susceptibility, fruit quality, and yield. Evaluation methods for these traits include visual scoring, manual methodologies, and computer vision, with the latter being the most scalable approach. Most of the existing computer vision approaches for processing cluster images often rely on conventional segmentation or machine learning with extensive training and limited generalization. The Segment Anything Model (SAM), a novel foundation model trained on a massive image dataset, enables automated object segmentation without additional training. This study demonstrates out-of-the-box SAM's high accuracy in identifying individual berries in 2-dimensional (2D) cluster images. Using this model, we managed to segment approximately 3,500 cluster images, generating over 150,000 berry masks, each linked with spatial coordinates within their clusters. The correlation between human-identified berries and SAM predictions was very strong (Pearson's r 2 = 0.96). Although the visible berry count in images typically underestimates the actual cluster berry count due to visibility issues, we demonstrated that this discrepancy could be adjusted using a linear regression model (adjusted R 2 = 0.87). We emphasized the critical importance of the angle at which the cluster is imaged, noting its substantial effect on berry counts and architecture. We proposed different approaches in which berry location information facilitated the calculation of complex features related to cluster architecture and compactness. Finally, we discussed SAM's potential integration into currently available pipelines for image generation and processing in vineyard conditions.

Plant phenotyping relevance

SAMを用いたブドウ房画像からの個別果粒セグメンテーション、検証、補正、および房構造・コンパクトネス形質の算出が研究の中心であるため。

Code and data availability

The authors state that all data and code to reproduce the study's grapevine cluster segmentation and architecture analysis are publicly available in their GitHub repository. Other URLs (SAM checkpoint, pycocotools, RMBG, arXiv refs) are generic third-party resources, not paper-specific assets.