Individual Segmentation of Intertwined Apple Trees in a Row via Prompt Engineering.

Abstract

Computer vision is of wide interest to perform the phenotyping of horticultural crops such as apple trees at high throughput. In orchards specially constructed for variety testing or breeding programs, computer vision tools should be able to extract phenotypical information form each tree separately. We focus on segmenting individual apple trees as the main task in this context. Segmenting individual apple trees in dense orchard rows is challenging because of the complexity of outdoor illumination and intertwined branches. Traditional methods rely on supervised learning, which requires a large amount of annotated data. In this study, we explore an alternative approach using prompt engineering with the Segment Anything Model and its variants in a zero-shot setting. Specifically, we first detect the trunk and then position a prompt (five points in a diamond shape) located above the detected trunk to feed to the Segment Anything Model. We evaluate our method on the apple REFPOP, a new large-scale European apple tree dataset and on another publicly available dataset. On these datasets, our trunk detector, which utilizes a trained YOLOv11 model, achieves a good detection rate of 97% based on the prompt located above the detected trunk, achieving a Dice score of 70% without training on the REFPOP dataset and 84% without training on the publicly available dataset.We demonstrate that our method equals or even outperforms purely supervised segmentation approaches or non-prompted foundation models. These results underscore the potential of foundational models guided by well-designed prompts as scalable and annotation-efficient solutions for plant segmentation in complex agricultural environments.

Plant phenotyping relevance

リンゴ樹を個体別に画像分割し、育種・品種試験向けの表現型情報抽出を可能にする手法を開発・評価しており、植物フェノタイピング手法が中心である。

Code and data availability

The paper's apple REFPOP image dataset (RGB orchard images with manual tree/trunk annotations used for the phenotyping segmentation task) is publicly deposited on Zenodo via DOI 10.57745/DZBMAM, stated in both the Supplementary Materials and Data Availability Statement. Other URLs (Ultralytics, FrontVeg, arXiv) are for