PlantSAM: An object detection-driven segmentation pipeline for herbarium specimens.

Abstract

Premise Deep learning-based classification of herbarium images is hampered by background heterogeneity, which introduces noise and artifacts that can potentially mislead models and degrade their accuracy. Addressing these effects is essential to enhance overall performance. Methods We introduce PlantSAM, an automated segmentation pipeline that integrates YOLOv10 for plant region detection and the Segment Anything Model (SAM2) for segmentation. YOLOv10 generates bounding box prompts to guide SAM2, enhancing segmentation accuracy. Both models were fine-tuned on herbarium images and evaluated using intersection over union (IoU) and Sørensen-Dice coefficient metrics. Results PlantSAM achieved state-of-the-art segmentation performance, with an IoU of 0.94 and a Sørensen-Dice coefficient of 0.97. Incorporating segmented images into classification models led to consistent performance improvements across five tested botanical traits, with accuracy gains of up to 4.36% and F1 score improvements of 4.15%. Conclusions Our findings highlight the importance of background removal in herbarium image analysis, as it significantly enhances classification performance by enabling models to focus more effectively on the foreground plant structures.

Plant phenotyping relevance

植物画像から背景を除去して植物領域を抽出するセグメンテーション手法の開発・評価が中心であり、植物形質分類への有用性も検証している。

Code and data availability

The paper's data availability statement provides public GitHub repositories with segmentation source code, examples, and trained models, plus figshare DOIs for the segmentation dataset, the YOLOv10 plant region detection dataset, and the SAM fine-tuning/out-of-distribution dataset — all paper-specific and directly used