Segment Any Leaf 3D: A Zero-Shot 3D Leaf Instance Segmentation Method Based on Multi-View Images.

Abstract

Exploring the relationships between plant phenotypes and genetic information requires advanced phenotypic analysis techniques for precise characterization. However, the diversity and variability of plant morphology challenge existing methods, which often fail to generalize across species and require extensive annotated data, especially for 3D datasets. This paper proposes a zero-shot 3D leaf instance segmentation method using RGB sensors. It extends the 2D segmentation model SAM (Segment Anything Model) to 3D through a multi-view strategy. RGB image sequences captured from multiple viewpoints are used to reconstruct 3D plant point clouds via multi-view stereo. HQ-SAM (High-Quality Segment Anything Model) segments leaves in 2D, and the segmentation is mapped to the 3D point cloud. An incremental fusion method based on confidence scores aggregates results from different views into a final output. Evaluated on a custom peanut seedling dataset, the method achieved point-level precision, recall, and F1 scores over 0.9 and object-level mIoU and precision above 0.75 under two IoU thresholds. The results show that the method achieves state-of-the-art segmentation quality while offering zero-shot capability and generalizability, demonstrating significant potential in plant phenotyping.

Plant phenotyping relevance

植物の葉を対象とするマルチビュー3Dインスタンスセグメンテーション手法を開発し、植物フェノタイピング用途としてデータセット上で技術評価しているため、方法が研究の中心である。

Code and data availability

The paper's peanut seedling multi-view image dataset and analysis code are not publicly deposited; the Data Availability Statement states they are available only upon request. CloudCompare is a generic third-party tool, not a paper-specific asset.

No evidence-backed public reproduction asset is currently recorded.