In-Field 3D Wheat Head Instance Segmentation from TLS Point Clouds Using Deep Learning without Manual Labels

Abstract

Abstract. 3D instance segmentation for laser scanning (LiDAR) point clouds remains a challenge in many remote sensing-related domains. Successful solutions typically rely on supervised deep learning and manual annotations, and consequently focus on objects that can be well delineated through visual inspection and manual labeling of point clouds. However, for tasks with more complex and cluttered scenes, such as in-field plant phenotyping in agriculture, such approaches are often infeasible. In this study, we tackle the task of in-field wheat head instance segmentation directly from terrestrial laser scanning (TLS) point clouds. To address the problem and circumvent the need for manual annotations, we propose a novel two-stage pipeline. To obtain the initial 3D instance proposals, the first stage uses 3D-to-2D multi-view projections, the Grounded SAM pipeline for zero-shot 2D object-centric segmentation, and multi-view label fusion. The second stage uses these initial proposals as noisy pseudo-labels to train a supervised 3D panoptic-style segmentation neural network. Our results demonstrate the feasibility of the proposed approach and show performance improvements relative to Wheat3DGS, a recent alternative solution for in-field wheat head instance segmentation without manual 3D annotations based on multi-view RGB images and 3D Gaussian Splatting, showcasing TLS as a competitive sensing alternative. Moreover, the results show that both stages of the proposed pipeline can deliver usable 3D instance segmentation without manual annotations, indicating promising, low-effort transferability to other comparable TLS-based point cloud segmentation tasks.

Plant phenotyping relevance

圃場小麦穂の3D個体セグメンテーションを対象に、TLS点群とゼロショット・マルチビュー融合・擬似ラベル学習を組み合わせた表現型取得手法を開発・評価しており、方法が研究の中心である。

Code and data availability

植物フェノタイピング解析を再現する公開資産であることを、入力本文と直接リンクから確認できなかったため保留しました。

No evidence-backed public reproduction asset is currently recorded.