A scalable and efficient UAV-based pipeline and deep learning framework for phenotyping sorghum panicle morphology from point clouds

Abstract

Sorghum canopy architecture in field trials is determined by various phenotypic traits, such plant and panicle count, leaf density and angle and panicle morphology, and canopy height. These traits together affect light capture and biomass production as well as conversion of photosynthates to grain yield. Panicle morphology exhibits considerable variation as influenced by genetics, environmental conditions and management practices. This study presents a framework for the 3D reconstruction of sorghum canopies and phenotyping panicle morphology. First, we developed a scalable, low-altitude Unmanned Aerial Vehicle (UAV)-based protocol that leverages videos for efficient data acquisition, combined with Neural Radiance Fields (NeRF)s to generate high-quality 3D point cloud reconstructions of sorghum canopies. Next, a 3D model was built to simulate 3D sorghum canopies to create annotated datasets for training deep learning-based semantic segmentation and panicle detection algorithms. Finally, we propose SegVoteNet, a novel multi-task deep learning model that integrates VoteNet and PointNet++ within a shared backbone architecture. Designed for semantic segmentation and 3D detection on pure point cloud data, SegVoteNet incorporates a voting and sampling module that leverages segmentation results to optimize object proposal generation. SegVoteNet is robust, achieving 0.986 Mean Average Precision (mAP) @ 0.5 Intersection Over Union (IOU) on synthetic datasets, and 0.850 mAP @ 0.5 IOU on real point cloud datasets for sorghum panicle detection, without fine-tuning. This set of pipelines provides a robust scalable method for phenotyping sorghum panicles in field trials in breeding and commercial applications. Further work is developing a capability to estimate grain number per panicle, which would provide breeders with additional phenotypes to select.

Plant phenotyping relevance

UAV・NeRF・点群再構成と深層学習によるソルガム穂形態の取得・推定パイプラインを開発し、実データで性能評価しており、植物フェノタイピング手法が研究の中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.