Smartphone-based high-fidelity 3D semantic segmentation of finger millet panicles using 3D Gaussian splatting for automated phenotyping and yield estimation

Abstract

Finger millet is an important cereal crop widely cultivated worldwide for food and fodder. Breeding programs aim to select genotypes with desirable architectural traits to develop new varieties with higher yields. In this effort, accurate high-throughput plant phenotyping is essential for accelerating crop improvement. To overcome the time-consuming and labor-intensive process of manual measurements, this study presents a comprehensive 3D imaging pipeline that leverages neural radiance fields (NeRF), 3D gaussian splatting (3DGS), and its advanced extensions (e.g., Feature 3DGS and Gaussian Grouping) to reconstruct, segment, and analyze finger millet yield component traits using multi-view 2D images. First, multiple-view RGB images of a single finger millet plant were captured, and COLMAP was then utilized to estimate the camera poses of the images and reconstruct the sparse point cloud, followed by advanced 3D reconstruction through 3DGS and NeRF. Second, feature 3DGS and gaussian grouping models were used to generate the 3D gaussian representation of finger millet panicles. This single-process framework enabled the generation of high-fidelity 3D point clouds and semantic feature fields without the need for expensive depth sensors or manual annotations. Our results demonstrated the effectiveness of these models in capturing morphological variations across different panicle phenotypes, including compact versus open panicle architectures. In addition, the 3D point clouds of the panicles were utilized to extract structural traits for yield prediction, achieving biologically meaningful correlations with grain productivity. This work highlights the potential of 3DGS-based phenotyping pipelines as a low-cost, near real-time, photorealistic solution for trait quantification, segmentation, and yield estimation in real-world agricultural settings.

Plant phenotyping relevance

3D画像再構成・セグメンテーション・形質抽出を統合した植物フェノタイピング手法の開発が中心であり、収量関連形質の定量と予測まで技術的に評価している。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.