PanicleNeRF: low-cost, high-precision in-field phenotypingof rice panicles with smartphone

Abstract

The rice panicle traits significantly influence grain yield, making them a primary target for rice phenotyping studies. However, most existing techniques are limited to controlled indoor environments and difficult to capture the rice panicle traits under natural growth conditions. Here, we developed PanicleNeRF, a novel method that enables high-precision and low-cost reconstruction of rice panicle three-dimensional (3D) models in the field using smartphone. The proposed method combined the large model Segment Anything Model (SAM) and the small model You Only Look Once version 8 (YOLOv8) to achieve high-precision segmentation of rice panicle images. The NeRF technique was then employed for 3D reconstruction using the images with 2D segmentation. Finally, the resulting point clouds are processed to successfully extract panicle traits. The results show that PanicleNeRF effectively addressed the 2D image segmentation task, achieving a mean F1 Score of 86.9% and a mean Intersection over Union (IoU) of 79.8%, with nearly double the boundary overlap (BO) performance compared to YOLOv8. As for point cloud quality, PanicleNeRF significantly outperformed traditional SfM-MVS (structure-from-motion and multi-view stereo) methods, such as COLMAP and Metashape. The panicle length was then accurately extracted with the rRMSE of 2.94% for indica and 1.75% for japonica rice. The panicle volume estimated from 3D point clouds strongly correlated with the grain number (R2 = 0.85 for indica and 0.82 for japonica) and grain mass (0.80 for indica and 0.76 for japonica). This method provides a low-cost solution for high-throughput in-field phenotyping of rice panicles, accelerating the efficiency of rice breeding.

Plant phenotyping relevance

スマートフォン画像、セグメンテーション、NeRFによる3D再構成を統合し、イネ穂の形質抽出を開発・検証した中心的な手法研究である。

Code and data availability

The supplied blocks describe the PanicleNeRF pipeline (smartphone video acquisition of 50 panicles per experiment, SAM+YOLOv8 segmentation, Instant-NGP reconstruction, trait extraction) but contain no public phenotype dataset, image deposit, author code repository, or trained model checkpoint with availability language

No evidence-backed public reproduction asset is currently recorded.