tance of around 2.5 m with a ground sampling distance of 0.3 mm (Fig. S1 a). The tagged and imaged spikes (Fig. S1 b) were sampled and ground truth volumes were acquired with a 3D light scanner (Shining 3D Einscan-SE V2, SHINING3D, Hangzhou, China) following the protocol of [ 76 ] . Links to the dataset and code can be found at https://oliviazum.github.io/3DKD-wheat/ . Detailed information about the dataset can be found in Sec. A . 3.3 Data Pre-Processing Field images contained approximately 300-500 spikes per genotype within a plot of about 1.5 m 2 m^{2} . To reduce background inference, spike detection was first performed, and all subsequent processing was restricted to the detected regio
Open resource ↗lines:91-104Unverified paper record
3D Reconstruction and Knowledge Distillation to Improve Multi-View Image Models to Explore Spike Volume Estimation in Wheat
arXiv (Cornell University) · 20 May 2026 · 10.48550/arxiv.2605.20940
Abstract
Accurate estimation of wheat spike volume is important for yield component analysis and stress resilience assessment, yet field-based measurement remains challenging. Active 3D sensing methods such as Light Detection and Ranging (LiDAR) or time-of-flight (ToF) are sensitive to plant motion or poorly suited to outdoor conditions, while 3D reconstructions are computationally expensive. Direct 2D image processing would offer computational advantages, but image-based models lack explicit geometric information. We therefore propose a hybrid 2D-3D approach with knowledge distillation during training while enabling efficient image-only inference. First, we train a rigid-invariant point cloud network using distance-based histogram features to obtain pose-robust geometric representations. We then combine the 3D model with a proposed multi-view image-based regulated Transformer (RT) in an ensemble architecture. Finally, we distill the ensemble knowledge into a purely image-based student model using either feature-based or label-based distillation. The two distilled RTs reduce the mean absolute error (MAE) from 654.31 mm$^3$ of the non-distilled RT to 639.93 mm$^3$ and 644.62 mm$^3$, and increase correlation from 0.76 to 0.77 and 0.82, respectively. At the same time, inference time is reduced from 160 ms to 1.4 ms per spike. Distillation further mitigates volume-dependent bias and reshapes the latent representation of the image model toward a geometry-aware shape. Our results demonstrate that 3D-informed training of a 2D Transformer allows for scalable and efficient spike volume estimation for high-throughput field phenotyping.
Plant phenotyping relevance
小麦穂の体積を画像・3D再構成・知識蒸留で推定する手法の開発と性能評価が中心であり、高スループット植物フェノタイピングへの応用も明示されている。
abstractWe therefore propose a hybrid 2D-3D approach with knowledge distillation during training while enabling efficient image-only inference.
abstractOur results demonstrate that 3D-informed training of a 2D Transformer allows for scalable and efficient spike volume estimation for high-throughput field phenotyping.
Code and data availability
The paper explicitly states that links to its wheat spike dataset (multi-view images and 3D scans) and its analysis code are available via the authors' project webpage, which is an allowed URL. Other URLs (pyrender, CORDIS projects) are generic libraries or unrelated funding projects, not paper-specific assets.
This is an automatically classified, unverified record. Curator approval is required before any resource enters the Catalog.