tance of around 2.5 m with a ground sampling distance of 0.3 mm (Fig. S1 a). The tagged and imaged spikes (Fig. S1 b) were sampled and ground truth volumes were acquired with a 3D light scanner (Shining 3D Einscan-SE V2, SHINING3D, Hangzhou, China) following the protocol of [ 76 ] . Links to the dataset and code can be found at https://oliviazum.github.io/3DKD-wheat/ . Detailed information about the dataset can be found in Sec. A . 3.3 Data Pre-Processing Field images contained approximately 300-500 spikes per genotype within a plot of about 1.5 m 2 m^{2} . To reduce background inference, spike detection was first performed, and all subsequent processing was restricted to the detected regio
Open resource ↗lines:91-104Paper record
3D Reconstruction and Knowledge Distillation to Improve Multi-View Image Models to Explore Spike Volume Estimation in Wheat
arXiv (Cornell University) · 20 May 2026 · 10.48550/arxiv.2605.20940
Abstract
Accurate estimation of wheat spike volume is important for yield component analysis and stress resilience assessment, yet field-based measurement remains challenging. Active 3D sensing methods such as Light Detection and Ranging (LiDAR) or time-of-flight (ToF) are sensitive to plant motion or poorly suited to outdoor conditions, while 3D reconstructions are computationally expensive. Direct 2D image processing would offer computational advantages, but image-based models lack explicit geometric information. We therefore propose a hybrid 2D-3D approach with knowledge distillation during training while enabling efficient image-only inference. First, we train a rigid-invariant point cloud network using distance-based histogram features to obtain pose-robust geometric representations. We then combine the 3D model with a proposed multi-view image-based regulated Transformer (RT) in an ensemble architecture. Finally, we distill the ensemble knowledge into a purely image-based student model using either feature-based or label-based distillation. The two distilled RTs reduce the mean absolute error (MAE) from 654.31 mm$^3$ of the non-distilled RT to 639.93 mm$^3$ and 644.62 mm$^3$, and increase correlation from 0.76 to 0.77 and 0.82, respectively. At the same time, inference time is reduced from 160 ms to 1.4 ms per spike. Distillation further mitigates volume-dependent bias and reshapes the latent representation of the image model toward a geometry-aware shape. Our results demonstrate that 3D-informed training of a 2D Transformer allows for scalable and efficient spike volume estimation for high-throughput field phenotyping.
Code and data availability