Design of a binocular multispectral stereo imaging system and its application in plant phenotyping

Abstract

With the continuous progress in micro-optical machine technology, miniature spectral imaging devices have been rapidly developed; however, three-dimensional (3D) imaging measurement technology has become increasingly mature and widely used. The evolution of these technologies has established a robust foundation for the integration of three-dimensional imaging and spectral information. To achieve accurate alignment between 3D data and spectral information to obtain a more comprehensive spectral representation of objects in 3D space, we developed a binocular multispectral stereo imaging (BMSI) system. This system acquires images in synchrony with a binocular multispectral imager, thereby ensuring accurate alignment between 3D data and spectral data at the pixel level and facilitating the construction of a four-dimensional (4D) dataset. The segmentation of leaf regions from shadow backgrounds in two distinct plant species was achieved through optimal band fusion and hue-saturation value (HSV) color space transformation, significantly improving the segmentation accuracy, processing efficiency, and robustness across different plant species. A systematic evaluation was conducted to quantify the reconstruction precision and system stability at different measurement distances. The designed system acquired 4D image spectral data with plants as the objects to be tested. The distribution characteristics of chlorophyll (Chl) on the 3D surface of plants were obtained by first-order derivatives of the spectral data and the normalized difference red edge (NDRE) index. This technique provides a new means for plant phenotyping research and a more effective technical approach for the digitalization and precision monitoring of the agricultural industry.

Code and data availability

The paper's data availability statement explicitly deposits raw data and essential source code for the BMSI plant phenotyping analysis on a public GitHub repository.