An Efficient and Automated Image Preprocessing Using Semantic Segmentation for Improving the 3D Reconstruction of Soybean Plants at the Vegetative Stage

Abstract

The investigation of plant phenotypes through 3D modeling has emerged as a significant field in the study of automated plant phenotype acquisition. In 3D model construction, conventional image preprocessing methods exhibit low efficiency and inherent inefficiencies, which increases the difficulty of model construction. In order to ensure the accuracy of the 3D model, while reducing the difficulty of image preprocessing and improving the speed of 3D reconstruction, deep learning semantic segmentation technology was used in the present study to preprocess original images of soybean plants. Additionally, control experiments involving soybean plants of different varieties and different growth periods were conducted. Models based on manual image preprocessing and models based on image segmentation were established. Point cloud matching, distance calculation and model matching degree calculation were carried out. In this study, the DeepLabv3+, Unet, PSPnet and HRnet networks were used to conduct semantic segmentation of the original images of soybean plants in the vegetative stage (V), and Unet network exhibited the optimal test effect. The values of mIoU, mPA, mPrecision and mRecall reached 0.9919, 0.9953, 0.9965 and 0.9953. At the same time, by comparing the distance results and matching accuracy results between the models and the reference models, a conclusion could be drawn that semantic segmentation can effectively improve the challenges of image preprocessing and long reconstruction time, greatly improve the robustness of noise input and ensure the accuracy of the model. Semantic segmentation plays a crucial role as a fundamental component in enabling efficient and automated image preprocessing for 3D reconstruction of soybean plants during the vegetative stage. In the future, semantic segmentation will provide a solution for the pre-processing of 3D reconstruction for other crops.

Code and data availability

The paper publicly releases its semantic segmentation dataset of 500 annotated soybean plant images, the 3D reconstruction model data from both preprocessing methods, and the authors' four segmentation network implementations (DeepLabv3+, Unet, PSPnet, HRnet) via Baidu pan links and GitHub repositories.