Self-Supervised Convolutional Neural Networks for Plant Reconstruction Using Stereo Imagery

Abstract

Stereo matching can provide complete and dense three-dimensional reconstruction to study plant growth. Recently, high-quality stereo matching results were achieved combining Semi-Global Matching (SGM) with deep learning. However, due to a lack of suitable training data, this technique is not readily applicable for plant reconstruction. We propose a self-supervised Matching Cost with a Convolutional Neural Network (MC-CNN) scheme to calculate matching cost and test it for plant reconstruction. The MC-CNN network is retrained using the initial matching results obtained from the standard MC-CNN weights. For the experiment, closerange photogrammetric imagery of an in-house plant is used. The results show that the performance of self-supervised MC-CNN is superior to the Census algorithm and comparable to MC-CNN trained by a Light Detection and Ranging point cloud. Another experiment is performed using stereo imagery of a field beech tree. The proposed self-training strategy is tested and has proved capable of identifying the drought condition of trees from the reconstructed leaves.

Code and data availability

The paper describes in-house stereo imagery of an indoor tree and a field beech tree, LiDAR ground truth, and self-trained MC-CNN models, but no blocks contain any public dataset deposit, author code release, or availability statement. allowed_urls is empty, so no actionable URL exists. Middlebury/KITTI are cited prior

No evidence-backed public reproduction asset is currently recorded.