Multimodal Data Fusion for Precise Lettuce Phenotype Estimation Using Deep Learning Algorithms

Abstract

Effective lettuce cultivation requires precise monitoring of growth characteristics, quality assessment, and optimal harvest timing. In a recent study, a deep learning model based on multimodal data fusion was developed to estimate lettuce phenotypic traits accurately. A dual-modal network combining RGB and depth images was designed using an open lettuce dataset. The network incorporated both a feature correction module and a feature fusion module, significantly enhancing the performance in object detection, segmentation, and trait estimation. The model demonstrated high accuracy in estimating key traits, including fresh weight (fw), dry weight (dw), plant height (h), canopy diameter (d), and leaf area (la), achieving an R2 of 0.9732 for fresh weight. Robustness and accuracy were further validated through 5-fold cross-validation, offering a promising approach for future crop phenotyping.

Code and data availability

The paper's RGB-D lettuce images and trait measurements come from the publicly available Third Autonomous Greenhouse Challenge dataset deposited at 4TU.ResearchData, with an explicit availability statement and URL matching an allowed URL. No author analysis code or trained model is disclosed.