Multimodal Data Fusion for Precise Lettuce Phenotype Estimation Using Deep Learning Algorithms

Abstract

Effective lettuce cultivation requires precise monitoring of growth characteristics, quality assessment, and optimal harvest timing. In a recent study, a deep learning model based on multimodal data fusion was developed to estimate lettuce phenotypic traits accurately. A dual-modal network combining RGB and depth images was designed using an open lettuce dataset. The network incorporated both a feature correction module and a feature fusion module, significantly enhancing the performance in object detection, segmentation, and trait estimation. The model demonstrated high accuracy in estimating key traits, including fresh weight (fw), dry weight (dw), plant height (h), canopy diameter (d), and leaf area (la), achieving an R2 of 0.9732 for fresh weight. Robustness and accuracy were further validated through 5-fold cross-validation, offering a promising approach for future crop phenotyping.

Plant phenotyping relevance

RGB・深度画像を融合した深層学習によるレタス形質推定手法を開発し、交差検証で性能評価しており、フェノタイピング手法が中心である。

Code and data availability

The paper's RGB-D lettuce images and trait measurements come from the publicly available Third Autonomous Greenhouse Challenge dataset deposited at 4TU.ResearchData, with an explicit availability statement and URL matching an allowed URL. No author analysis code or trained model is disclosed.