De-occlusion models and diffusion-based data augmentation for size estimation of on-plant oriental melons.

Abstract

Accurate fruit size estimation is crucial for plant phenotyping, as it enables precise crop management and enhances agricultural productivity by providing essential data for growth and resource efficiency analysis. In this study, we estimated the size of on-plant oriental melons grown in a vertical cultivation system to address the challenges posed by leaf occlusion. Data augmentation was achieved using a diffusion model to generate synthetic leaves to cover existing fruits and create an enriched dataset. Three instance segmentation models-mask region-based convolutional neural network (CNN), Mask2Former, and detection transformer (DETR)-and six de-occlusion models derived from these architectures were implemented. These models successfully inferred both visible and occluded areas of the fruit. Notably, Amodal Mask2Former and occlusion-aware RCNN (ORCNN) achieved average precision scores of 85.92 ​% and 85.35 ​%, respectively. The inferred masks were used to estimate the height and diameter of the fruit, with Amodal Mask2Former yielding a mean absolute error of 5.46 ​mm and 4.20 ​mm and a mean absolute percentage error of 4.86 ​% and 5.33 ​%, respectively. The results indicate enhanced performance of the transformer-based Amodal Mask2Former over CNN architectures in de-occlusion tasks and size estimation. Finally, the enhancement in de-occlusion models compared to conventional models was assessed and demonstrated across occlusion ratios ranging from 0 to 70 ​%. However, generating synthetic datasets with occlusion ratios over 70 ​% remains a limitation.

Plant phenotyping relevance

果実の遮蔽領域を復元し、画像から果実サイズを推定する手法の開発・評価が研究の中心であり、植物表現型計測に該当する。

Code and data availability

The authors explicitly state that the code for training/testing the segmentation models and the size estimation analysis is publicly available at their GitHub repository (https://github.com/sungjay-kim). The oriental melon image dataset itself is only available upon request from the corresponding author, so it is not a