D4: Text-guided diffusion model-based domain adaptive data augmentation for vineyard shoot detection

Abstract

In agricultural practices, plant phenotyping using object detection models is gaining attention, plant phenotyping is a technology that accurately measures the quality and condition of cultivated crops from images, contributing to the improvement of crop yield and quality, as well as reducing environmental impact. However, collecting the training data necessary to create generic and high-precision models is extremely challenging due difficulties associated with annotations and the diversity of domains. Such difficulties arise from the unique shapes and backgrounds of plants, as well as the significant changes in appearance due to environmental conditions and growth stages. Furthermore, it is difficult to transfer training data across different crops, and although machine learning models effective for specific environments, conditions, and crops have been developed, they cannot be widely applied in real-world conditions. Therefore, in this study, we propose a generative artificial intelligence data augmentation method (D4) and investigated its application towards a shoot detection task in a vineyard. D4 uses a pre-trained text-guided diffusion model based on a large number of original images culled from video data collected by unmanned ground vehicles or other means, and a small number of annotated datasets. The proposed method generates new annotated images with background information adapted to the target domain while retaining annotation information necessary for object detection. In addition, D4 overcomes the lack of training data in agriculture, including the difficulty of annotation and diversity of domains. We confirmed that this generative data augmentation method improved the mean average precision by up to 28.65% for the BBox detection task and the average precision by up to 13.73% for the keypoint detection task for vineyard shoot detection. D4 generative data augmentation is expected to simultaneously solve the cost and domain diversity issues of training data generation for agricultural applications and improve the generalization performance of detection models. • Proposed novel data augmentation method D4 using text-guided diffusion model. • Analyzed detection accuracy using D4 for BBox detection and keypoint detection. • D4 improved detection accuracy and demonstrated effectiveness of domain adaptation. • D4 uses automatic selection with DreamSim to maintain generated image quality. • D4 overcomes lack of training data in agriculture.

Plant phenotyping relevance

ブドウのシュートを画像から検出する植物フェノタイピング向けのデータ拡張・ドメイン適応手法を開発し、検出精度を検証しており、表現型取得手法が中心である。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.