Stereo Vision for Plant Detection in Dense Scenes

Abstract

Automated precision weed control requires visual methods to discriminate between crops and weeds. State-of-the-art plant detection methods fail to reliably detect weeds, especially in dense and occluded scenes. In the past, using hand-crafted detection models, both color (RGB) and depth (D) data were used for plant detection in dense scenes. Remarkably, the combination of color and depth data is not widely used in current deep learning-based vision systems in agriculture. Therefore, we collected an RGB-D dataset using a stereo vision camera. The dataset contains sugar beet crops in multiple growth stages with a varying weed densities. This dataset was made publicly available and was used to evaluate two novel plant detection models, the D-model, using the depth data as the input, and the CD-model, using both the color and depth data as inputs. For ease of use, for existing 2D deep learning architectures, the depth data were transformed into a 2D image using color encoding. As a reference model, the C-model, which uses only color data as the input, was included. The limited availability of suitable training data for depth images demands the use of data augmentation and transfer learning. Using our three detection models, we studied the effectiveness of data augmentation and transfer learning for depth data transformed to 2D images. It was found that geometric data augmentation and transfer learning were equally effective for both the reference model and the novel models using the depth data. This demonstrates that combining color-encoded depth data with geometric data augmentation and transfer learning can improve the RGB-D detection model. However, when testing our detection models on the use case of volunteer potato detection in sugar beet farming, it was found that the addition of depth data did not improve plant detection at high vegetation densities.

Plant phenotyping relevance

RGB-Dデータセットと植物検出モデルを開発・評価し、密集環境での植物個体の画像ベース検出を中心的に扱っているため、植物フェノタイピング手法として収録する。

Code and data availability

The paper's RGB-D plant detection dataset is paper-specific, but the only availability statement uses a placeholder DOI ('http://doi.org/tbdt') with future-tense language ('will be made available'), so it is not an actionable public URL. No author analysis code, trained model checkpoints, or supplementary data deposits

No evidence-backed public reproduction asset is currently recorded.