Semantic Mapping for Orchard Environments by Merging Two-Sides Reconstructions of Tree Rows

Abstract

Measuring semantic traits for phenotyping is an essential but labor-intensive activity in horticulture. Researchers often rely on manual measurements which may not be accurate for tasks such as measuring tree volume. To improve the accuracy of such measurements and to automate the process, we consider the problem of building coherent three dimensional (3D) reconstructions of orchard rows. Even though 3D reconstructions of side views can be obtained using standard mapping techniques, merging the two side-views is difficult due to the lack of overlap between the two partial reconstructions. Our first main contribution in this paper is a novel method that utilizes global features and semantic information to obtain an initial solution aligning the two sides. Our mapping approach then refines the 3D model of the entire tree row by integrating semantic information common to both sides, and extracted using our novel robust detection and fitting algorithms. Next, we present a vision system to measure semantic traits from the optimized 3D model that is built from the RGB or RGB-D data captured by only a camera. Specifically, we show how canopy volume, trunk diameter, tree height and fruit count can be automatically obtained in real orchard environments. The experiment results from multiple datasets quantitatively demonstrate the high accuracy and robustness of our method.

Plant phenotyping relevance

果樹列の3D再構成と画像解析により、樹冠体積・幹径・樹高・果実数を自動測定する手法が研究の中心であり、植物表現型の取得方法を実環境で検証している。

Code and data availability

The supplied blocks describe orchard RGB/RGB-D data collection and trunk detection using Mask R-CNN and the VGG Image Annotator, but both are generic third-party tools, not paper-specific assets. No public dataset of the paper's orchard images/annotations, no author analysis code, and no trained model checkpoint is de-

No evidence-backed public reproduction asset is currently recorded.