Monocular Camera Based Fruit Counting and Mapping With Semantic Data Association

Abstract

In this letter, we present a cheap, lightweight, and fast fruit counting pipeline. Our pipeline relies only on a monocular camera, and achieves counting performance comparable to a state-of-the-art fruit counting system that utilizes an expensive sensor suite including a monocular camera, LiDAR and GPS/INS on a mango dataset. Our pipeline begins with a fruit and tree trunk detection component that uses state-of-the-art convolutional neural networks (CNNs). It then tracks fruits and tree trunks across images, with a Kalman Filter fusing measurements from the CNN detectors and an optical flow estimator. Finally, fruit count and map are estimated by an efficient fruit-as-feature semantic structure from motion algorithm that converts two-dimensional (2-D) tracks of fruits and trunks into 3-D landmarks, and uses these landmarks to identify double counting scenarios. There are many benefits of developing such a low cost and lightweight fruit counting system, including applicability to agriculture in developing countries, where monetary constraints or unstructured environments necessitate cheaper hardware solutions.

Plant phenotyping relevance

果実数という植物器官の明示的形質を、単眼カメラ画像から検出・追跡・三次元推定する手法開発が研究の中心であるため、植物フェノタイピング手法として含める。

Code and data availability

The article blocks describe a monocular fruit counting pipeline on a mango dataset, but no public dataset, image/sensor inputs, author code, or trained model is made available with an authors' URL. The only URL present (https://label.ag/ral19.mp4) is a demonstration video of the algorithm, not a phenotype dataset, raw/

No evidence-backed public reproduction asset is currently recorded.