Monocular Camera Based Fruit Counting and Mapping With Semantic Data Association

Abstract

In this letter, we present a cheap, lightweight, and fast fruit counting pipeline. Our pipeline relies only on a monocular camera, and achieves counting performance comparable to a state-of-the-art fruit counting system that utilizes an expensive sensor suite including a monocular camera, LiDAR and GPS/INS on a mango dataset. Our pipeline begins with a fruit and tree trunk detection component that uses state-of-the-art convolutional neural networks (CNNs). It then tracks fruits and tree trunks across images, with a Kalman Filter fusing measurements from the CNN detectors and an optical flow estimator. Finally, fruit count and map are estimated by an efficient fruit-as-feature semantic structure from motion algorithm that converts two-dimensional (2-D) tracks of fruits and trunks into 3-D landmarks, and uses these landmarks to identify double counting scenarios. There are many benefits of developing such a low cost and lightweight fruit counting system, including applicability to agriculture in developing countries, where monetary constraints or unstructured environments necessitate cheaper hardware solutions.

Code and data availability

The article blocks describe a monocular fruit counting pipeline on a mango dataset, but no public dataset, image/sensor inputs, author code, or trained model is made available with an authors' URL. The only URL present (https://label.ag/ral19.mp4) is a demonstration video of the algorithm, not a phenotype dataset, raw/

No evidence-backed public reproduction asset is currently recorded.