Vegetable Size Measurement Based on Stereo Camera and Keypoints Detection

Abstract

This work focuses on the problem of non-contact measurement for vegetables in agricultural automation. The application of computer vision in assisted agricultural production significantly improves work efficiency due to the rapid development of information technology and artificial intelligence. Based on object detection and stereo cameras, this paper proposes an intelligent method for vegetable recognition and size estimation. The method obtains colorful images and depth maps with a binocular stereo camera. Then detection networks classify four kinds of common vegetables (cucumber, eggplant, tomato and pepper) and locate six points for each object. Finally, the size of vegetables is calculated using the pixel position and depth of keypoints. Experimental results show that the proposed method can classify four kinds of common vegetables within 60 cm and accurately estimate their diameter and length. The work provides an innovative idea for solving the vegetable's non-contact measurement problems and can promote the application of computer vision in agricultural automation.

Plant phenotyping relevance

野菜の長さ・直径という植物器官形質を、ステレオカメラ、深度画像、キーポイント検出で非接触推定する手法が研究の中心であるため。

Code and data availability

The authors publicly released both the vegetable keypoint dataset (1600 COCO-format images with ROI boxes and six keypoints) and the implementation code for their size estimation method on GitHub. Labelme is a generic third-party annotation tool and is excluded.