Y-HRNet: Research on multi-category cherry tomato instance segmentation model based on improved YOLOv7 and HRNet fusion

Abstract

Accurate recognition of multi-category targets in cherry tomato images is a technical prerequisite for automated picking. However, in unstructured real-world scenarios, the existing network parameters are numerous and computationally intensive, and the models have low recognition accuracy when deployed on picking robots. Additionally, tomato detection and segmentation face challenges due to variable lighting, tomato overlap, similar backgrounds, and color transitions. In this context, this study focuses on the accurate segmentation of cherry tomato ripeness in large scenarios. This paper proposes a ”coarse detection, fine segmentation” method named Y-HRNet for greenhouse cherry tomatoes, which utilizes a multi-class cherry tomato dataset divided into four categories: green, turning, ripe, and fully ripe, achieving pixel-accurate segmentation of tomatoes of different ripeness levels. Firstly, a lightweight network model is constructed using YOLOv7 to build a lightweight object detection model. The ROI(Regions of Interest) is selected for segmentation, reducing the interference of complex backgrounds in large environments on the second-stage tomato segmentation task. Then, the ECA (Efficient Channel Attention) module and the DR-ASPP module are introduced into the Y-HRNet network. This enhances the model’s segmentation accuracy, enabling more effective capture of cherry tomatoes at four different maturity stages. The experiments demonstrate that Y-HRNet achieves segmentation of cherry tomatoes with the MIoU of 84.69%, MPA of 91.52%, and an overall accuracy of 94.39%. The average processing time of a single cherry tomato image is 0.35s. Compared to classic segmentation methods, our approach significantly improves performance. Therefore, this method provides technical support for the maturity grading and harvest management decisions of cherry tomatoes.

Plant phenotyping relevance

チェリートマトの成熟度という植物状態を画像から画素単位で推定する新規セグメンテーション手法を開発・評価しており、フェノタイピング手法が中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.