Dense cotton boll counting with transformer-based video tracking and a customized phenotyping robot for data collection

Abstract

Accurately estimating the number of cotton bolls is vital for plant phenotyping, offering essential insights for both breeders and growers. This trait offers valuable phenotypic information on plant productivity and supports crop management decisions to optimize yield and profitability for growers. Manual counting of bolls in the field, however, is impractical because it is labor-intensive and time-consuming. This study presented a video-based cotton boll counting approach that integrated a transformer-based detector (RT-DETR) with multi-object tracking techniques. To prevent double-counting bolls across frames, two motion estimation methods, FlowFormer and TAPIR were explored to predict the movement of bolls between adjacent frames and a two-stage association process combining Intersection over Union (IoU) and Euclidean distances was developed to track bolls across time. To further enhance counting accuracy, a virtual counting line was introduced to reduce ID switch errors. Experimental results demonstrated the effectiveness of the RT-DETR model, achieving an mAP0.5 exceeding 0.93 for dense boll detection. Furthermore, both FlowFormer and TAPIR can be used for tracking cotton bolls in the videos while the tracking performance of the FlowFormer-based method was slightly higher than that of the TAPIR-based method with an MOTA of 73.36 % and an IDF1 of 79.89 %. The tracking approach integrating RT-DETR and FlowFormer exhibited a relatively strong correlation between the predicted and the ground-truth boll number with an R² of 0.60 and an MAPE of 14.34 % on multi-plant plots. In single-plant plots, the approach achieved a high correlation with an R² of 0.97 and a MAPE of 10.33%. These findings indicated the potential of the proposed approach as an effective, automated tool to support breeding programs and yield assessments in cotton production. Both the code and dataset can be accessed at: https://github.com/UGA-BSAIL/Dense_cotton_boll_counting.

Plant phenotyping relevance

綿花のボール数という植物生産形質を、動画検出・追跡とロボット収集で自動推定する手法の開発・評価が研究の中心であり、mAP、MOTA、IDF1、R²、MAPEによる技術検証も行っている。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.