Multimodal data-driven method for throughput prediction in combine harvesters

Abstract

Aiming at the hysteresis problem of traditional contact-type mechanical throughput detection method during combine harvester operation, this study proposes a multimodal data-driven online throughput prediction method. By building a multimodal sensor system that integrates vehicle-mounted cameras, GPS, grain moisture content, and feeding auger power sensors, a throughput prediction framework based on wheat ear biomass characteristics was established: Firstly, the WEC-MVFF wheat ear online counting density map estimation model is designed, and MobileViT is used to build a feature extraction backbone network. The multi-scale fusion module and the centralized conversion module are combined to realize the collaborative extraction of shallow texture features and deep semantic features of dense wheat ears in the field. Secondly, divide the image into regions of interest and establish a throughput prediction model based on multimodal information of wheat ear number (image) − moisture content (sensor) − travel speed. At the same time, a detection model based on feeding auger power is constructed as a comparison benchmark. Field tests show that the WEC-MVFF model maintains an average counting accuracy of more than 90 % under different wheat ear density, travel speed and light intensity conditions. The model’s online counting advantage is verified through ablation study and comparative tests with other counting models. The throughput prediction method achieved an MAE of 0.70 kg/s and 0.77 kg/s in test area 1 and 2 respectively, the first-order difference fluctuation was stable in the range of ±0.50 kg/s, and the prediction frame rate of 10-13fps met the real-time requirements. Compared with the single-modal image prediction method, the accuracy of the multimodal method considering moisture content was improved by 0.80 kg/s and 0.75 kg/s in the two test areas, respectively. Compared with traditional mechanical quantity detection methods, it has higher accuracy and stability while achieving early prediction, providing reliable feedforward information support for the intelligent control of harvesters.

Plant phenotyping relevance

小麦穂の画像計数とマルチモーダルセンサを用いて、穂密度・バイオマス特性および収量流量を推定する手法を開発・検証しており、植物形質の取得・推定が研究の中心である。

Code and data availability

公開状態または取得可能な本文経路を確認できませんでした。

No evidence-backed public reproduction asset is currently recorded.