Explicitly incorporating spatial information to recurrent networks for agriculture

Abstract

In agriculture, the majority of vision systems perform still image classification. Yet, recent work has highlighted the potential of spatial and temporal cues as a rich source of information to improve the classification performance. In this paper, we propose novel approaches to explicitly capture both spatial and temporal information to improve the classification of deep convolutional neural networks. We leverage available RGB-D images and robot odometry to perform inter-frame feature map spatial registration. This information is then fused within recurrent deep learnt models, to improve their accuracy and robustness. We demonstrate that this can considerably improve the classification performance with our best performing spatial-temporal model (ST-Atte) achieving absolute performance improvements for intersection-over-union (IoU[%]) of 4.7 for crop-weed segmentation and 2.6 for fruit (sweet pepper) segmentation. Furthermore, we show that these approaches are robust to variable framerates and odometry errors, which are frequently observed in real-world applications.

Plant phenotyping relevance

RGB-D画像とロボットオドメトリを用いて空間・時間情報を統合する画像解析手法を開発し、作物・果実のセグメンテーション性能を検証しているため、植物表現型取得手法が中心である。

Code and data availability

The paper uses two RGB-D agricultural datasets (SB20, BUP20) and mentions a codebase, but no public URL or deposit for either is provided; the code is only 'to be made available upon publication', and no dataset availability statement with an authors' public URL appears in the supplied blocks.

No evidence-backed public reproduction asset is currently recorded.