3D Robotics and LMM for Vineyard Inspection

Abstract

Autonomous mobile robotic solutions are increasingly being explored in precision agriculture to aid human workers in labour-intensive or repetitive tasks. Moreover, the emergence of foundation models in vision-based AI domain presents an opportunity to perform automated interpretation of in-field collected data. This study presents a cost-effective mobile robotic research platform designed for autonomous vineyard inspection: it integrates mission planning, real-world navigation and a post-processing pipeline of multimodal data. The system, based on the Leo rover, is equipped with LiDAR, RGB cameras and GNSS-visual-inertial positioning, ensuring reliable operation in GNSS-degraded vineyard environments. We propose a novel methodology for automating several stages of the workflow using various open and in-situ collected data. The robotic platform and processing pipeline were validated through simulation and field experiments, demonstrating its capability for autonomous navigation, 3D reconstruction, AI-based fruit detection and an initial plant health assessment through Large Multimodal Models (LMM). Results show that while 3D mapping provides highresolution spatial data, AI-driven object detection and vision models require further domain adaptation for reaching reliable and trustable operation. The study highlights the feasibility of cost-effective mobile robotic solutions in vineyard monitoring and the potential of integrating AI to enhance agricultural automation.

Plant phenotyping relevance

自律型ロボットとマルチモーダル処理パイプラインを開発・検証し、3D再構成、果実検出、植物健全性評価という植物状態の取得を中核的に扱っているため。

Code and data availability

The paper describes in-situ collected vineyard imagery, a retrained YOLOv8 grape-detection model, and LMM evaluation results, but no public deposit of these datasets, images, annotations, trained checkpoints, or authors' analysis code is stated. All URLs present (Leo rover, Fixposition, Ollama, Ultralytics, Trento cart

No evidence-backed public reproduction asset is currently recorded.