Integration of LLMs and VLMs in plant stress phenotyping: From trait recognition to decision support.

Abstract

The integration of Large Language Models (LLMs) with Vision-Language Models (VLMs) holds transformative potential for plant stress phenotyping, enhancing high-throughput crop monitoring, trait identification, and decision support. Traditional phenotyping methods, often reliant on manual assessments and task-specific Machine Learning (ML) models, face persistent limitations in scalability, adaptability, and contextual interpretation, especially under complex and overlapping stress conditions. VLMs address these challenges by combining deep visual recognition with contextual reasoning, enabling real-time analysis of multimodal inputs such as high-resolution imagery, agronomic text data, and environmental sensor readings. Complementarily, LLMs contribute to text mining, semantic annotation of trait descriptors, and the integration of external knowledge via Retrieval-Augmented Generation (RAG), thereby enhancing the interpretability and adaptability of phenotyping workflows. This review critically evaluates the emerging role of integrating LLMs with VLMs in plant stress phenotyping, highlighting their applications in visual trait recognition, knowledge extraction, and autonomous decision-making. We synthesize current advances and identify key challenges, including data quality, domain-specific generalization, model transparency, and equitable access to AI technologies. As one of the first comprehensive reviews on this topic, we propose a forward-looking framework that integrates LLMs, VLMs, and RAG systems to enable scalable, explainable, and user-centric phenotyping solutions. This interdisciplinary convergence offers a promising pathway toward sustainable and resilient AI-driven agriculture.

Plant phenotyping relevance

植物ストレス・形質認識のためのLLM/VLM統合を批判的に評価するレビューであり、植物フェノタイピング手法が中心です。

Code and data availability

The supplied blocks are from a review article on LLMs/VLMs in plant stress phenotyping. The text describes other studies' datasets and models (e.g., IDADP, Plant-Village, PhenoGPT) but contains no authors' own phenotype datasets, images, code, or trained models, and no availability/deposit statements with public URLs.

No evidence-backed public reproduction asset is currently recorded.