The effectiveness of large language models with RAG for auto-annotating trait and phenotype descriptions.

Abstract

Ontologies are highly prevalent in biology and medicine and are always evolving. Annotating biological text, such as observed phenotype descriptions, with ontology terms is a challenging and tedious task. The process of annotation requires a contextual understanding of the input text and of the ontological terms available. While text-mining tools are available to assist, they are largely based on directly matching words and phrases and so lack understanding of the meaning of the query item and of the ontology term labels. Large Language Models (LLMs), however, excel at tasks that require semantic understanding of input text and therefore may provide an improvement for the auto-annotation of text with ontological terms. Here we describe a series of workflows incorporating OpenAI GPT's capabilities to annotate Arabidopsis thaliana and forest tree phenotypic observations with ontology terms, aiming for results that resemble manually curated annotations. These workflows make use of an LLM to intelligently parse phenotypes into short concepts, followed by finding appropriate ontology terms via embedding vector similarity or via Retrieval-Augmented Generation (RAG). The RAG model is a state-of-the-art approach that augments conversational prompts to the LLM with context-specific data to empower it beyond its pre-trained parameter space. We show that the RAG produces the most accurate automated annotations that are often highly similar or identical to expert-curated annotations.

Plant phenotyping relevance

植物の表現型観察記述をオントロジー語に自動アノテーションするLLM/RAGワークフローの開発・精度評価が中心であり、再利用可能な計算ツールとして植物表現型データを処理する。

Code and data availability

The paper's phenotype descriptors, gold-standard annotations, LLM-parsed concepts, auto-annotations, and evaluation scores are publicly available as supplementary files, and the authors' analysis code (DE, DCE, DCRAG workflows) is publicly deposited on GitHub. A specific AraPheno trait (#278) used as an input example/`