Backbone diversity beats text supervision: a systematic study of frozen multi-foundation model fusion for in-the-wild plant disease recognition.

Abstract

Automated plant disease recognition from field photographs remains challenging: models trained on laboratory datasets collapse on in-the-wild images, and the current state of the art on PlantWild-the largest open in-the-wild benchmark (18,542 images, 89 classes)-relies on text prototypes derived from a language model to reach 76.18% top 1 accuracy. We ask whether text supervision is truly necessary or whether the bottleneck is the diversity of the visual representation. Our central methodological finding is that backbone selection and fusion matter far more than classifier-head engineering : across 118 experiments, a simple fixed-weight linear-prototype combination on top of three complementary frozen backbones yields larger and more reproducible gains than any head-level adaptive routing mechanism we test. Specifically, through a systematic study of six frozen vision backbones (three CLIP, two DINOv3, and one DINOv2), three classifier heads, and four fusion configurations, completed in a single day on one consumer GPU, we establish three findings. (i) A single self-supervised backbone (DINOv2 ViT-L/14) already surpasses text-augmented MVPDR (77.56% vs. 76.18%). (ii) Concatenating three complementary backbones (DINOv2 + DINOv3 + CLIP) reaches 80.23% ± 0.41% (five seeds), exceeding the published MVPDR accuracy by +4.05 points and our own reproduction of MVPDR under an identical evaluation protocol by +7.96 points, without any language supervision. (The difference between the two deltas reflects evaluation-protocol differences-our split, model-selection criterion, and training schedule-rather than any discrepancy in the reported numbers; see Section 4.6.6 for a full reconciliation.) (iii) Every form of learned routing we test-per-class gating, backbone gating, sample-wise gating-is inessential; the gain is entirely attributable to backbone diversity and a simple linear-prototype scoring combination. On the smaller PlantDoc benchmark, the same principle transfers but with substantially higher seed variance: the best configuration reaches 80.09% at a favourable seed but 76.97% ± 1.48% over five seeds-a suggestive rather than robust gain. Beyond the accuracy headline, we provide a pathology-aware per-class analysis showing that DINOv2/v3 dominate on fine-texture lesion classes (rusts, mildews, and leaf spots) whilst CLIP's narrow advantage concentrates on organ/species-level identification (rice leaf and potato late blight). All primary claims are validated over five seeds, and all code, feature caches, and result files are released for full reproducibility.

Plant phenotyping relevance

植物病害の画像認識を対象に、複数の視覚基盤モデル融合と分類器を体系比較・検証しており、病害状態の推定手法が研究の中心である。

Code and data availability

The paper states that its code, cached features, and result files will be released, but no authors' public URL or repository is provided in the supplied blocks (future-tense availability language only). The DINOv3 GitHub link and OpenReview link are third-party backbone resources, not paper-specific assets.