Fruit-In-Sight: a deep learning-based framework for secondary metabolite class prediction using fruit and leaf images

Abstract

Fruits produce a wide variety of secondary metabolites of great economic value. Analytical measurement of secondary metabolites is tedious, time-consuming and expensive. Additionally, metabolite concentration varies greatly from tree to tree, making it difficult to choose trees for fruit collection. The current study tested whether deep learning-based models can be developed using fruit and leaf images alone to predict a metabolites concentration class (high or low). We collected fruits and leaves (n = 1045) from neem trees grown in the wild across 0.6 million sq km, imaged those, measured concentration of five metabolites (azadirachtin, deacetyl-salannin, salannin, nimbin and nimbolide) using high-performance liquid chromatography and used those to train deep learning models for metabolite class prediction. The best model out of the seven tested (YOLOv5, GoogLeNet, InceptionNet, EfficientNet_B0, Resnext_50, Resnet18, and SqueezeNet) provided a validation F1 score of 0.93 and a test F1 score of 0.88. The sensitivity and specificity of the fruit model alone in the test set were 83.52 {+/-} 6.19 and 82.35 {+/-} 5.96 and 79.40 {+/-} 8.50 and 85.64 {+/-} 6.21, for the low and the high class, respectively. The sensitivity was further boosted to 92.67{+/-} 5.25 for the low class and 88.11 {+/-} 9.17 for the high class and the specificity to 100% for both classes, using a multi-analyte framework. We incorporated the model in an Android mobile App Fruit-In-Sight that uses fruit and leaf images to decide whether to pick or not pick the fruits from a specific tree based on the metabolite concentration class. Our study provides evidence that images of fruits and leaves alone can predict the concentration class of a secondary metabolite without using extensive analytical laboratory procedures and equipment and makes the process of choosing the right tree for fruit collection easy and free of equipment and additional cost.

Plant phenotyping relevance

果実・葉画像から二次代謝産物濃度クラスを推定する深層学習手法を開発・検証し、モバイルアプリにも実装しており、植物形質推定が研究の中心である。

Code and data availability

植物フェノタイピング解析を再現する公開資産であることを、入力本文と直接リンクから確認できなかったため保留しました。

No evidence-backed public reproduction asset is currently recorded.