Making deep neural networks right for the right scientific reasons by interacting with their explanations

Abstract

Deep neural networks have shown excellent performances in many real-world applications. Unfortunately, they may show "Clever Hans"-like behavior -- making use of confounding factors within datasets -- to achieve high performance. In this work, we introduce the novel learning setting of "explanatory interactive learning" (XIL) and illustrate its benefits on a plant phenotyping research task. XIL adds the scientist into the training loop such that she interactively revises the original model via providing feedback on its explanations. Our experimental results demonstrate that XIL can help avoiding Clever Hans moments in machine learning and encourages (or discourages, if appropriate) trust into the underlying model.

Plant phenotyping relevance

植物フェノタイピング課題を対象に、説明への研究者フィードバックを学習ループへ組み込む新しい機械学習手法を提案・実証しており、フェノタイピング解析手法が中心である。

Code and data availability

The paper's plant phenotyping RGB/hyperspectral dataset is publicly deposited on TU Datalib, and the authors' analysis code (runnable Code Ocean capsule with pre-trained models reproducing figures/results) plus the user study materials are publicly available on GitHub. Generic benchmarks (Fashion-MNIST, PASCAL VOC) are