GANana: Unsupervised Domain Adaptation for Volumetric Regression of Fruit

Abstract

3D reconstruction of fruit is important as a key component of fruit grading and an important part of many size estimation pipelines.Like many computer vision challenges, the 3D reconstruction task suffers from a lack of readily available training data in most domains, with methods typically depending on large datasets of high-quality image-model pairs.In this paper, we propose an unsupervised domain-adaptation approach to 3D reconstruction where labelled images only exist in our source synthetic domain, and training is supplemented with different unlabelled datasets from the target real domain.We approach the problem of 3D reconstruction using volumetric regression and produce a training set of 25,000 pairs of images and volumes using hand-crafted 3D models of bananas rendered in a 3D modelling environment (Blender).Each image is then enhanced by a GAN to more closely match the domain of photographs of real images by introducing a volumetric consistency loss, improving performance of 3D reconstruction on real images.Our solution harnesses the cost benefits of synthetic data while still maintaining good performance on real world images.We focus this work on the task of 3D banana reconstruction from a single image, representing a common task in plant phenotyping, but this approach is general and may be adapted to any 3D reconstruction task including other plant species and organs.

Plant phenotyping relevance

果実の3D再構成と体積回帰を対象とする教師なしドメイン適応手法を開発しており、植物器官の形態・サイズ推定に用いるフェノタイピング手法が研究の中心である。

Code and data availability

The paper's synthetic banana image-volume dataset (25,000 image-volume pairs) is publicly deposited at the authors' project site, and the pipeline/training code is deposited on the authors' GitHub. Both are paper-specific, public, and actionable.