Photorealistic Arm Robot Simulation for 3D Plant Reconstruction and Automatic Annotation using Unreal Engine 5

Abstract

Robotics paired with computer vision are widely used in precision agriculture. Simulations are critical for safety and performance estimation by verifying their routine in a virtual world before real-world testing and deployment. However, many simulators used in agricultural robots lack photorealism in their virtual worlds compared to the real world. We implemented Unreal Engine 5 (UE5) and the Robot Operating System (ROS) to develop a robot simulator tailored to agricultural tasks and synthetic data generation with RGB, segmentation, and depth images. We designed a method for assigning multiple segmentation labels within a single plant mesh. We experimented with a semi-spherical routine for two robot arms to perform 3D point cloud reconstruction across 10 plant assets. We showed our simulator produces much more accurate segmentation images and reconstruction compared to existing UE5 solutions. We extend our results with Neural Radiance Field (NeRF) reconstructions. The packaged simulator, UE5 project, and ROS package with the Python routine can be found at https://github.com/NCSU-BAE-ARLab/AgriRoboSimUE5.

Plant phenotyping relevance

植物の3D再構成・自動アノテーションのためのロボットシミュレータと合成画像生成手法を開発し、精度比較も行っており、表現型取得・抽出基盤が研究の中心である。

Code and data availability

公開本文の所在を確認できませんでした。非公開または購読が必要な可能性があります。

No evidence-backed public reproduction asset is currently recorded.