A two-stage fine-tuning strategy for 3D object segmentation from multi-view images

Abstract

Neural Radiance Fields (NeRF) have been widely adopted for reconstructing high-quality 3D scenes from 2D RGB images. However, achieving accurate 3D object segmentation within these reconstructed scenes remains challenging. Existing NeRF-based segmentation methods either rely on post-processing (SA3D), which produces noisy point clouds due to the absence of density field optimization, or employ joint training with additional segmentation heads (FruitNeRF), which can lead to suboptimal performance due to conflicting learning objectives. In this work, we propose InvNeRF-Seg (Input-substitution NeRF for Segmentation), a two-stage fine-tuning strategy for 3D object segmentation that preserves the original NeRF architecture and loss function entirely. We first train a standard NeRF on RGB images and then fine-tune it using 2D segmentation masks formatted as RGB-like inputs, without introducing any architectural modifications or additional loss functions. This input-substitution approach reshapes the density field to align with object regions while suppressing background density. We validate InvNeRF-Seg through comprehensive ablation studies examining the roles of density and color MLPs, loss function choices, and training strategies. Field density analysis reveals consistent semantic refinement: densities of object regions increase while background densities are suppressed. Experiments on synthetic fruit datasets and real-world soybean imagery demonstrate that InvNeRF-Seg produces cleaner 3D segmented point clouds compared to both SA3D and FruitNeRF, enabling more accurate downstream object counting. The method is further validated on a self-collected soybean dataset to demonstrate its applicability in real-world agricultural scenarios. Our code is available at https://github.com/ZJiangsan/InvNeRF-Seg .

Plant phenotyping relevance

植物画像から3D物体領域を抽出するNeRFベース手法の開発・比較検証が中心で、果実・ダイズ画像を対象に物体カウントへ応用しているため、植物器官の形態・数量推定に関わるフェノタイピング手法として含める。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.