Privileged-depth modulation for close-range pepper organ segmentation in precision crop spraying

Abstract

Smartphone-based precision spraying aims to restrict chemical application to target organs, but it also requires an organ-level perception model that operates on RGB alone at deployment, without dedicated depth sensors. Combining with monocular depth foundation models such as Depth Anything V2, RGB images can be enriched with dense pseudo-depth during training, turning close-range crop segmentation into a privileged-depth-guided learning problem where depth is available at training but absent at inference. Based on such close-range handheld capture, intra-foreground depth separability collapses while only the foreground–background depth gap remains partially preserved, exposing an imaging regime that we refer to as Close-Range Depth Degeneracy (CRDD) and that undermines the discriminative value of monocular depth for organ segmentation. Even with the available privileged-depth signal, existing RGB-D fusion and depth-branch hallucination routes still either introduce inference-time depth dependence or fail to retain the depth-conditioned effect inside RGB-side parameters under CRDD. How monocular pseudo-depth can serve as training-only privileged information for RGB-only close-range crop spraying under CRDD remains insufficiently studied. To this end, this paper proposes a privileged-depth modulation framework, MDM-Seg, by converting monocular depth from an inference-time modality into a CRDD-conditioned training signal to remove the inference-time depth dependence. Specifically, we characterize CRDD and propose a four-indicator depth-usability diagnostic that delivers a pre-training verdict on the target dataset, with verdicts validated against post-training segmentation outcomes. Then, we formalize the CRDD-conditioned training intervention as a depth-conditioned affine network (DepAN) that injects depth-gradient cues into decoder features, and define a depth hard-pixel mining loss (DHPM) to reweight per-pixel cross-entropy with depth-value evidence, both operating inside the main segmentation stream. Finally, we deploy MDM-Seg on RGB input alone and derive a lightweight consistency trace between the RGB-only and depth-conditioned heads, producing an image-level reliability cue for targeted-versus-fallback spraying without depth at inference. Extensive experiments on Pepper-Field-3940 demonstrate that our MDM-Seg achieves effective depth-conditioned RGB-only segmentation under CRDD, i.e., depth-conditioned modulation retained inside RGB-side parameters with no inference-time depth dependence, such as mIoU 0.8995 with the ResNet-101 configuration ( + 3.73 pp over the RGB-only DeepLabV3+ baseline), a lightweight training-side DepAN module, and an image-level reliability cue rank-correlated with segmentation quality (Spearman ρ = 0.74 ). On a Jetson Orin NX 16 GB with TensorRT FP16, the complete deployment-oriented MobileNetV2 dual-head prescription pipeline runs at 23.87 ms per image, corresponding to 41.89 FPS, 0.62 GB peak memory, and 0.56 J per image.

Plant phenotyping relevance

植物器官の画像セグメンテーション手法を開発・評価し、散布対象器官の状態をRGB画像から推定することが中心であるため。

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.