Multi‐Spectral Gaussian Splatting with Neural Color Representation

Abstract

Abstract 3D Gaussian Splatting (3DGS) [KKLD23] has transformed novel‐view synthesis from RGB images, yet remains restricted to the visible spectrum. Many applications, including agricultural monitoring, rely on multi‐spectral imaging, where spectral camera alignment and scalability pose major challenges. We present MS‐Splatting—a multi‐spectral 3DGS framework enabling unified multi‐view consistent reconstruction and rendering across both visible and invisible spectra. Our key component is a neural color representation that encodes per‐primitive features shared across spectral bands, decoded through a shallow multi‐layer perceptron into spectrum‐specific radiance. By leveraging inter‐band correlations, this formulation enhances detail while reducing memory consumption compared to independent band modeling via per‐channel modeling with spherical harmonics. Our method enables accurate parallax‐free novel‐view vegetation index rendering for plant monitoring and enhances RGB novel view synthesis quality by exploiting details revealed through multi‐spectral bands. Our evaluation demonstrates that MS‐Splatting exceeds the current leading methods in both categories. In addition, we introduce a multi‐spectral dataset from aerial captures covering outdoor environments, specifically designed for evaluating these applications. We will release our code and dataset to facilitate further research. The project page is located at: https://meyerls.github.io/ms_splatting

Plant phenotyping relevance

マルチスペクトル3D再構成法を開発し、植物モニタリング用の植生指数レンダリングを実現することが中心で、評価用データセットも提供している。

Code and data availability

The paper introduces the MS-Splatting multi-spectral dataset (drone-captured RGB/NIR/RE/G/R imagery for vegetation-index plant monitoring) and author analysis code, but both are only promised for future release; no confirmed public deposit exists yet. The project page URL is given, so the authors' page is the contact/с

No evidence-backed public reproduction asset is currently recorded.