A broadband hyperspectral image sensor with high spatio-temporal resolution.

Abstract

Hyperspectral imaging provides high-dimensional spatial-temporal-spectral information showing intrinsic matter characteristics 1-5 . Here we report an on-chip computational hyperspectral imaging framework with high spatial and temporal resolution. By integrating different broadband modulation materials on the image sensor chip, the target spectral information is non-uniformly and intrinsically coupled to each pixel with high light throughput. Using intelligent reconstruction algorithms, multi-channel images can be recovered from each frame, realizing real-time hyperspectral imaging. Following this framework, we fabricated a broadband visible-near-infrared (400-1,700 nm) hyperspectral image sensor using photolithography, with an average light throughput of 74.8% and 96 wavelength channels. The demonstrated resolution is 1,024 × 1,024 pixels at 124 fps. We demonstrated its wide applications, including chlorophyll and sugar quantification for intelligent agriculture, blood oxygen and water quality monitoring for human health, textile classification and apple bruise detection for industrial automation, and remote lunar detection for astronomy. The integrated hyperspectral image sensor weighs only tens of grams and can be assembled on various resource-limited platforms or equipped with off-the-shelf optical systems. The technique transforms the challenge of high-dimensional imaging from a high-cost manufacturing and cumbersome system to one that is solvable through on-chip compression and agile computation.

Plant phenotyping relevance

植物のクロロフィルおよび糖含量を定量可能なオンチップ・ハイパースペクトル画像センサーを開発しており、センサー技術と植物形質取得への応用が中心的である。

Code and data availability

The paper explicitly states that all data generated or analysed are available in a public GitHub repository (hyperspectral image/video dataset collected with the HyperspecI sensors) and that demo code is available in another public GitHub repository. Both are paper-specific, public, and actionable.