From Appearance to Inherence: A Hyperspectral Image Dataset and Benchmark of Material Classification for Surveillance

Image understanding and analysis primarily rely on object appearances. However, when faced with challenges such as occlusion, camouflage, and small targets in surveillance scenarios, the discriminative features of objects cannot be effectively extracted. This paper proposes the utilization of hyperspectral imaging techniques as a solution to these issues, aiming to unlock new potential in the field. Hyperspectral images have the unique capability to identify a variety of materials, thereby offering a distinct advantage in surveillance. However, existing hyperspectral image datasets are not specifically tailored for image classification tasks within surveillance scenarios. To address this issue, we introduce an innovative hyperspectral image dataset designed explicitly for real-world surveillance, with the goal of setting a new benchmark for material classification. Our aspiration extends beyond merely deploying deep learning methods for hyperspectral material classification, aiming to contribute insightful understanding of spectral patterns inherent in natural surveillance scenes. The proposed dataset is currently the largest of its kind and the first one designed specifically for surveillance scenarios. It encompasses 128 spectral bands and provides annotations for 28 common material categories. Furthermore, we introduce a novel texture-metric-based spatial and spectral fusion network, meticulously crafted to accommodate our unique scenario and dataset. This model significantly outperforms existing networks in enhancing the fusion of spatial and spectral features, achieving state-of-the-art results on both our proposed dataset and existing public hyperspectral image classification datasets.