Current Issue Cover
近红外高光谱图像数据预测技术

梅少辉1, 张博威1, 马明阳1, 贾森2(1.西北工业大学电子信息学院, 西安 710129;2.深圳大学计算机与软件学院, 深圳 518060)

摘 要
目的 受到传感器光谱响应范围的影响,可见光区域和近红外区域(400~2 500 nm)的高光谱数据通常使用不同的感光芯片进行成像,现有这一光谱区域典型的高光谱成像系统,如AVIRIS (airborne visible infrared imaging spectrometer)成像光谱仪,通常由多组感光芯片组成,整个成像系统成本和体积通常比较大,严重限制了该谱段高光谱探测技术的发展。为了能够扩展单感光芯片成像系统获得的高光谱图像的光谱范围,本文探索基于卷积神经网络的近红外光谱数据预测技术。方法 结合AVIRIS成像光谱仪的光谱配置,设计了基于残差学习的红外谱段图像预测网络,利用计算成像的方式从可见光范围的高光谱图像预测出近红外波段的光谱图像,并在典型的卫星高光谱遥感数据上进行红外光谱预测重构和基于重构的数据分类实验,以验证论文提出的红外光谱数据预测技术的可行性以及有效性。结果 本文设计的预测网络在Cuprite数据集上得到的预测近红外图像峰值信噪比为40.145 dB,结构相似度为0.996,光谱角为0.777 rad;在Salinas数据集上得到的预测近红外图像峰值信噪比为39.55 dB,结构相似性为0.997,光谱角为1.78 rad。在分类实验中,相比于只使用可见光图像,利用预测的近红外图像使得支持向量机(support vector machine,SVM)的准确率提升了0.6%,LeNet的准确率提升了1.1%。结论 基于AVIRIS传感器获取的两组典型卫星高光谱数据实验表明,本文提出的红外光谱数据预测技术不仅可基于计算成像的方式扩展可见光光谱成像系统的光谱成像范围,对于减小成像系统体积和质量具有重要意义,而且可有效提高可见光区域光谱图像数据在典型应用中的处理性能,对于提高高光谱数据处理精度提供新的技术支撑。
关键词
Predicting near-infrared hyperspectral images from visible hyperspectral images

Mei Shaohui1, Zhang Bowei1, Ma Mingyang1, Jia Sen2(1.School of Electronics and Information, Northwestren Polytechnical University, Xi'an 710129, China;2.College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China)

Abstract
Objective Hyperspectral remote sensing method is a major development in remote sensing field. It uses a lot of narrow band electromagnetic bands to obtain spectral data. It covers visible, near infrared, middle infrared, and far infrared bands, and its spectral resolution can reach the nanometer level. Therefore, hyperspectral remote sensing can find more surface features and has been widely used in covering global environment, land use, resource survey, natural disasters, and even interstellar exploration. Compared with RGB and multispectral images, hyperspectral images not only can improve the information richness but also can provide more reasonable and effective analysis and processing for the related tasks. As a result, they have important application value in many fields. However, the cost of spectral detection systems is relatively high, especially the optical detector that is used to acquire high spectral data. At present, most of the spectrometers can support the spectral imaging from 400 nm to 1 000 nm, while few of them support that from 1 000 nm to 2 500 nm. The reason is that the spectrometer is harder to produce and more expensive with the increase in spectra. The bands of hyperspectral images have internal relations. The performance of low-spectrum spectrometer can be improved by fully utilizing the low spectra to predict high spectra. In other words, the low spectrum spectrometer can be used to obtain the high spectra that are near the spectra which are usually obtained by high-spectrum spectrometer. The cost of getting hyperspectral images will be greatly reduced. Therefore, high spectra prediction has promising applications and prospects in improving spectrometer performance. Nowadays, a single sensor can generally take a limited number of spectra. Thus, the commonly used spectrometers contain multiple sensors. If one of these sensors suffers from a sudden situation and cannot work normally in the process of flight aerial photography, then the data we can obtain will be unusable and we will have to have a flight again, which will cause cost increase and resource waste. Similarly, if a spectrometer mounted on a satellite fails to work normally in case of emergency, then it will suffer much greater loss. However, if we can fully utilize the low spectra to predict high spectra, which means using the low-spectrum spectrometer to obtain the hyperspectral image that is near the spectra from real high-spectrum spectrometer, the loss caused by these situations can be compensated in a great extent. Method In recent years, convolutional neural networks (CNNs) have been widely used in various image processing tasks. We propose a hyperspectral image prediction framework based on a CNN as inspired by the great achievements of deep learning in the field of image spatial super resolution. The designed network is based on the residual network, which can fully use multiscale feature maps to obtain better performance and ensure fast convergence. In the CNN, 2D convolution layers use convolution kernels to obtain feature maps, and convolution kernels use relation between space and spectra, which is also helpful to obtain better results. In our network, each of the convolution layers has an activation layer, in which the rectified linear unit function is used. Batch normal layers are used to normalize the feature map, which can improve the feature extracting ability of CNN. Given an input, the proposed network extracts the low-band data features of the hyperspectral image. Then, it uses the extracted features together with the original low-spectra data to predict the high-spectra data for predicting the high spectra with the low spectra. We also design an evaluation system to prove the feasibility and effectiveness of the infrared spectrum prediction. The feasibility is evaluated by three classical image quality evaluation indices (peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and spectral angle (SA)). The feasibility is also evaluated by two classical classification evaluation indices (accuracy and average accuracy) by applying our predicted infrared spectrum to classification tasks. Result Experiments on Cuprite and Salinas datasets are conducted to validate the effectiveness of the proposed method. On Cuprite dataset, we directly measure the quality of the predicted image through PSNR, SSIM, and SA. On Salinas dataset, we mainly use the predicted image data for classification tasks with support vector machine (SVM) and LeNet. All the experiments are implemented using Torch 1.3 platform with Python 3.7. In our experiments on Cuprite dataset, we use the spectra of the first two sensors to predict the spectra of the third sensor. Five hyperspectral images are present in the original data of Cuprite. The first three spectra of Cuprite are spliced into a large image as the training dataset, and the last two spectra are spliced as the test dataset. In this experiment, 30 training epochs are conducted. The PSNR, SSIM, and SA of the predicted images by the trained network on the test set are 40.145 dB, 0.996, and 0.777 rad, respectively, which indicates that the proposed method can predict high spectra from low spectra, which is near the ground truth. The PSNR, SSIM, and SA on the Salinas dataset are 39.55 dB, 0.997, and 1.78 rad, respectively. The accuracy and average accuracy of SVM and LeNet by using the predicted high-spectra data for classification are both improved by approximately 1% compared with the results which use only low-spectra data. Conclusion Although many CNN methods have been proposed to realize spatial super resolution, few of them realize spectral super resolution, which is also important. Therefore, we propose the new application in remote sensing field called spectrum prediction, which uses a CNN to predict high spectra from low spectra. The proposed method can expand the use efficiency of sensor chips and also help deal with spectrometer failure and improve the quality of spectral data.
Keywords

订阅号|日报