Objective

2

Hyperspectral remote sensing method is a major development in remote sensing field. It uses a lot of narrow band electromagnetic bands to obtain spectral data. It covers visible

near infrared

middle infrared

and far infrared bands

and its spectral resolution can reach the nanometer level. Therefore

hyperspectral remote sensing can find more surface features and has been widely used in covering global environment

land use

resource survey

natural disasters

and even interstellar exploration. Compared with RGB and multispectral images

hyperspectral images not only can improve the information richness but also can provide more reasonable and effective analysis and processing for the related tasks. As a result

they have important application value in many fields. However

the cost of spectral detection systems is relatively high

especially the optical detector that is used to acquire high spectral data. At present

most of the spectrometers can support the spectral imaging from 400 nm to 1 000 nm

while few of them support that from 1 000 nm to 2 500 nm. The reason is that the spectrometer is harder to produce and more expensive with the increase in spectra. The bands of hyperspectral images have internal relations. The performance of low-spectrum spectrometer can be improved by fully utilizing the low spectra to predict high spectra. In other words

the low spectrum spectrometer can be used to obtain the high spectra that are near the spectra which are usually obtained by high-spectrum spectrometer. The cost of getting hyperspectral images will be greatly reduced. Therefore

high spectra prediction has promising applications and prospects in improving spectrometer performance. Nowadays

a single sensor can generally take a limited number of spectra. Thus

the commonly used spectrometers contain multiple sensors. If one of these sensors suffers from a sudden situation and cannot work normally in the process of flight aerial photography

then the data we can obtain will be unusable and we will have to have a flight again

which will cause cost increase and resource waste. Similarly

if a spectrometer mounted on a satellite fails to work normally in case of emergency

then it will suffer much greater loss. However

if we can fully utilize the low spectra to predict high spectra

which means using the low-spectrum spectrometer to obtain the hyperspectral image that is near the spectra from real high-spectrum spectrometer

the loss caused by these situations can be compensated in a great extent.

Method

2

In recent years

convolutional neural networks (CNNs) have been widely used in various image processing tasks. We propose a hyperspectral image prediction framework based on a CNN as inspired by the great achievements of deep learning in the field of image spatial super resolution. The designed network is based on the residual network

which can fully use multiscale feature maps to obtain better performance and ensure fast convergence. In the CNN

2D convolution layers use convolution kernels to obtain feature maps

and convolution kernels use relation between space and spectra

which is also helpful to obtain better results. In our network

each of the convolution layers has an activation layer

in which the rectified linear unit function is used. Batch normal layers are used to normalize the feature map

which can improve the feature extracting ability of CNN. Given an input

the proposed network extracts the low-band data features of the hyperspectral image. Then

it uses the extracted features together with the original low-spectra data to predict the high-spectra data for predicting the high spectra with the low spectra. We also design an evaluation system to prove the feasibility and effectiveness of the infrared spectrum prediction. The feasibility is evaluated by three classical image quality evaluation indices (peak signal-to-noise ratio (PSNR)

structural similarity (SSIM)

and spectral angle (SA)). The feasibility is also evaluated by two classical classification evaluation indices (accuracy and average accuracy) by applying our predicted infrared spectrum to classification tasks.

Result

2

Experiments on Cuprite and Salinas datasets are conducted to validate the effectiveness of the proposed method. On Cuprite dataset

we directly measure the quality of the predicted image through PSNR

SSIM

and SA. On Salinas dataset

we mainly use the predicted image data for classification tasks with support vector machine (SVM) and LeNet. All the experiments are implemented using Torch 1.3 platform with Python 3.7. In our experiments on Cuprite dataset

we use the spectra of the first two sensors to predict the spectra of the third sensor. Five hyperspectral images are present in the original data of Cuprite. The first three spectra of Cuprite are spliced into a large image as the training dataset

and the last two spectra are spliced as the test dataset. In this experiment

30 training epochs are conducted. The PSNR

SSIM

and SA of the predicted images by the trained network on the test set are 40.145 dB

0.996

and 0.777 rad

respectively

which indicates that the proposed method can predict high spectra from low spectra

which is near the ground truth. The PSNR

SSIM

and SA on the Salinas dataset are 39.55 dB

0.997

and 1.78 rad

respectively. The accuracy and average accuracy of SVM and LeNet by using the predicted high-spectra data for classification are both improved by approximately 1% compared with the results which use only low-spectra data.

Conclusion

2

Although many CNN methods have been proposed to realize spatial super resolution

few of them realize spectral super resolution

which is also important. Therefore

we propose the new application in remote sensing field called spectrum prediction

which uses a CNN to predict high spectra from low spectra. The proposed method can expand the use efficiency of sensor chips and also help deal with spectrometer failure and improve the quality of spectral data.