Objective

2

GaoFen-5 (GF-5) hyperspectral data are important hyperspectral data sources at present. However

its 30 m spatial resolution limits its extensive application. Spatial-spectral fusion can fully utilize data of high spatial resolution like multispectral image (MSI) and data of high spectral resolution like hyperspectral image (HSI). It aims to generate data with high spatial and spectral resolutions at the same time. The ratio of spatial resolution between the two fusion images is usually 4. The Sentinel-2 MSI

which is 10 m resolution

is moderate for fusion with GF-5 HSI. Some scholars have applied a few typical pan-sharpening methods for MSI and HSI fusion based on artificial dataset

such as component substitution and multiresolution analysis methods. Others have adopted some model-based algorithms

such as Bayesian and matrix factorization approaches. However

the result performed on artificial dataset obeys the Wald's protocol instead of the real dataset. The main problem in the methods mentioned above is that some methods enhance the spatial information obviously but distort its spectrum

while others have high spectral fidelity but insufficient enhancement in spatial information. Therefore

we propose a new fusion algorithm called spatial-spectral fusion based on band-adaptive detail injection for GF-5 and Sentinel-2 remote sensing images to obtain the fused image with enhanced spatial resolution and high spectral fidelity. This method is based on the Gram-Schmidt (GS) transform and the nonsubsampled contourlet transform (NSCT).

Method

2

First

the band-adaptive grouping strategy is proposed to solve the difficulty in directly fusing two multi-band images. Each band of the HSI is grouped into the most relevant band of the MSI according to the correlation coefficient. This grouping strategy also improves the spectral fidelity of the fused image to some extent. Second

the minimizing mean square error estimator is used to calculate the coefficient for generating the low-resolution MSI (LMSI) in GS transform

which reduces the spectral distortion caused by simulating the LMSI with average weight coefficients in the traditional GS fusion. Then

the GS transformation is applied for the LMSI and the HSI to obtain each GS component. Third

NSCT has advantages in image denoising and enhancement

which can improve the spatial details of the fused image. NSCT is applied on the MSI

the LMSI

and the detail image generated from their difference to obtain high- and low-frequency coefficients. Next

a new high-resolution MSI (HMSI) is produced using weighted strategies. The HMSI has high spatial resolution and some spectral information because the spatial and spectral information in MSI

LMSI

and detail image are integrated into the reconstructed HMSI. Finally

the first component of GS components is replaced with the HMSI

and the image of high spatial and spectral resolutions is generated through the GS inverse transformation.

Result

2

We perform experiments on the real GF-5 data and Sentinel-2 data with abundant feature types

such as buildings

roads

mountains

plants

water

farmland

and bare land

to verify the reliability and effectiveness of the proposed method. The standard deviation

entropy

universal image quality index

correlation coefficient (CC)

erreur relative globale adimensionnelle de Synthèse (ERGAS)

and spectral angle mapper (SAM) are used as the quantitative indices to evaluate the quality of the fusion images. Compared with the typical fusion methods

the proposed method has advantages in spatial resolution and spectral fidelity. This method can improve the spatial resolution and has good spectral fidelity for the urban area which is rarely affected by the time phase. The fused image is sharp and has no noise for the vegetation area which is greatly affected by time. Compared with the indices of the traditional GS method

the CC

ERGAS

and SAM of the proposed method are improved by 8%

26%

and 28%

respectively

which indicates that the spectral fidelity of this method is greatly improved. In addition

spectral curve is an important index to evaluate the quality of HSI. The spectral quality of the fused image is evaluated by comparing the shape and numerical difference of the spectral curve between the fused image and the original HSI in each band. The result shows that the spectral curve of the proposed method is consistent with the original HSI and closer to the original HSI than the GS method.

Conclusion

2

We propose a new spatial-spectral fusion method based on band-adaptive detail injection. No noise is observed in the result

and the spectral curve is consistent with the original spectral curve. Experimental results show that the proposed method has high spatial resolution while mitigating spectral distortion. The high- and low-frequency coefficients of NSCT in the proposed method are reorganized by the weighted fusion rule. In the following research

the NSCT fusion rule can be improved to generate fusion images with better spatial details and higher spectral fidelity.