A new pan-sharpening method based on adaptive weight mechanism
fangshuai,chaolei,caofengyun(HFUT;Anhui Province Key Laboratory of Industry Safety and Emergency Technology, Hefei)
Objective：In remote sensing image processing, Pan-sharpening is used to obtain a multispectral image with a high resolution both spatially and spectrally by merging a original multispectral image with low spatial resolution and a panchromatic image with a high spatial resolution of the corresponding scene. Pan-sharpening has been widely used as a pre-processing tool in a variety of vision applications, including change detection, object recognition, military intelligence, medical assistance, and disaster monitoring. Traditional Pan-sharpening methods are mainly divided into component substitution (CS) method and multiresolution analysis (MRA) method. The fusion result of the CS method has high spatial resolution but is prone to generating spectral distortion. MRA method can maintain spectral information better but there will generate spatial degradation. In order to improve the spatial resolution of the fusion result while reduce spectral distortion. We use a variational approach based on some assumptions to solve the problem. In general, the difference among bands in multispectral images is rarely considered in the variational method, resulting in the same spatial information injected into each band to cause spectral distortion. Thus, in order to reduce spectral distortion, it is necessary to inject different spatial information for each band. Most previous studies use upsampled images as prior knowledge. But there will be distorted when image is upsampled.In order to use the original multispectral image more accurately, we use a new idea to use prior knowledge. Because of the degradation of the upsampled image, the degradation process is added to the method to better maintain the spectral relationship among bands.
Method：In this study, a new pan-sharpening based on variational method is built.Because there are differences between the bands of the MS, it is not suitable to inject the same spatial information for each band. In order to avoid spectral distortion and over sharpening caused by injecting too much spatial information into multispectral images. We use spatial information constraints based on adaptive weights to inject different spatial information into each band. For the problem of degradation of upsampled multispectral image, we use both the original multispectral image and the upsampled multispectral image information in the model in different way. First, the original multispectral image used as a priori knowledge to maintain spectral information by using channel gradient constraints and local spectral consistnecy constraints. Secondly, in order to avoid the introduction of inaccurate inter-band relationship to reduce the accuracy of the fusion result. By considering the degradation process into the model, the spectral relationship correction constraint is used to constrain the fused result after the degradation to maintain the relative relationship among the bands of original multispectral image. The gradient descent algorithm is used to solve the objective function and a numerical solution framework is designed to get fused result. All the experiments are implemented in Matlab2017 and executed on a computer with an Intel (R) 3.6 GHz central processing unit and 32 GB memory.
Result： We compared the proposed model with 6 state of the art Pan-sharpening algorithms, including wavelet, GLP_R (gaussian low pass full scale regressionbased approaches), Joint IHS (joint intensity-hue-saturation and variational method), P+XS, Guided filter, NIHS (nonlinear intensity-hue-saturation method). The quantitative evaluation metrics contained CC(correlation coefficient), ERGAS (relative global synthesis error), RMSE (root mean squared error), PSNR (peak signal to noise ratio), SAM (spectral angle mapper), RASE (relative average spectral error) and SID (spectral information divergence). In order to show the validity of the model, we did 4 sets of experiments for comparison. Experimental results show that our model is outperforms all other methods in the Geoeye and Pleiades datasets, except in SAM and CC. And Comparative experiments demonstrated that the fusion algorithm improves spatial resolution while reduces spectral distortion. Compared with the suboptimal of all comparison algorithm in Pleiades dataset, the average of CC, PSNR (higher is better) increased by 0.74%, 1.85% and average of ERGAS, RASE, RMSE, SID (i.e., less is better) decreased by 8.27%, 6.71%, 6.57%, and 8.07% respectively. Compared with the suboptimal of all comparison algorithms in Geoeye dataset, the PSNR increased by an average of 1.16%, and the average of ERGAS, RASE, RMSE, SAM, SID decreased by 8.84%, 3.90% , 4.17%, 5.83% and 15.81%, respectively.
Conclusion：In this study, we proposed a new pan-sharpening model based on adaptive weight mechanism. Geoeye and Pleiades data were used as test data, and the model was compared with the 6 excellent algorithms. The experiment results show that our model outperforms these 6 state-of-the-art pan-sharpening approaches and the proposed method improves high spatial resolution while mitigating spectral distortion.