Current Issue Cover
通道融合的渐进增强遥感图像全色锐化算法

贾雅男, 郭晓杰(天津大学智能与计算学部, 天津 300350)

摘 要
目的 遥感图像融合的目的是将低空间分辨率的多光谱图像和对应的高空间分辨率全色图像融合为高空间分辨率的多光谱图像。为了解决上采样多光谱图像带来的图像质量下降和空间细节不连续问题,本文提出了渐进式增强策略,同时为了更好地融合两种图像互补的信息,提出在通道维度上进行融合的策略。方法 构建了一种端到端的网络,网络分为两个阶段:渐进尺度细节增强阶段和通道融合阶段。考虑到上采样低空间分辨率多光谱图像导致的细节模糊问题,在第1阶段将不同尺度的全色图像作为额外的信息,通过两个细节增强模块逐步增强多光谱图像;在第2阶段,全色图像在多光谱图像的每个通道上都通过结构保持模块进行融合,更好地利用两种图像的互补信息,获得高空间分辨率的多光谱图像。结果 实验在GaoFen-2和QuickBird数据集上与表现优异的8种方法进行了比较,本文算法在有参考指标峰值信噪比(peak signal-to-noise ratio,PSNR)、结构相似度(structural similarity,SSIM)、相关系数(correlation coefficient,CC)和总体相对误差(erreur relative globale adimensionnelle de synthese,ERGAS)等评价指标上均取得最优值。在GaoFen-2数据集上PSNR、CC和ERGAS分别平均提高了0.872 dB、0.01和0.109,在QuickBird数据集上分别平均提高了0.755 dB、0.011和0.099。结论 本文算法在空间分辨率和光谱保持方面都取得了良好的效果,生成了质量更高的融合结果。
关键词
Remote sensing pan-sharpening based on channel fusion and progressive enhancement

Jia Yanan, Guo Xiaojie(College of Intelligence and Computing, Tianjin University, Tianjin 300350, China)

Abstract
Objective Remote sensing (RS) image fusion issue is focused on developing high-resolution multispectral (HRMS) images through the integration of low-resolution multispectral (LRMS) images and corresponding panchromatic (PAN) high spatial resolution images. Pan-sharpening has been widely developing as a pre-processing tool in the context of multiple vision applications like object detection, environmental surveillance, landscape monitoring, and scenario segmentation. The key issue for pan-sharpening is concerned of different and specific information gathering from multi-source images. The pan-sharpening methods can be divided into multiple methods in related to 1) component substitution (CS), 2) multi-resolution analysis (MRA), 3) model-based, and 4) deep-learning-based. The CS-based easy-use method is challenged for the severe spectral-distorted problem of multi-features-derived between PAN and LRMS images. The multi-resolution analysis (MRA) methods can be used to extract the spatial features from PAN images by multi-scale transformation. These features of high resolution are melted into the up-sampled LRMS images. Although spatial details can be preserved well by these methods, spectral information is likely to be corrupted by the features-melted. For model-based methods, an optimized algorithm is complicated and time-consuming for the model. The deep-learning-based method is qualified but two challenging problems to be resolved: 1) multispectral images up-sampling-derived image quality degradation; 2) multichannel variability-lacked insufficient integration. To alleviate the problems mentioned above, we implement a channel-fused strategy to mine two modalities of information better. Additionally, to resolve the image quality degradation caused by up-sampling multispectral images, a detailed progressive-enhanced module is proposed as well. Method Most of deep learning-based methods are linked to up-sample the multispectral image straightforward to maintain the same size as the panchromatic image, which degrades the image quality and lose some of the spatial details. To obtain enhanced results gradually, we carry out an implementation for progressive scale detail enhancement via the information of multi-scale panchromatic images. A channel fusion strategy is used to fuse two images in terms of an enhanced multispectral image and the corresponding panchromatic image. The effective and efficient information of the two modalities can be captured for the HRMS-predicted. The process of channel fusion can be summarized in three steps, which are 1) decomposition, 2) fusion, and 3) reconstruction. Specifically, each channel of the enhanced multispectral image is concatenated with the panchromatic image in the decomposition, and a shallow feature is obtained by two 3×3 convolutional layers. The following fusion step is based on a new fusion strategy in terms of 8 structure-preserved modules over the channels. Each structure-preserved module has four branches, the number of them is equal to the number of the channels in the multispectral image, and each branch can be used to extract features from convolutional layers, while residual connections are added to each branch for efficient information transfer. For the reconstruction, to reconstruct high-resolution multispectral images, the obtained features of each channel are first re-integrated through remapping the features. Result Our model is compared to 8 state-of-the-art saliency models, including the traditional approaches and deep learning methods on two datasets, called GaoFen-2 and QuickBird. The quantitative evaluation metrics are composed of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), correlation coefficient (CC), spectral angle mapper (SAM), erreur relative globale adimensionnelle de synthese (ERGAS), quality-with-no-reference (QNR) Dλ and DS. Compared to the other results in two datasets, the PSNR, SSIM, CC, and ERGAS can be increased by 0.872 dB, 0.005, 0.01, and 0.109 on average for GaoFen-2 dataset, and each of the four factors are improved by 0.755 dB, 0.011, 0.004, and 0.099 on the QuickBird dataset. Furthermore, to clarify the effectiveness of different modules of the fusion algorithm, a series of ablation experiments are conducted. Conclusion A novel of framework is developed based on the two phases for pan-sharpening, which can effectively enhance the detail information of LRMS images and produce appealing HRMS images. The progressive detail enhancement step can enhance the LRMS images via the extra multi-scale PAN images-related information fusion, while the channel fusion step can fuse the channel features in terms of structure-preserved modules. To verify the effectiveness of our designs, a series of ablation studies are carried out. Experimental results on several widely-used datasets are also provided to demonstrate the advances of our method in comparison with other state-of-the-art methods.
Keywords

订阅号|日报