Current Issue Cover
多尺度密集残差网络的单幅图像超分辨率重建

应自炉, 龙祥(五邑大学信息工程学院, 江门 529020)

摘 要
目的 近几年应用在单幅图像超分辨率重建上的深度学习算法都是使用单种尺度的卷积核提取低分辨率图像的特征信息,这样很容易造成细节信息的遗漏。另外,为了获得更好的图像超分辨率重建效果,网络模型也不断被加深,伴随而来的梯度消失问题会使得训练时间延长,难度加大。针对当前存在的超分辨率重建中的问题,本文结合GoogleNet思想、残差网络思想和密集型卷积网络思想,提出一种多尺度密集残差网络模型。方法 本文使用3种不同尺度卷积核对输入的低分辨率图像进行卷积处理,采集不同卷积核下的底层特征,这样可以较多地提取低分辨率图像中的细节信息,有利于图像恢复。再将采集的特征信息输入残差块中,每个残差块都包含了多个由卷积层和激活层构成的特征提取单元。另外,每个特征提取单元的输出都会通过短路径连接到下一个特征提取单元。短路径连接可以有效地缓解梯度消失现象,加强特征传播,促进特征再利用。接下来,融合3种卷积核提取的特征信息,经过降维处理后与3×3像素的卷积核提取的特征信息相加形成全局残差学习。最后经过重建层,得到清晰的高分辨率图像。整个训练过程中,一幅输入的低分辨率图像对应着一幅高分辨率图像标签,这种端到端的学习方法使得训练更加迅速。结果 本文使用两个客观评价标准PSNR(peak signal-to-noise ratio)和SSIM(structural similarity index)对实验的效果图进行测试,并与其他主流的方法进行对比。最终的结果显示,本文算法在Set5等多个测试数据集中的表现相比于插值法和SRCNN算法,在放大3倍时效果提升约3.4 dB和1.1 dB,在放大4倍时提升约3.5 dB和1.4 dB。结论 实验数据以及效果图证明本文算法能够较好地恢复低分辨率图像的边缘和纹理信息。
关键词
Single-image super-resolution construction based on multi-scale dense residual network

Ying Zilu, Long Xiang(School of Information Engineering, Wuyi University, Jiangmen 529020, China)

Abstract
Objective Single-image super-resolution aims to generate a visually pleasing high-resolution image from its degraded low-resolution measurement. Single-image super-resolution is used in various computer vision tasks, such as security and surveillance imaging, medical imaging, and image generation. However, image super-resolution is an ill-posed inverse problem because of a multitude of solutions for any low-resolution input. In recent years, a series of convolution neural networks model has been proposed for single-image super-resolution. The deep learning algorithms applied to single-image super-resolution reconstructions have used single-scale convolutional kernels to extract feature information of low-resolution images, thereby causing omissions of detailed information easily. Moreover, to obtain better image super-resolution reconstruction effects, the network model is constantly deepened, and the accompanying problems of gradients vanished, thereby resulting in longer training time and difficulty. A multi-scale dense residual network model based on GoogleNet, residual network, and intensive convolution network ideas is proposed to address these existing super-resolution reconstruction problems. Method Different from the traditional single-scale feature extraction convolutional kernel, this study uses three different scales of convolution kernels 3×3, 5×5, and 7×7 to perform convolution processing on the input low-resolution images and collects the underlying features of different convolution kernels. Therefore, more detailed information on low-resolution images, which are beneficial to image restoration, are extracted. Then, the collected feature information is inputted into the residual block. Each residual block contains a number of feature extraction units consisting of convolutional and active layers. In addition, the output of each feature extraction unit is connected to the next feature extraction unit through a short path. Short-path connections can effectively alleviate the disappearance of gradients, enhance the propagation of features, and promote the reuse of features. Then, the feature information extracted by the three convolution kernels is merged, and the feature information extracted by the 3×3 pixels convolution kernel is added after dimensionality reduction processing to form a global residual learning. After a final reconstruction, a clear, high-resolution image is obtained. Throughout the training process, an input low-resolution image corresponds to a high-resolution image tag. This end-to-end learning method results in faster training. We use the mean squared error as the loss function. The loss is minimized using stochastic gradient descent with the standard backpropagation. We use a training data of 1 000 images from DIV2k, and the flipped and rotated versions of the training images are considered. We rotate the original images by 90° and 270°. Results This study uses two objective evaluation criteria, namely, peak signal-to-noise ratio and structural similarity index to test the effect map of the experiment and compare it with other mainstream methods. The final results show that compared with interpolation method and SRCNN algorithm on Set5 dataset, the proposed algorithm improves approximately 3.4 dB and 1.1 dB at three times of magnification and 3.5 dB and 1.4 dB at four times of magnification, respectively. Conclusion We propose a multi-scale dense residual network for single-image super-resolution. The experimental data and the effect graph confirm that the proposed algorithm can better recover the edge and texture information of low-resolution images. However, our networks have a large number of parameters because our algorithm uses three channels to recover image details. Therefore, our algorithm requires more convergence time. We will also reduce the number of weight parameters by decomposing the convolution kernel.
Keywords

订阅号|日报