Current Issue Cover
密集连接卷积网络图像去模糊

吴迪, 赵洪田, 郑世宝(上海交通大学电子信息与电气工程学院, 上海 200240)

摘 要
目的 非均匀盲去运动模糊是图像处理和计算机视觉中的基础课题之一。传统去模糊算法有处理模糊种类单一、耗费时间两大缺点,且一直未能有效解决。随着神经网络在图像生成领域的出色表现,本文把去运动模糊视为图像生成的一种特殊问题,提出一种基于神经网络的快速去模糊方法。方法 首先,将图像分类方向表现优异的密集连接卷积网络(dense connected convolutional network, DenseNets)应用到去模糊领域,该网络能充分利用中间层的有用信息。在损失函数方面,采用更符合去模糊目的的感知损失(perceptual loss),保证生成图像和清晰图像在内容上的一致性。采用生成对抗网络(generative adversarial network,GAN),使生成的图像在感官上与清晰图像更加接近。结果 通过测试生成图像相对于清晰图像的峰值信噪比 (peak signal to noise ratio,PSNR),结构相似性 (structural similarity,SSIM)和复原时间来评价算法性能的优劣。相比DeblurGAN(blind motion deblurring using conditional adversarial networks),本文算法在GOPRO测试集上的平均PSNR提高了0.91,复原时间缩短了0.32 s,能成功恢复出因运动模糊而丢失的细节信息。在Kohler数据集上的性能也优于当前主流算法,能够处理不同的模糊核,鲁棒性强。结论 本文算法网络结构简单,复原效果好,生成图像的速度也明显快于其他方法。同时,该算法鲁棒性强,适合处理各种因运动模糊而导致的图像退化问题。
关键词
Motion deblurring method based on DenseNets

Wu Di, Zhao Hongtian, Zheng Shibao(School of Electronic information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China)

Abstract
Objective Non-uniform blind deblurring is a challenging problem in image processing and computer vision communities. Motion blur can be caused by a variety of reasons, such as the motion of multiple objects, camera shake, and scene depth variation. Traditional methods applied various constraints to model the characteristics of blur and utilized different natural image prior to the regularization of the solution space. Most of these methods involve heuristic parameter-tuning and expensive computation. Blur kernels are more complicated than these assumptions. Thus, these methods are not useful for real world images. Impressive results have been obtained in image processing with the development of neural networks. Scholars use neural networks for image generation. In this study, motion deblurring is regarded as a special problem of image generation. We also propose a fast deblurring method based on neural network without using multi-scale, unlike other scholars. Method First, this study adopts the densely connected convolutional network (DenseNets) which recently performed well in image classification direction. Improvements are made for the model to make it suitable for image generation. Our network is a full convolutional network designed to accept various sizes of input images. The input images are trained through two convolutional layers to obtain a total of 256 feature maps with the dimension of 64×64 pixels. Then, these feature maps are introduced into the DenseNets containing bottleneck layers and transitions. The output of bottleneck layers in each dense block is 1 024 feature maps, while the output of the last convolution layer of each dense block is 256 feature maps. Finally, the output of the DenseNets is restored to the size of the original image by three convolutional layers. A residual connection is added between the input and output to preserve the color information of the original image as much as possible. We also speed up the time of training. To ensure the efficiency of deblurring, this network uses only nine dense layers and does not use the multi-scale model adopted by other scholars. However, this method can still guarantee the quality of the restored image. In this study, every layer in DenseNets is connected. With this method, the intermediate information in the network can be fully utilized. In our experiment, the denser the connection among the layers, the better the results obtained. Generative adversarial networks (GANs) help image generation domain achieve a qualitative fly-by. The images generated this way are closer to the real-world image in terms of overall details. Therefore, we also use this idea to improve the performance of deblurring. In terms of the loss function, we use perceptual loss to measure the difference between restored and sharp images. We choose the VGG(visual geometry group)-19 conv3.3 feature maps to define the loss function because a shallower network can better represent the texture information of an image. In the image deblurring field, we focus on restoring the texture and edges of the object rather than wasting too much computation to determine its exact position. Thus, we choose the latter one in meansquare error (MSE) loss and perceptual loss. Result Compared with typical traditional deblurring methods and the recent neural network-based method, the performance of the algorithm is estimated by testing the restoring time, peak signal to noise ratio (PSNR), and structural similarity (SSIM) between the restored and sharp images. Experimental results show that the restored image via our method is clearer than those of others. Moreover, the average PSNR on the test set increased by 0.91, which is obviously superior to that of the traditional deblurring algorithm. According to the qualitative comparison, our method can handle the details of blurry image better than other approaches can. Our proposed method can also restore the small objects, such as text in the image. Results of other relative datasets are also better than other methods. Abandoning the multi-scale approach leads to multiple advantages, such as effectively reduced parameters, shorter training time, and reduced restoring time up to 0.32 s. Conclusion Our algorithm has a simple network structure and good restoring effect. The speed of image generation is also significantly faster than that of other methods. At the same time, the proposed method is robust and suitable for dealing with various image degradation problems caused by motion blur.
Keywords

订阅号|日报