浏览全部资源
扫码关注微信
Received：06 August 2019，
Revised：2019-10-28，
Accepted：04 November 2019，
Published：16 May 2020
移动端阅览
目的
2
非均匀盲去运动模糊是图像处理和计算机视觉中的基础课题之一。传统去模糊算法有处理模糊种类单一、耗费时间两大缺点，且一直未能有效解决。随着神经网络在图像生成领域的出色表现，本文把去运动模糊视为图像生成的一种特殊问题，提出一种基于神经网络的快速去模糊方法。
方法
2
首先，将图像分类方向表现优异的密集连接卷积网络（dense connected convolutional network，DenseNets）应用到去模糊领域，该网络能充分利用中间层的有用信息。在损失函数方面，采用更符合去模糊目的的感知损失（perceptual loss），保证生成图像和清晰图像在内容上的一致性。采用生成对抗网络（generative adversarial network，GAN），使生成的图像在感官上与清晰图像更加接近。
结果
2
通过测试生成图像相对于清晰图像的峰值信噪比（peak signal to noise ratio，PSNR），结构相似性（structural similarity，SSIM）和复原时间来评价算法性能的优劣。相比DeblurGAN（blind motion deblurring using conditional adversarial networks），本文算法在GOPRO测试集上的平均PSNR提高了0.91，复原时间缩短了0.32 s，能成功恢复出因运动模糊而丢失的细节信息。在Kohler数据集上的性能也优于当前主流算法，能够处理不同的模糊核，鲁棒性强。
结论
2
本文算法网络结构简单，复原效果好，生成图像的速度也明显快于其他方法。同时，该算法鲁棒性强，适合处理各种因运动模糊而导致的图像退化问题。
Objective
2
Non-uniform blind deblurring is a challenging problem in image processing and computer vision communities. Motion blur can be caused by a variety of reasons
such as the motion of multiple objects
camera shake
and scene depth variation. Traditional methods applied various constraints to model the characteristics of blur and utilized different natural image prior to the regularization of the solution space. Most of these methods involve heuristic parameter-tuning and expensive computation. Blur kernels are more complicated than these assumptions. Thus
these methods are not useful for real world images. Impressive results have been obtained in image processing with the development of neural networks. Scholars use neural networks for image generation. In this study
motion deblurring is regarded as a special problem of image generation. We also propose a fast deblurring method based on neural network without using multi-scale
unlike other scholars.
Method
2
First
this study adopts the densely connected convolutional network (DenseNets) which recently performed well in image classification direction. Improvements are made for the model to make it suitable for image generation. Our network is a full convolutional network designed to accept various sizes of input images. The input images are trained through two convolutional layers to obtain a total of 256 feature maps with the dimension of 64×64 pixels. Then
these feature maps are introduced into the DenseNets containing bottleneck layers and transitions. The output of bottleneck layers in each dense block is 1 024 feature maps
while the output of the last convolution layer of each dense block is 256 feature maps. Finally
the output of the DenseNets is restored to the size of the original image by three convolutional layers. A residual connection is added between the input and output to preserve the color information of the original image as much as possible. We also speed up the time of training. To ensure the efficiency of deblurring
this network uses only nine dense layers and does not use the multi-scale model adopted by other scholars. However
this method can still guarantee the quality of the restored image. In this study
every layer in DenseNets is connected. With this method
the intermediate information in the network can be fully utilized. In our experiment
the denser the connection among the layers
the better the results obtained. Generative adversarial networks (GANs) help image generation domain achieve a qualitative fly-by. The images generated this way are closer to the real-world image in terms of overall details. Therefore
we also use this idea to improve the performance of deblurring. In terms of the loss function
we use perceptual loss to measure the difference between restored and sharp images. We choose the VGG(visual geometry group)-19 conv3.3 feature maps to define the loss function because a shallower network can better represent the texture information of an image. In the image deblurring field
we focus on restoring the texture and edges of the object rather than wasting too much computation to determine its exact position. Thus
we choose the latter one in meansquare error (MSE) loss and perceptual loss.
Result
2
Compared with typical traditional deblurring methods and the recent neural network-based method
the performance of the algorithm is estimated by testing the restoring time
peak signal to noise ratio (PSNR)
and structural similarity (SSIM) between the restored and sharp images. Experimental results show that the restored image via our method is clearer than those of others. Moreover
the average PSNR on the test set increased by 0.91
which is obviously superior to that of the traditional deblurring algorithm. According to the qualitative comparison
our method can handle the details of blurry image better than other approaches can. Our proposed method can also restore the small objects
such as text in the image. Results of other relative datasets are also better than other methods. Abandoning the multi-scale approach leads to multiple advantages
such as effectively reduced parameters
shorter training time
and reduced restoring time up to 0.32 s.
Conclusion
2
Our algorithm has a simple network structure and good restoring effect. The speed of image generation is also significantly faster than that of other methods. At the same time
the proposed method is robust and suitable for dealing with various image degradation problems caused by motion blur.
Bahat Y, Efrat N and Irani M. 2017. Non-uniform blind deblurring by reblurring//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE: 3306-3314[ DOI: 10.1109/ICCV.2017.356 http://dx.doi.org/10.1109/ICCV.2017.356 ]
Deng J, Dong W, Socher R, Li L J, Li K and Li F F. 2009. ImageNet: a large-scale hierarchical image database//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 248-255[ DOI: 10.1109/CVPR.2009.5206848 http://dx.doi.org/10.1109/CVPR.2009.5206848 ]
Fergus R, Singh B, Hertzmann A, Roweis S T and Freeman W T. 2006. Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3):787-794[DOI:10.1145/1141911.1141956]
Gong D, Yang J, Liu L Q, Zhang Y N, Reid L, Shen C H, Van Den Hengel A andShi Q F. 2017. From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 3806-3815[ DOI: 10.1109/CVPR.2017.405 http://dx.doi.org/10.1109/CVPR.2017.405 ]
Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press: 2672-2680
Gupta A, Joshi N, Zitnick C L, Cohen M and Curless B. 2010. Single image deblurring using motion density functions//Proceedings of the 11th European Conference on Computer Vision. Heraklion: Springer: 171-184[ DOI: 10.1007/978-3-642-15549-9_13 http://dx.doi.org/10.1007/978-3-642-15549-9_13 ]
Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2261-2269[ DOI: 10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]
Johnson J, Alahi A and Li F F. 2016. Perceptual losses for real-time style transfer and super-resolution//Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer: 694-711[ DOI: 10.1007/978-3-319-46475-6_43 http://dx.doi.org/10.1007/978-3-319-46475-6_43 ]
Kim T H and Lee K M. 2014. Segmentation-free dynamic scene deblurring//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 2766-2773[ DOI: 10.1109/CVPR.2014.348 http://dx.doi.org/10.1109/CVPR.2014.348 ]
Kupyn O, Budzan V, Mykhailych M, Mishkin D and Matas J. 2018. DeblurGAN: blind motion deblurring using conditional adversarial networks//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 8183-8192[ DOI: 10.1109/CVPR.2018.00854 http://dx.doi.org/10.1109/CVPR.2018.00854 ]
Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z H and Shi W Z. 2017. Photo-realistic single image super-resolution using a generative adversarial network//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 105-114[ DOI: 10.1109/CVPR.2017.19 http://dx.doi.org/10.1109/CVPR.2017.19 ]
Nah S, Kim T H and Lee K M. 2017. Deep multi-scale convolutional neural network for dynamic scene deblurring//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 257-265[ DOI: 10.1109/CVPR.2017.35 http://dx.doi.org/10.1109/CVPR.2017.35 ]
Noroozi M, Chandramouli P and Favaro P. 2017. Motion deblurring in the wild//Proceedings of the 39th German Conference on Pattern Recognition. Basel: Springer: 65-77[ DOI: 10.1007/978-3-319-66709-6_6 http://dx.doi.org/10.1007/978-3-319-66709-6_6 ]
Su S C, Delbracio M, Wang J, Sapiro G, Heidrich W and Wang O. 2017. Deep video deblurring for hand-held cameras//Proceedings ofthe IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 237-246[ DOI: 10.1109/CVPR.2017.33 http://dx.doi.org/10.1109/CVPR.2017.33 ]
Sun J, Cao W F, Xu Z B and Ponce J. 2015. Learning a convolutional neural network for non-uniform motion blur removal//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 769-777[ DOI: 10.1109/CVPR.2015.7298677 http://dx.doi.org/10.1109/CVPR.2015.7298677 ]
Sun S J, Zhao H C, Li B, Hao M G and Lyu J F. 2017. Blind deconvolution for noisy and blurry images using low rank prior. Journal of Electronics and Information Technology, 39(8):1919-1926
Whyte O, Sivic J, Zisserman A and Ponce J. 2012. Non-uniform deblurring for shaken images. International Journal of Computer Vision, 98(2):168-186[DOI:10.1007/s11263-011-0502-7]
Xu L and Jia J Y. 2010. Two-phase kernel estimation for robust motion deblurring//Proceedings of the 11th European Conference on Computer Vision. Heraklion: Springer: 157-170[ DOI: 10.1007/978-3-642-15549-9_12 http://dx.doi.org/10.1007/978-3-642-15549-9_12 ]
Xu L, Zheng S C and Jia J Y. 2013. Unnatural l0 sparse representation for natural image deblurring//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE: 1107-1114[ DOI: 10.1109/CVPR.2013.147 http://dx.doi.org/10.1109/CVPR.2013.147 ]
Zeiler M D and Fer gus R. 2014. Visualizing and understanding convolutional networks//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: Springer: 818-833[ DOI: 10.1007/978-3-319-10590-1_53 http://dx.doi.org/10.1007/978-3-319-10590-1_53 ]
Zhang L, Sheng B and Ma L Z. 2013. Motion deblurring based on image segmentation. Journal of Image and Graphics, 18(4):467-475
张璐, 盛斌, 马利庄. 2013.分区域去运动模糊.中国图象图形学报, 18(4):467-475[DOI:10.11834/jig.20130415]
相关文章
相关作者
相关机构
京公网安备11010802024621
