深度卷积神经网络降噪模型的技术瓶颈与研究展望

徐少平; 刘婷云; 林珍玉; 张贵珍; 李崇禧

doi:10.11834/jig.190165

学者观点 | 浏览量 : 0 下载量: 2425 CSCD: 0

PDF
导出
分享
收藏
专辑

深度卷积神经网络降噪模型的技术瓶颈与研究展望
Main bottlenecks and research prospects of the deep convolutional neural network-based denoising model
2019年24卷第8期页码：1207-1214
收稿：2019-04-29，

修回：2019-5-21，

纸质出版：2019-08-16
DOI： 10.11834/jig.190165
稿件说明：

移动端阅览

徐少平, 刘婷云, 林珍玉, 张贵珍, 李崇禧. 深度卷积神经网络降噪模型的技术瓶颈与研究展望[J]. 中国图象图形学报, 2019,24(8):1207-1214. DOI： 10.11834/jig.190165.

Shaoping Xu, Tingyun Liu, Zhenyu Lin, Guizhen Zhang, Chongxi Li. Main bottlenecks and research prospects of the deep convolutional neural network-based denoising model[J]. Journal of Image and Graphics, 2019, 24(8): 1207-1214. DOI： 10.11834/jig.190165.

摘要

现有的深度卷积神经网络（DCNN）图像降噪模型受其技术路线内在固有特性的制约，降噪性能仍然有待进一步改进。为了推动现有DCNN图像降噪模型技术的发展，需要正视并及时解决制约其进一步完善的瓶颈问题。本文简要概述了传统的基于自然图像非局部自相似性、稀疏性和低秩性这3种先验知识设计的图像降噪算法的技术路线特点和优缺点，从传统图像降噪算法存在的问题中引出基于DCNN构建图像降噪模型的技术优势，并梳理并总结了DCNN降噪模型未来的发展瓶颈，就相应的解决方案（研究方向）进行详细讨论。通过深入分析发现，可以从扩大卷积核的感受野、降低网络参数与训练集之间的依赖关系以及充分利用DCNN网络的建模能力这3个角度入手，突破现有基于数据驱动的DCNN降噪模型的瓶颈制约，把图像降噪算法的研究水平推向新的高度。

Abstract

As a representative technique for deep learning

the deep convolution neural network (DCNN) with strong feature learning and nonlinear mapping ability in the field of digital image processing offers a novel opportunity for image denoising research. DCNN-based denoising models show significant advantages over traditional methods in terms of their denoising effect and execution efficiency. However

most of the existing image denoising models are driven by data. Given their inherent restrictions

the denoising performance of these models can be further improved. To promote the development of existing image denoising technologies

some key challenges that restrict their further improvement must be analyzed and addressed. We first summarize the core ideas of traditional image denoising algorithms based on three types of prior knowledge of the natural image

namely

non-local self-similarity

sparsity

and low rank

and then analyze the advantages and disadvantages of these algorithms. Image denoising algorithms modeled with prior knowledge can flexibly deal with distorted images under different noise levels. Unfortunately

they demonstrate the following limitations:1) the limited hand-crafted image priors are not enough to describe all changes in the image structure

thereby limiting the denoising ability of these algorithms; 2) most of the traditional image denoising algorithms iteratively solve their objective functions

thereby resulting in a high computational complexity; and 3) the optimal solution of the objective function needs to adjust several parameters manually according to the actual situation. Based on the above problems

we point out that the technical advantage of the DCNN-based denoising model lies in its strong nonlinear approximation supported by a graphics processing unit. The inherent characteristics of the DCNN-based denoising model are then analyzed

the bottleneck problems that restrict their future development are presented

and the possible solutions (research directions) to these problems are discussed in detail. A thorough analysis reveals many bottleneck problems in data-driven DCNN-based denoising models that need to be solved

including:1) the small receptive field of the DCNN network that limits the range of image feature representation and the ability to fully utilize the priors contained in natural images; 2) the strong dependence of DCNN-based model parameters on the training dataset

that is

the optimal denoising effect can only be obtained if the distortion level of the observed image is close to that of the training images; and 3) the training set cannot be easily constructed and the denoising model cannot be easily trained to denoise an image if both the noise type and level of noisy image are unknown. To solve these problems

we expand the receptive field of the convolution kernel

weaken the dependency between the network parameters and the training set

or fully utilize the modeling ability of the DCNN network. Therefore

the bottlenecks of the existing DCNN denoising models can be addressed

and research on image denoising algorithms can move to a higher level. In this paper

the technical advantages and development bottlenecks of DCNN in the field of image denoising are summarized

and some future research directions for the image denoising method are proposed. This paper should be of interest to readers in the area of image denoising.

关键词

Keywords

references

Chen L, Yang G, Chen Z Y, et al. Linearized Bregman iteration algorithm for matrix completion with structural noise[J]. Chinese Journal of Computers, 2015, 38(7):1357-1371.

陈蕾, 杨庚, 陈正宇, 等.基于线性Bregman迭代的结构化噪声矩阵补全算法[J].计算机学报, 2015, 38(7):1357-1371. [DOI:10.11897/SP.J.1016.2015.01357]

Shen J M, Li J F, Dai W Z. No-reference image quality assessment based on structure information and luminance statistics[J]. Acta Electronica Sinica, 2016, 44(4):804-812.

沈军民, 李俊峰, 戴文战.结合结构信息和亮度统计的无参考图像质量评价[J].电子学报, 2016, 44(4):804-812. [DOI:10.3969/j.issn.0372-2112.2016.04.008]

Xu S P, Zeng X X, Tang Y L, et al. A noise level estimation algorithm using prior knowledge of similar images[J]. Journal of Computer Research and Development, 2018, 55(12):2741-2752.

徐少平, 曾小霞, 唐祎玲, 等.基于多图像先验知识的噪声水平评估算法[J].计算机研究与发展, 2018, 55(12):2741-2752. [DOI:10.7544/issn1000-1239.2018.20170336]

Xu S P, Zeng X X, Tang Y L. Fast noise level estimation algorithm based on two-stage support vector regression[J]. Journal of Computer-Aided Design&Computer Graphics, 2018, 30(3):447-458.

徐少平, 曾小霞, 唐祎玲.基于两阶段支持向量回归的快速噪声水平估计算法[J].计算机辅助设计与图形学学报, 2018, 30(3):447-458. [DOI:10.3724/SP.J.1089.2018.16422]

Gong X C, Li Z M. An image denoising method using deep asymmetrical skip connection[J]. Journal of Computer-Aided Design&Computer Graphics, 2019, 31(2):295-302.

公绪超, 李宗民.深度非对称跳跃连接的图像降噪方法[J].计算机辅助设计与图形学学报, 2019, 31(2):295-302. [DOI:10.3724/SP.J.1089.2019.17167]

Smith S M, Brady J M. SUSAN——a new approach to low level image processing[J]. International Journal of Computer Vision, 1997:23(1):45-78.[DOI:10.1023/a:1007963824710]

Tomasi C, Manduchi R. Bilateral filtering for gray and color images[C]//Proceedings of the 6th International Conference on Computer Vision. Bombay, India: IEEE, 1998: 839-846.[ DOI: 10.1109/iccv.1998.710815 http://dx.doi.org/10.1109/iccv.1998.710815 ]

Aharon M, Elad M, Bruckstein A. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11):4311-4322.[DOI:10.1109/tsp.2006.881199]

Zhang L, Dong W S, Zhang D, et al. Two-stage image denoising by principal component analysis with local pixel grouping[J]. Pattern Recognition, 2010, 43(4):1531-1549.[DOI:10.1016/j.patcog.2009.09.023]

Tang C, Xu J L, Zhou Z G. Strong noise image-denoising algorithm based on improved curvature filters[J]. Journal of Image and Graphics, 2019, 24(3):346-356.

汤成, 许建龙, 周志光.改进的曲率滤波强噪声图像去噪方法[J].中国图象图形学报, 2019, 24(3):346-356. [DOI:10.11834/jig.180302]

Xu S P, Zhang X Q, Jiang Y N, et al. Noise level estimation based on local means and its application to the blind BM3D denoising algorithm[J]. Journal of Image and Graphics, 2017, 22(4):422-434.

徐少平, 张兴强, 姜尹楠, 等.局部均值噪声估计的盲3维滤波降噪算法[J].中国图象图形学报, 2017, 22(4):422-434. [DOI:10.11834/jig.20170402]

Remez T, Litany O, Giryes R, et al. Class-aware fully convolutional Gaussian and Poisson denoising[J]. IEEE Transactions on Image Processing, 2018, 27(11):5707-5722.[DOI:10.1109/TIP.2018.2859044]

Buades A, Coll B, Morel J M. A non-local algorithm for image denoising[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA: IEEE, 2005: 60-65.[ DOI: 10.1109/CVPR.2005.38 http://dx.doi.org/10.1109/CVPR.2005.38 ]

Dabov K, Foi A, Katkovnik V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8):2080-2095.[DOI:10.1109/TIP.2007.901238]

Mairal J, Bach F, Ponce J, et al. Non-local sparse models for image restoration[C]//Proceedings of the 12th International Conference on Computer Vision. Kyoto, Japan: IEEE, 2009: 2272-2279.[ DOI: 10.1109/ICCV.2009.5459452 http://dx.doi.org/10.1109/ICCV.2009.5459452 ]

Dong W S, Zhang L, Shi G M, et al. Nonlocally centralized sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2013, 22(4):1620-1630.[DOI:10.1109/TIP.2012.2235847]

Zhang J, Zhao D B, Gao W. Group-based sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2014, 23(8):3336-3351.[DOI:10.1109/TIP.2014.2323127]

Dong W S, Shi G M, Li X. Nonlocal image restoration with bilateral variance estimation:a low-rank approach[J]. IEEE Transactions on Image Processing, 2013, 22(2):700-711.[DOI:10.1109/TIP.2012.2221729]

Gu S H, Zhang L, Zuo W M, et al. Weighted nuclear norm minimization with application to image denoising[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 2862-2869.[ DOI: 10.1109/CVPR.2014.366 http://dx.doi.org/10.1109/CVPR.2014.366 ]

Xie J Y, Xu L L, Chen E H. Image denoising and inpainting with deep neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: ACM, 2012: 341-349. http://cn.bing.com/academic/profile?id=8008a6f168a5fee75d5682204f849790&encoded=0&v=paper_preview&mkt=zh-cn .

Burger H C, Schuler C J, Harmeling S. Image denoising: can plain neural networks compete with BM3D?[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE, 2012: 2392-2399.[ DOI: 10.1109/CVPR.2012.6247952 http://dx.doi.org/10.1109/CVPR.2012.6247952 ]

Santhanam V, Morariu V I, Davis L S. Generalized deep image to image regression[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 5395-5405.[ DOI: 10.1109/CVPR.2017.573 http://dx.doi.org/10.1109/CVPR.2017.573 ]

Zhang K, Zuo W M, Gu S H, et al. Learning deep CNN denoiser prior for image restoration[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 2808-2817.[ DOI: 10.1109/CVPR.2017.300 http://dx.doi.org/10.1109/CVPR.2017.300 ]

Zhang K, Zuo W M, Chen Y J, et al. Beyond a Gaussian denoiser:residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155.[DOI:10.1109/TIP.2017.2662206]

Du B, Xiong W, Wu J, et al. Stacked convolutional denoising auto-encoders for feature representation[J]. IEEE Transactions on Cybernetics, 2017, 47(4):1017-1027.[DOI:10.1109/TCYB.2016.2536638]

Wu C Z, Chen X, Ji D, et al. Image denoising via residual network based on perceptual loss[J]. Journal of Image and Graphics, 2018, 23(10):1483-1491.

吴从中, 陈曦, 季栋, 等.结合深度残差学习和感知损失的图像去噪[J].中国图象图形学报, 2018, 23(10):1483-1491. [DOI:10.11834/jig.180069]

Zhang J S, Zhang Y, Bai L F, et al. Lossless-constraint denoising based auto-encoders[J]. Signal Processing:Image Communication, 2018, 63:92-99.[DOI:10.1016/j.image.2018.02.002]

Zhang K, Zuo W M, Zhang L. FFDNet:toward a fast and flexible solution for CNN-based image denoising[J]. IEEE Transactions on Image Processing, 2018, 27(9):4608-4622.[DOI:10.1109/TIP.2018.2839891]

Plötz T, Roth S. Neural nearest neighbors networks[EB/OL]. 2018-10-30[2019-04-04] . http://arxiv.org/pdf/1810.12575.pdf http://arxiv.org/pdf/1810.12575.pdf .

Liu H F, Xiong R Q, Ma S W, et al. Non-local extension of total variation regularization for image restoration[C]//Proceedings of 2014 IEEE International Symposium on Circuits and Systems. Melbourne VIC, Australia: IEEE, 2014: 1102-1105.[ DOI: 10.1109/ISCAS.2014.6865332 http://dx.doi.org/10.1109/ISCAS.2014.6865332 ]

Xu J, Zhang L, Zuo W M, et al. Patch group based nonlocal self-similarity prior learning for image denoising[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 244-252.[ DOI: 10.1109/ICCV.2015.36 http://dx.doi.org/10.1109/ICCV.2015.36 ]

Xu S P, Yang X H, Jiang S L. A fast nonlocally centralized sparse representation algorithm for image denoising[J]. Signal Processing, 2017, 131:99-112.[DOI:10.1016/j.sigpro.2016.08.006]

Jin K H, Ye J C. Sparse and low-rank decomposition of a Hankel structured matrix for impulse noise removal[J]. IEEE Transactions on Image Processing, 2018, 27(3):1448-1461.[DOI:10.1109/TIP.2017.2771471]

Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[EB/OL]. 2016-04-30[2019-04-04] . http://arxiv.org/pdf/1511.07122.pdf http://arxiv.org/pdf/1511.07122.pdf .

Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.[DOI:10.1109/TPAMI.2016.2572683]

Ahn B, Cho N I. Block-matching convolutional neural network for image denoising[EB/OL]. 2017-04-03[2019-04-04] . http://arxiv.org/pdf/1704.00524.pdf http://arxiv.org/pdf/1704.00524.pdf .

Cruz C, Foi A, Katkovnik V, et al. Nonlocality-reinforced convolutional neural networks for image denoising[J]. IEEE Signal Processing Letters, 2018, 25(8):1216-1220.[DOI:10.1109/LSP.2018.2850222]

Yang D, Sun J. BM3D-Net:a convolutional neural network for transform-domain collaborative filtering[J]. IEEE Signal Processing Letters, 2018, 25(1):55-59.[DOI:10.1109/lsp.2017.2768660]

Chen J W, Chen J W, Chao H Y, et al. Image blind denoising with generative adversarial network based noise modeling[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 3155-3164.[ DOI: 10.1109/CVPR.2018.00333 http://dx.doi.org/10.1109/CVPR.2018.00333 ]

Ulyanov D, Vedaldi A, Lempitsky V. Deep image prior[EB/OL]. 2018-04-05[2019-04-04] . http://arxiv.org/pdf/1711.10925.pdf http://arxiv.org/pdf/1711.10925.pdf .