浏览全部资源
扫码关注微信
纸质出版日期： 2023-07-16 ，
移动端阅览
丁岳皓， 吴昊， 孔凤玲， 徐丹， 袁国武. 2023. 面向真实图像噪声的两阶段盲去噪. 中国图象图形学报， 28(07):2026-2036
Ding Yuehao， Wu Hao， Kong Fengling， Xu Dan， Yuan Guowu. 2023. A dual of real image noise-related blind denoising technique. Journal of Image and Graphics， 28(07):2026-2036
目的
2
现有的深度图像去噪算法在去除加性高斯噪声上效果显著，但在去除任意分布的真实图像噪声时表现不佳；去噪模型的深度在不断增加，但去噪效果上却并未能显著提高。对此，设计了一种简单有效的两阶段深度图像去噪算法。
方法
2
首先基于注意力机制估计真实图像上的噪声分布水平，然后使用一个混合膨胀卷积和普通卷积的多尺度去噪模块进行非盲降噪。
结果
2
在DND（darmstadt noise dataset）、SIDD（smartphone image denoising dataset）、Nam和PolyU（the Hong Kong Polytechnic University）等4个图像去噪领域常用数据集上进行去噪实验，选择峰值信噪比（peak signal-to-noise ratio，PSNR）和结构相似性（structual similarity，SSIM）作为去噪效果的评价指标，得到的平均PSNR值分别为39.23 dB，38.54 dB，40.45 dB，37.34 dB，并与几种传统去噪方法和基于深度学习的去噪方法进行比较。实验结果表明，本文的去噪算法在去噪效果和视觉质量上有明显提升。同时，在SIDD数据集上进行消融实验以验证算法中模块的有效性。
结论
2
本文算法使用的跳跃连接、噪声水平估计以及多尺度模块均可以有效提升真实图像去噪效果。与现有方法相比，本文算法不仅能有效去除真实图像噪声，而且能通过简单的模块参数设置控制去噪网络的计算效率。
Objective
2
Image denoising technique can be as one of essential aspects in relevance to low-level vision. For image processing tasks， preprocessing is required for such contexts of high image quality， image segmentation， fine-grained detection and flaw detection. For real life scenario， image noise is inevitable because the image is collected and defected from the digital device itself， which will affect people’s visual perception of the image. Image denoising-related research is originated from removing additive Gaussian white noise. There is a great difference between additive Gaussian white noise and real noise. The causes of real noise are diverse， and its distribution is different from that of Gaussian noise. The distribution of real noise is complex and may not only manipulated for one distribution， in which poor denoising results are obtained in terms of the removal of real noise. Moreover， traditional image denoising methods are required to set parameters manually， and deep learning based denoising algorithm has a poor effect in removing real noise. To get better results of denoising， the network model of these denoising methods is complex and time-consuming， but actual denoising effect is not significantly improved. It is difficult to use an end-to-end algorithm to remove real image noise well beyond non-blind noise reduction. Therefore， we develop a dual of real image denoising model based on deep learning， which uses a simplified network structure and has ideal denoising results and visual performance on the image denoising dataset.
Method
2
A blind denoising network model of two-stage modular denoising network （TMNet） is designed and illustrated， which is divided into two stages， the first stage is concerned of noise level estimation， and the second stage is concerned of non-blind denoising of the image. For the noise level estimation， a 4-layer fully convolutional neural network is used to estimate the noise level of the input image. The pooling layer is not used， but a convolutional layer is linked with a convolution kernel size of 3 only， and a padding operation is performed after the convolutional layer to keep the obtained feature size consistent from each layer. At the same time， due to the attention mechanism can effectively extract network features and improve network performance， before the last convolutional layer， the squeeze and excitation （SE） module can be used to process the features obtained via the noise level estimation， and such weights is adaptively assigned to the features of different channels. Finally， the estimated noise level and the input image are used as the output of this stage. The non-blind denoising stage can be focused on taking the estimated noise level and the image together as the input of this stage， and real image denoising becomes the problem of image denoising， which can remove the known noise distribution level. At this stage， we design a multi-scale structure， which can melt image features into two branches for processing. One branch is as an expanded convolutional layer， and the other branch is as an ordinary convolutional layer as well. Dilation convolution can be applied to get a larger receptive field by injecting holes into features of non-additional new parameters. In this way， image characteristics can be obtained under different receptive fields. The algorithm is designed for those skip connections are used in the multi-scale structure because effective skip connections are appeared in image denoising， and the loss of image information will be reduced in the process of parameter transmission when the number of network layers increases. For the convolution operation used in the non-blind denoising stage， the size of the convolution kernel is 3， and the corresponding padding operation is also guaranteed that the size of the image feature will not change in the entire network. Due to the high resolution of the original image of the smartphone image denoising dataset（SIDD） dataset used in the training model， our experiment is focused on dividing the image into 256 patches and performs 4 000 epochs of training randomly. The loss function used is L1 loss. All experiments are done on NVIDIA GTX 2080ti GPU.
Result
2
Four sort of commonly used datasets in image denoising fields including darmstadt noise dataset （DND）， SIDD， Nam， and the Hong Kong Polytechnic University （PolyU） are used to verify the effectiveness of the algorithm， and the peak signal-to-noise ratio （PSNR） and structural similarity （SSIM） are used to evaluate the denoising results. Denoising experiments are carried out on DND， SIDD， Nam， and PolyU datasets， and the average PSNR values obtained are 39.23 dB， 38.54 dB， 40.45 dB， and 37.34 dB， respectively. Compared to the traditional algorithm， the denoising effect and visual sense are significantly improved， and the denoising effect can be similar to that of the deep learning denoising algorithm in related to fewer network layers. Furthermore， ablation experiments are designed to verify the effectiveness of the methods proposed， such as skip connection， channel attention， and noise level estimation. Experiments show that our methods proposed can effectively improve the effect of real image denoising to a certain extent.
Conclusion
2
We design a convolutional neural network （CNN） model for removing noise from real images. Our model has a simple network structure， in which most of the convolution operations with a convolution kernel size of 3 are carried out， and the network training speed is fast. Our network can get richer residual structure in image denoising as well. Additionally， we use multiple convolution to obtain more effective feature information. Experimental results show that our network is potential to removing real image noise further to some extent.
深度学习真实图像去噪注意力机制噪声水平估计多尺度模块
deep learningreal image denoisingattention mechanismnoise level estimationmultiscale module
Abdelhamed A， Lin S and Brown M S. 2018. A high-quality denoising dataset for smartphone cameras//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 1692-1700 ［DOI： 10.1109/CVPR.2018.00182http://dx.doi.org/10.1109/CVPR.2018.00182］
Aharon M， Elad M and Bruckstein A. 2006. K-SVD： an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing， 54（11）： 4311-4322 ［DOI： 10.1109/TSP.2006.881199http://dx.doi.org/10.1109/TSP.2006.881199］
Anwar S and Barnes N. 2019. Real image denoising with feature attention//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 3155-3164 ［DOI： 10.1109/ICCV.2019.00325http://dx.doi.org/10.1109/ICCV.2019.00325］
Buades A， Coll B and Morel J M. 2005. A non-local algorithm for image denoising//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego， USA： IEEE： 60-65 ［DOI： 10.1109/CVPR.2005.38http://dx.doi.org/10.1109/CVPR.2005.38］
Burger H C， Schuler C J and Harmeling S. 2012. Image denoising： can plain neural networks compete with BM3D?//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence， USA： IEEE： 2392-2399 ［DOI： 10.1109/CVPR.2012.6247952http://dx.doi.org/10.1109/CVPR.2012.6247952］
Chang S G， Yu B and Vetterli M. 2000. Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing， 9（9）： 1532-1546 ［DOI： 10.1109/83.862633http://dx.doi.org/10.1109/83.862633］
Chen Y J and Pock T. 2017. Trainable nonlinear reaction diffusion： a flexible framework for fast and effective image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（6）： 1256-1272 ［DOI： 10.1109/TPAMI.2016.2596743http://dx.doi.org/10.1109/TPAMI.2016.2596743］
Cheng S， Wang Y Z， Huang H B， Liu D H， Fan H Q and Liu S C. 2021. NBNet： noise basis learning for image denoising with subspace projection//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 4896-4906 ［DOI： 10.1109/CVPR46437.2021.00486http://dx.doi.org/10.1109/CVPR46437.2021.00486］
Dabov K， Foi A， Katkovnik V and Egiazarian K. 2008. Image restoration by sparse 3D transform-domain collaborative filtering//Proceedings of SPIE 6812， Image Processing： Algorithms and Systems VI. San Jose， USA： SPIE： #681207 ［DOI： 10.1117/12.766355http://dx.doi.org/10.1117/12.766355］
Gu S H， Zhang L， Zuo W M and Feng X C. 2014. Weighted nuclear norm minimization with application to image denoising//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus， USA： IEEE： 2862-2869 ［DOI： 10.1109/CVPR.2014.366http://dx.doi.org/10.1109/CVPR.2014.366］
Guan M J， Zhu J and Wu Y R. 2020. Research progress of image denoising methods. Computer Era， 2： 29-32， 35
管美静， 珠杰， 吴燕如. 2020. 图像去噪方法研究进展. 计算机时代， （2）： 29-32， 35 ［DOI： 10.16644/j.cnki.cn33-1094/tp.2020.02.008http://dx.doi.org/10.16644/j.cnki.cn33-1094/tp.2020.02.008］
Guo S， Yan Z F， Zhang K， Zuo W M and Zhang L. 2019. Toward convolutional blind denoising of real photographs//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 1712-1722 ［DOI： 10.1109/CVPR.2019.00181http://dx.doi.org/10.1109/CVPR.2019.00181］
He K M， Sun J and Tang X O. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence， 35（6）： 1397-1409 ［DOI： 10.1109/TPAMI.2012.213http://dx.doi.org/10.1109/TPAMI.2012.213］
Hu J， Shen L， Albanie S， Sun G and Wu E. 2019. Squeeze-and-excitation networks//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. New York， USA： IEEE： 183-192 ［DOI： 10.1109/TPAMI.2019.2913372http://dx.doi.org/10.1109/TPAMI.2019.2913372］
Jain V and Seung H S. 2008. Natural image denoising with convolutional networks//Proceedings of the 21st International Conference on Neural Information Processing Systems. Vancouver British， Canada： Curran Associates Inc.： 769-776
Kim J， Lee J K and Lee K M. 2016. Accurate image super-resolution using very deep convolutional networks//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 1646-1654 ［DOI： 10.1109/CVPR.2016.182http://dx.doi.org/10.1109/CVPR.2016.182］
Kim Y， Soh J W， Park G Y and Cho N I. 2020. Transfer learning from synthetic to real-noise denoising with adaptive instance normalization//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 3479-3489 ［DOI： 10.1109/CVPR42600.2020.00354http://dx.doi.org/10.1109/CVPR42600.2020.00354］
Lebrun M， Colom M and Morel J M. 2015. The noise clinic： a blind image denoising algorithm. Image Processing on Line， 5： 1-54 ［DOI： 10.5201/ipol.2015.125http://dx.doi.org/10.5201/ipol.2015.125］
Li C Y， Guo J C， Porikli F， Guo C， Fu H Z and Li X. 2017. DR-Net： transmission steered single image dehazing network with weakly supervised refinement ［EB/OL］. ［2021-12-02］. https://arxiv.org/pdf/1712.00621.pdfhttps://arxiv.org/pdf/1712.00621.pdf
Liu J S. 2021. Research on Real-Word Image Denoising Based on Deep Neural Network. Xi’an： Xidian University
刘金山. 2021. 基于深度神经网络的真实图像去噪的研究. 西安： 西安电子科技大学 ［DOI： 10.27389/d.cnki.gxadu.2021.002156http://dx.doi.org/10.27389/d.cnki.gxadu.2021.002156］
Liu L P， Qiao L L and Jiang L C. 2021. Overview of image denoising methods. Journal of Frontiers of Computer Science and Technology， 15（8）： 1418-1431
刘利平， 乔乐乐， 蒋柳成. 2021. 图像去噪方法概述. 计算机科学与探索， 15（8）： 1418-1431 ［DOI： 10.3778/j.issn.1673-9418.2101035http://dx.doi.org/10.3778/j.issn.1673-9418.2101035］
Nam S， Hwang Y， Matsushita Y and Kim S J. 2016. A holistic approach to cross-channel image noise modeling and its application to image denoising//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 1683-1691 ［DOI： 10.1109/CVPR.2016.186http://dx.doi.org/10.1109/CVPR.2016.186］
Plötz T and Roth S. 2017. Benchmarking denoising algorithms with real photographs//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 2750-2759 ［DOI： 10.1109/CVPR.2017.294http://dx.doi.org/10.1109/CVPR.2017.294］
Roth S and Black M J. 2005. Fields of experts： a framework for learning image priors//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego， USA： IEEE： 860-867 ［DOI： 10.1109/CVPR.2005.160http://dx.doi.org/10.1109/CVPR.2005.160］
Tai Y， Yang J， Liu X M and Xu C Y. 2017. MemNet： a persistent memory network for image restoration//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 4549-4557 ［DOI： 10.1109/ICCV.2017.486http://dx.doi.org/10.1109/ICCV.2017.486］
Tian C W， Xu Y and Zuo W M. 2020. Image denoising using deep CNN with batch renormalization. Neural Networks， 121： 461-473 ［DOI： 10.1016/j.neunet.2019.08.022http://dx.doi.org/10.1016/j.neunet.2019.08.022］
Tomasi C and Manduchi R. 1998. Bilateral filtering for gray and color images//Proceedings of the 6th International Conference on Computer Vision. Bombay， India： IEEE： 839-846 ［DOI： 10.1109/ICCV.1998.710815http://dx.doi.org/10.1109/ICCV.1998.710815］
Wang H C. 2021. Research on Real Image Denoising Algorithm Based on Deep Learning. Chengdu： University of Electronic Science and Technology of China
王瀚琛. 2021. 基于深度学习的真实图像去噪算法研究. 成都： 电子科技大学 ［DOI： 10.27005/d.cnki.gdzku.2021.004590http://dx.doi.org/10.27005/d.cnki.gdzku.2021.004590］
Wang Z， Bovik A C， Sheikh H R and Simoncelli E P. 2004. Image quality assessment： from error visibility to structural similarity. IEEE Transactions on Image Processing， 13（4）： 600-612 ［DOI： 10.1109/TIP.2003.819861http://dx.doi.org/10.1109/TIP.2003.819861］
Wu C Z， Chen X， Ji D and Zhan S. 2018. Image denoising via residual network based on perceptual loss. Journal of Image and Graphics， 23（10）： 1483-1491
吴从中， 陈曦， 季栋， 詹曙. 2018. 结合深度残差学习和感知损失的图像去噪. 中国图象图形学报， 23（10）： 1483-1491 ［DOI： 10.11834/jig.180069http://dx.doi.org/10.11834/jig.180069］
Xie S M， Deng L， Xie Z M and Li H P. 2020. Denoising algorithm based on convolutional neural network in real scene. Computer and Telecommunication， （11）： 39-43
谢舒闽， 邓磊， 谢中梅， 李海平. 2020. 真实场景中基于卷积神经网络的去噪算法. 电脑与电信， （11）： 39-43 ［DOI： 10.15966/j.cnki.dnydx.2020.11.012http://dx.doi.org/10.15966/j.cnki.dnydx.2020.11.012］
Xu J， Li H， Liang Z T， Zhang D and Zhang L. 2018a. Real-world noisy image denoising： a new benchmark ［EB/OL］. ［2021-11-04］. https://arxiv.org/pdf/1804.02603.pdfhttps://arxiv.org/pdf/1804.02603.pdf
Xu J， Zhang L and Zhang D. 2018b. A trilateral weighted sparse coding scheme for real-world image denoising//Proceedings of the 15th European Conference on Computer Vision. Munich， Germany： Springer ［DOI： 10.1007/978-3-030-01237-3_2http://dx.doi.org/10.1007/978-3-030-01237-3_2］
Xu S P， Liu T Y， Lin Z Y， Zhang G Z and Li C X. 2019. Main bottlenecks and research prospects of the deep convolutional neural network-based denoising model. Journal of Image and Graphics， 24（8）： 1207-1214
徐少平， 刘婷云， 林珍玉， 张贵珍， 李崇禧. 2019. 深度卷积神经网络降噪模型的技术瓶颈与研究展望. 中国图象图形学报， 24（8）： 1207-1214 ［DOI： 10.11834/jig.190165http://dx.doi.org/10.11834/jig.190165］
Yu H and Zhen T. 2020. Research on classical image denoising algorithms. China Computer and Communication， 32（14）： 66-67
于虹， 甄彤. 2020. 图像去噪经典算法研究. 信息与电脑（理论版）， 32（14）： 66-67 ［DOI： 10.3969/j.issn.1003-9767.2020.14.021http://dx.doi.org/10.3969/j.issn.1003-9767.2020.14.021］
Yu K， Wang X T， Dong C， Tang X O and Loy C C. 2022. Path-restore： learning network path selection for image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（10）： 7078-7092 ［DOI： 10.1109/TPAMI.2021.3096255http://dx.doi.org/10.1109/TPAMI.2021.3096255］
Yu S， Park B and Jeong J. 2019. Deep iterative down-up CNN for image denoising//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach， USA： IEEE： 2095-2103 ［DOI： 10.1109/CVPRW.2019.00262http://dx.doi.org/10.1109/CVPRW.2019.00262］
Zamir S W， Arora A， Khan S， Hayat M， Khan S， Yang M H and Shao L. 2020a. CycleISP： real image restoration via improved data synthesis//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE ［DOI： 10.1109/CVPR42600.2020.00277http://dx.doi.org/10.1109/CVPR42600.2020.00277］
Zamir S W， Arora A， Khan S， Hayat M， Khan F S， Yang M H and Shao L. 2020b. Learning enriched features for real image restoration and enhancement//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 492-511 ［DOI： 10.1007/978-3-030-58595-2_30http://dx.doi.org/10.1007/978-3-030-58595-2_30］
Zhang K， Zuo W M， Chen Y J， Meng D Y and Zhang L. 2017. Beyond a gaussian denoiser： residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing， 26（7）： 3142-3155 ［DOI： 10.1109/TIP.2017.2662206http://dx.doi.org/10.1109/TIP.2017.2662206］
Zhang K， Zuo W M and Zhang L. 2018. FFDNet： toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing， 27（9）： 4608-4622 ［DOI： 10.1109/TIP.2018.2839891http://dx.doi.org/10.1109/TIP.2018.2839891］
Zhao Y Y， Jiang Z Q， Men A D and Ju G D. 2019. Pyramid real image denoising network//Proceedings of 2019 IEEE Visual Communications and Image Processing. Sydney， Australia： IEEE： 1-4 ［DOI： 10.1109/VCIP47243.2019.8965754http://dx.doi.org/10.1109/VCIP47243.2019.8965754］
Zoran D and Weiss Y. 2011. From learning models of natural image patches to whole image restoration//Proceedings of 2011 International Conference on Computer Vision. Barcelona， Spain： IEEE： 479-486 ［DOI： 10.1109/ICCV.2011.6126278http://dx.doi.org/10.1109/ICCV.2011.6126278］
相关作者
相关机构
京公网安备11010802024621