浏览全部资源
扫码关注微信
Published： 16 May 2022 ，
Accepted： 04 July 2021
移动端阅览
Shuying Huang, Yating Xu, Yong Yang, Juwei Guan. Image deraining algorithm based on morphological filtering and convolution sparse coding. [J]. Journal of Image and Graphics 27(5):1522-1536(2022)
目的
2
图像去雨技术是对雨天拍摄图像中雨纹信息进行检测和去除，恢复目标场景的细节信息，从而获得清晰的无雨图像。针对现有方法对雨纹信息检测不完全、去除不彻底的问题，提出一种联合自适应形态学滤波和多尺度卷积稀疏编码(multi-scale convolution sparse coding
MS-CSC)的单幅图像去雨方法。
方法
2
考虑雨纹信息的形状结构特点，构造一种自适应形态学滤波器来滤除有雨图像中的雨纹信息，获得包含图像自身纹理的低频成分；利用全变分模型正则化方法来增强低频成分的纹理信息，并利用有雨图像减去低频成分获得包含雨纹信息的高频成分；针对高频成分，根据雨纹的方向性提出一种基于方向梯度正则化的MS-CSC方法来重构高频成分，并通过迭代求解获得包含精确雨纹的高频成分，即雨层；利用有雨图像减去雨层得到最终的去雨图像。
结果
2
为验证本文方法的有效性，与一些主流的去雨方法进行实验比较。实验结果表明，本文方法在模拟数据集上的平均峰值信噪比(peak signal-to-noise ratio
PSNR)和平均结构相似度(structural similarity
SSIM)指标分别提高了0.95和0.005 2，能得到较好的主观视觉效果和客观评价，特别是在真实数据集上具有明显优势。
结论
2
本文方法可有效去除雨纹，同时可保留更多的原始图像纹理信息，经证明是一种有效的去雨算法。
Objective
2
Images captured in rainy days are blurred due to the influence of raindrops or rain lines
which directly affects the accuracy of outdoor vision tasks
such as image segmentation
object recognition
and object tracking. Image deraining technology aims to recover the lost details in the target scene by detecting and removing the rain streak information in the rain image. According to the imaging principle of the image
the rain streak information in the image can be regarded as a kind of additive noise. Many traditional methods can use some prior information such as the direction
morphological structure and color of rain lines to remove rain noise and obtain a background layer without rain. However
due to the incomplete detection of rain streak information
many methods cannot completely remove rain streaks
resulting in residual rain streaks in the rain removal results.
Method
2
Based on the existing problems
we propose a rain removal method based on adaptive morphological filtering and multi-scale convolutional sparse coding. First
according to the diversity of the width and length of the rain streak structure
a set of multi-scale structuring element operators is constructed by using the dilation operation in the morphology to filter out the rain streak information in the image. The size of the filter operator determines the degree of filtering. If the size of the filter operator is too large
more information will be filtered out
resulting in the loss of image texture. On the contrary
if the size of the filter operator is too small
the rain streak information cannot be completely filter out
resulting in the residual rain streak information in the image. Therefore
in order to control the degree of morphological filtering and avoid filtering out too much texture information of the image itself
an adaptive morphological filtering method is proposed. The method controls the size of the filter operator by comparing the wavelet filtering results with the morphological filtering results
so as to obtain low-frequency components with less rain streak information and rich texture information. Next
this method aims to preserve the texture information of the image itself in the low-frequency component as much as possible. Therefore
according considering the inevitable loss of image details in image filtering
the steerable total variation (STV) regularization method is used to detect the texture information of the image itself filtered out in the filtering
and supplement it to the low-frequency components. to obtain good deraining results
detecting accurate rain layer information is the key to deraining methods. Based on the above steps
a high frequency component (i.e.
rain layer) containing rain streak information the rain layer can be obtained by subtracting the detected low frequency component from the rain image. Then
to further refine the rain layer
we propose a high-frequency separation method based on multi-scale convolution sparse coding (MS-CSC) to reduce the non-rain streak information in the high-frequency components and obtain a more accurate rain layer. In this process
in order to better detect the rain pattern information
by analyzing the falling direction of rain
two gradient constraints for the horizontal direction
x
and the vertical direction
y
are constructed to constrain the solution of the rain layer. The rain grain gradient information mainly exists in the
x
direction
and the gradient information in the
y
direction is sparse. Therefore
because the gradient information of the rain streaks in the
x
direction is relatively rich
this paper constructs the gradient sparse term in the
x
direction by using the difference between the high frequency component and the reconstructed rain layer to ensure the rain streak reconstruction in the main direction
y
. Since the rain streak gradient information in the
y
direction is sparse
the gradient sparse term of the rain layer in the
y
direction is directly constructed to constrain the rain pattern information in the
x
direction. By combining the CSC method and the constructed two gradient constraints
a directional gradient-regularized MS-CSC model is constructed to solve for an accurate rain layer. In this model
a dictionary needs to be constructed to implement the encoding of the rain layer. According to the diversity of rain streak structures in image scenes
this paper constructs a multi-scale dictionary to encode rain streak information of different sizes by using dictionaries of different sizes. Finally
the image de-rained result (i.e.
the background layer) is obtained by subtracting the rain layer from the rain image.
Result
2
To verify the effectiveness of the proposed method
we compared it with some state-of-the-art deraining methods
including some traditional and deep learning-based methods. Our demonstrated results illustrate that the proposed average peak signal-to-noise ratio(PSNR) and structural similarity(SSIM) index is improved by 0.95 and 0.005 2 based on the simulated dataset. Moreover
our simulation method has its priority of qualitative visual effects and quantitative evaluation indexes.
Conclusion
2
The integrated MS-CSC and adaptive morphological filtering method can remove rain streaks effectively. The recovery image has its clearer edge texture and details after rain removal. Additionally
our deraining algorithm method has strong generalization capability.
图像去雨自适应形态学滤波全变分模型方向梯度正则化多尺度卷积稀疏编码(MS-CSC)
image derainingadaptive morphological filtertotal variation modeldirectional gradient regularizationmulti-scale convolution sparse coding(MS-CSC)
Bristow H, Eriksson A and Lucey S. 2013. Fast convolutional sparse coding//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE: 391-398 [DOI: 10.1109/CVPR.2013.57http://dx.doi.org/10.1109/CVPR.2013.57]
Chen D Y, Chen C C and Kang L W. 2014. Visual depth guided color image rain streaks removal using sparse coding. IEEE Transactions on Circuits and Systems for Video Technology, 24(8): 1430-1455 [DOI: 10.1109/TCSVT.2014.2308627]
Ding X H, Chen L Q, Zheng X H, Huang Y and Zeng D L. 2016. Single image rain and snow removal via guided L0 smoothing filter. Multimedia Tools and Applications, 75(5): 2697-2712 [DOI: 10.1007/s11042-015-2657-7]
Freeman W T and Adelson E H. 1991. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9): 891-906 [DOI: 10.1109/34.93808]
Fu X Y, Huang J B, Zeng D L, Huang Y, Ding X H and Paisley J. 2017. Removing rain from single images via a deep detail network//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE: 1715-1723 [DOI: 10.1109/CVPR.2017.186http://dx.doi.org/10.1109/CVPR.2017.186]
Fu X Y, Liang B R, Huang Y, Ding X H and Paisley J. 2020. Lightweight pyramid networks for image deraining. IEEE Transactions on Neural Networks and Learning Systems, 31(6): 1794-1807 [DOI: 10.1109/TNNLS.2019.2926481]
Garg K and Nayar S K. 2004. Detection and removal of rain from videos//Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. Washington, USA: IEEE: 528-535 [DOI: 10.1109/CVPR.2004.1315077http://dx.doi.org/10.1109/CVPR.2004.1315077]
Garg K and Nayar S K. 2005. When does a camera see rain?//Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China: IEEE: 1067-1074 [DOI: 10.1109/ICCV.2005.253http://dx.doi.org/10.1109/ICCV.2005.253]
Gu S H, Meng D Y, Zuo W M and Zhang L. 2017. Joint convolutional analysis and synthesis sparse representation for single image layer separation//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE: 1717-1725 [DOI: 10.1109/ICCV.2017.189http://dx.doi.org/10.1109/ICCV.2017.189]
Jiang T X, Huang T Z, Zhao X L, Deng L J and Wang Y. 2019. FastDeRain: a novel video rain streak removal method using directional gradient priors. IEEE Transactions on Image Processing, 28(4): 2089-2102 [DOI: 10.1109/TIP.2018.2880512]
Jiang Z L, Lin Z and Davis L S. 2011. Learning a discriminative dictionary for sparse coding via label consistent K-SVD//Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE: 1697-1704 [DOI: 10.1109/CVPR.2011.5995354http://dx.doi.org/10.1109/CVPR.2011.5995354]
Jiao J and Wu L D. 2019. Fusion of multispectral and panchromatic images via morphological filter and improved PCNN in NSST domain. Journal of Image and Graphics, 24(3): 435-446
焦姣, 吴玲达. 2019. 形态学滤波和改进PCNN的NSST域多光谱与全色图像融合. 中国图象图形学报, 24(3): 435-446 [DOI: 10.11834/jig.180399]
Kang L W, Lin C W and Fu Y H. 2012. Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing, 21(4): 1742-1755 [DOI: 10.1109/TIP.2011.2179057]
Lei T, Fan Y Y and Mao L. 2011. Generalized self-dual morphological filters and the applications in image-denoising. Journal of Optoelectronics·Laser, 22(1): 136-143, 158
雷涛, 樊养余, 毛力. 2011. 广义自对偶形态学滤波器及其在图像去噪中的应用. 光电子·激光, 22(1): 136-143, 158 [DOI: 10.16136/j.joel.2011.01.028]
Li Y, Tan R T, Guo X J, Lu J B and Brown M S. 2016. Rain streak removal using layer priors//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 2736-2744 [DOI: 10.1109/CVPR.2016.299http://dx.doi.org/10.1109/CVPR.2016.299]
Luo Y, Xu Y and Ji H. 2015. Removing rain from a single image via discriminative sparse coding//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, China: IEEE: 3397-3405 [DOI: 10.1109/ICCV.2015.388http://dx.doi.org/10.1109/ICCV.2015.388]
Rudin L I, Osher S and Fatemi E. 1992. Nonlinear total variation based noise removal algorithm. Physica D: Nonlinear Phenomena, 60(1/4): 259-268 [DOI: 10.1016/0167-2789(92)90242-F]
Tibshirani R. 2011. Regression shrinkage and selection via the LASSO: a retrospective. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(3): 273-282 [DOI: 10.1111/j.1467-9868.2011.00771.x]
Tripathi A K and Mukhopadhyay S. 2013. Meteorological approach for detection and removal of rain from videos. IET Computer Vision, 7(1): 36-47 [DOI: 10.1049/iet-cvi.2011.0032]
Wang H, Xie Q, Wu Y C, Zhao Q and Meng D Y. 2020a. Single image rain streaks removal: a review and an exploration. International Journal of Machine Learning and Cybernetics, 11(4): 853-872 [DOI: 10.1007/s13042-020-01061-2]
Wang Y L, Liu S C, Chen C and Zeng B. 2017. A hierarchical approach for rain or snow removing in a single color image. IEEE Transactions on Image Processing, 26(8): 3936-3950 [DOI: 10.1109/TIP.2017.2708502]
Wang Y L, Liu S C, Xie D H and Zeng B. 2020b. Removing rain streaks by a linear mode. IEEE Access, 8: 54802-54815 [DOI: 10.1109/ACCESS.2020.2981643]
Wang Y Q, Yao Q M, Kwok J T and Ni L M. 2018. Scalable online convolutional sparse coding. IEEE Transactions on Image Processing, 27(10): 4850-4859 [DOI: 10.1109/TIP.2018.2842152]
Wei W, Meng D Y, Zhao Q, Xu Z B and Wu Y. 2019. Semi-supervised transfer learning for image rain removal//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE: 3872-3881 [DOI: 10.1109/CVPR.2019.00400http://dx.doi.org10.1109/CVPR.2019.00400]
Xu J, Zhao W, Liu P and Tang X L. 2012a. Removing rain and snow in a single image using guided filter//Proceedings of 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE). Zhangjiajie, China: IEEE: 304-307 [DOI: 10.1109/CSAE.2012.6272780http://dx.doi.org/10.1109/CSAE.2012.6272780]
Xu J, Zhao W, Liu P and Tang X L. 2012b. An improved guidance image based method to remove rain and snow in a single image. Computer and Information Science, 5(3): 49-55 [DOI: 10.5539/cis.v5n3p49]
Yang W H, Tan R T, Wang S Q, Fang Y M and Liu J Y. 2021. Single image deraining: from model-based to data-driven and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11): 4059-4077 [DOI: 10.1109/TPAMI.2020.2995190]
Zhang H and Patel V M. 2018. Density-aware single image de-raining using a multi-stream dense network//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 695-704 [DOI: 10.1109/CVPR.2018.00079http://dx.doi.org/10.1109/CVPR.2018.00079]
Zhang W X, Cao Y Z, Zhang R X and Wang Y Q. 2014. Image denoising using total variation model guided by steerable filter. Mathematical Problems in Engineering, 2014: #423761 [DOI: 10.1155/2014/423761]
Zheng X H, Liao Y H, Guo W, Fu X Y and Ding X H. 2013. Single-image-based rain and snow removal using multi-guided filter//Proceedings of the 20th International Conference on Neural Information Processing. Daegu, Korea(South): Springer: 258-265 [DOI: 10.1007/978-3-642-42051-1_33http://dx.doi.org/10.1007/978-3-642-42051-1_33]
Zhou L N, Wang D M, Guo Y B and Yang Y X. 2008. Exposing digital forgeries by detecting image blurred mathematical morphology edge. Acta Electronica Sinica, 36(6): 1047-1051
周琳娜, 王东明, 郭云彪, 杨义先. 2008. 基于数字图像边缘特性的形态学滤波取证技术. 电子学报, 36(6): 1047-1051 [DOI: 10.3321/j.issn:0372-2112.2008.06.002]
相关作者
相关机构
京公网安备11010802024621