Image defogging algorithm using a two-phase feature extraction strategy

Feiniu Yuan; Zhiqiang Li; Jinting Shi; Xue Xia; Ya Li

doi:10.11834/jig.200057

Image Analysis and Recognition | Views : 0 下载量: 276 CSCD: 4

PDF
Export
Share
Collection
Album

Image defogging algorithm using a two-phase feature extraction strategy
Vol. 26, Issue 3, Pages: 568-580(2021)
Received：01 March 2020，

Revised：2020-5-13，

Accepted：20 May 2020，

Published：16 March 2021
DOI： 10.11834/jig.200057
稿件说明：

移动端阅览

DOI：

Feiniu Yuan, Zhiqiang Li, Jinting Shi, Xue Xia, Ya Li. Image defogging algorithm using a two-phase feature extraction strategy[J]. Journal of Image and Graphics, 2021, 26(3): 568-580. DOI： 10.11834/jig.200057.

摘要

目的

传统以先验知识为基础的去雾算法，如最大化饱和度、暗通道等，在某些特定场景下效果非常不稳定，会出现色彩扭曲和光晕等现象。由于标注好的训练数据严重不足、特征的冗余性等原因，传统基于学习的去雾算法容易导致模型过拟合。为克服这些问题，本文提出一种基于两阶段特征提取的场景透射率回归去雾方法。

方法

在第1阶段，提取图像在颜色空间上的饱和度、最小通道、最大通道以及灰度图的盖博响应等43维特征作为初始雾的特征，并在提取的特征图像局部窗口内，进一步提取最小值、最大值、均值、方差、偏度、峰度、高斯均值等7维特征。在第2阶段，将提取的43×7=301个维度特征组成表征雾的二阶段特征向量。最后采用支持向量机进行训练，得到雾的特征向量和场景透射率的回归模型。

结果

实验结果表明，本文算法取得了非常好的去雾效果。平均梯度值为4.475，高于所有对比算法；峰值信噪比为18.150 dB，仅次于多尺度卷积神经网络去雾算法；结构相似性为0.867，处于较高水平；去雾后的亮度和对比度，也均排于前列。本文算法的去雾测试性能接近甚至超过了已有的基于深度学习的去雾算法，表明本文提出的两阶段特征能够很好地对雾进行表征，实现了小样本学习的高效去雾。

结论

本文通过两阶段的特征提取策略，极大提升了算法的鲁棒性，仅需要极少量样本就能训练得到性能很好的去雾模型，具有很好的泛化性能。

Abstract

Objective

Haze is a very common phenomenon in nature scenes

which mainly leads to poor image quality. Image dehazing has important research significance in practical applications

such as machine vision and intelligent transportation primarily because fog reduces image contrast

resulting in instability of feature extraction methods and recognition systems. Traditional defogging algorithms based on prior knowledge

such as maximizing saturation and dark channels

have unstable effects on certain specific scenes

such as color distortion and halo. Traditional learning-based defogging methods are prone to overfitting due to lack of insufficient labeled training data and feature redundancy.

Method

We propose a two-phase strategy for feature extraction to improve effectiveness and representation capabilities of existing features for fog and haze by analyzing the advantages and drawbacks of existing image defogging methods. In the first phase

inspired by the successes of dark channel defogging method

similarity of Gabor filters

and human vision responses

the paper extracts color saturation

minimum color channel

maximum color channel

and Gabor responses of gray-scale images as initial fog features. Gabor filters are set to have eight orientations and five scales for a total of 40 responses of Gabor filters. Hence

the initial fog features in the first phase are 43 dimensional for each pixel. This paper extracts the minimum

maximum

mean

variance

skewness

kurtosis

and Gaussian average of each local region on every feature map of the first phase to improve the robustness of these initial features. These features are seven dimensional for each pixel of a feature map extracted in the first phase. Thus

The paper extracts 43×7=301 features in the second phase for each pixel. These features are formed as a 301-dimensional feature vector for each pixel

which has very powerful representation capabilities. Finally

we adopt support vector machine with 301-dimensional feature vectors to train a regression model between a 301-dimensional feature vector and a transmission rate.

Result

The paper performs experiments on several public data sets to compare our method and several image defogging methods

including deep-learning-based algorithms. Experimental results show that the algorithm in this paper achieves a very good defogging effect. For the evaluation index of average gradient

the value of this algorithm is 4.475

which is higher than all the comparison algorithms. For the peak signal-to-noise ratio

the value of the algorithm in this paper is second

with a value of 18.150 dB

second only to the multiscale convolutional neural network defogging algorithm. For structural similarity

the algorithm in this paper is 0.867

which is high. For brightness and contrast after defogging

the proposed algorithm is also in the forefront. Our method's defogging results are similar to those obtained by existing deep-learning-based defogging methods. In several cases

our results are even better than those of deep learning methods. The experimental results validate that our two-phase features can represent fog and haze. We also implement an effective image defogging method from a small labeled training dataset.

Conclusion

This paper can greatly enhance the robustness and representation capabilities of initial fog features

and we can use only a small training data set to train a suitable model that has very good generalization performance by using a two-phase feature extraction strategy.

关键词

Keywords

references

Ancuti C O, Ancuti C, Hermans C and Bekaert P. 2010. A fast semi-inverse approach to detect and remove the haze from a single image//Proceedings of the 10th Asian Conference on Computer Vision. Queenstown, New Zealand: Springer: 501-514[ DOI: 10.1007/978-3-642-19309-5_39 http://dx.doi.org/10.1007/978-3-642-19309-5_39 ]

Berman D, Treibitz T and Avidan S. 2016. Non-local image dehazing//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 1674-1682[ DOI: 10.1109/CVPR.2016.185 http://dx.doi.org/10.1109/CVPR.2016.185 ]

Bui T M and Kim W. 2018. Single image dehazing using color ellipsoid prior. IEEE Transactions on Image Processing, 27(2): 999-1009[DOI:10.1109/TIP.2017.2771158]

Cai B L, Xu X M, Jia K, Qing C and Tao D. 2016. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11): 5187-5198[DOI:10.1109/TIP.2016.2598681]

Choi L K, You J and Bovik A C. 2015. Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Transactions on Image Processing, 24(11): 3888-3901[DOI:10.1109/tip.2015.2456502]

Fattal R. 2008. Single image dehazing. ACM Transactions on Graphics, 27(3): 1-9[DOI:10.1145/1399504.1360671]

Fattal R. 2014. Dehazing using color-lines. ACM Transactions on Graphics, 34(1): 1-14[DOI:10.1145/2651362]

Gibson K B and Nguyen T Q. 2013. Fast single image fog removal using the adaptive wiener filter//Proceedings of 2013 IEEE International Conference on Image Processing. Melbourne, Australia: IEEE: 714-718[ DOI: 10.1109/ICIP.2013.6738147 http://dx.doi.org/10.1109/ICIP.2013.6738147 ]

Gibson K B, Vo D T and Nguyen T Q. 2012. An investigation of dehazing effects on image and video coding. IEEE Transactions on Image Processing, 21(2): 662-673[DOI:10.1109/TIP.2011.2166968]

Guo J M, Syue J Y, Radzicki V R and Lee H. 2017. An efficient fusion-based defogging. IEEE Transactions on Image Processing, 26(9): 4217-4228[DOI:10.1109/TIP.2017.2706526]

He K M, Sun J and Tang X O. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2341-2353[DOI:10.1109/TPAMI.2010.168]

He K M, Sun J and Tang X O. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6): 1397-1409[DOI:10.1109/TPAMI.2012.213]

Li R D, Pan J S, Li Z C and Tang J H. 2018. Single image dehazing via conditional generative adversarial network//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 8202-8211[ DOI: 10.1109/CVPR.2018.00856 http://dx.doi.org/10.1109/CVPR.2018.00856 ]

Li Z, Li J Z, Hu Y J and Zhang Y. 2019. Mixed prior and weighted guided filter image dehazing algorithm. Journal of Image and Graphics, 24(2): 170-179

李喆, 李建增, 胡永江, 张岩. 2019. 混合先验与加权引导滤波的图像去雾算法. 中国图象图形学报, 24(2): 170-179[DOI:10.11834/jig.180450]

Li Z G and Zheng J H. 2018. Single image de-hazing using globally guided image filtering. IEEE Transactions on Image Processing, 27(1): 442-450[DOI:10.1109/TIP.2017.2750418]

Liu W J, Bai W S, Qu H C and Zhao Q G. 2019. Image dehazing based on GF-MSRCR and dark channel prior. Journal of Image and Graphics, 24(11): 1893-1905

刘万军, 白宛司, 曲海成, 赵庆国. 2019. 融合GF-MSRCR和暗通道先验的图像去雾. 中国图象图形学报, 24(11): 1893-1905[DOI:10.11834/jig.190089]

McCartney E J and Hall Jr F F. 1977. Optics of the atmosphere: scattering by molecules and particles. Physics Today, 30(5): 76-77[DOI:10.1063/1.3037551]

Meng G F, Wang Y, Duan J Y, Xiang S M and Pan C H. 2013. Efficient image dehazing with boundary constraint and contextual regularization//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, Australia: IEEE: 617-624[ DOI: 10.1109/ICCV.2013.82 http://dx.doi.org/10.1109/ICCV.2013.82 ]

Ren W Q, Liu S, Zhang H, Pan J S, Cao X C and Yang M H. 2016. Single image dehazing via multi-scale convolutional neural networks//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 154-169[ DOI: 10.1007/978-3-319-46475-6_10 http://dx.doi.org/10.1007/978-3-319-46475-6_10 ]

Santra S, Mondal R and Chanda B. 2018. Learning a patch quality comparator for single image dehazing. IEEE Transactions on Image Processing, 27(9): 4598-4607[DOI:10.1109/TIP.2018.2841198]

Silberman N and Fergus R. 2011. Indoor scene segmentation using a structured light sensor//Proceedings of 2011 IEEE International Conference on Computer Vision Workshops. Barcelona, Spain: IEEE: 601-608[ DOI: 10.1109/ICCVW.2011.6130298 http://dx.doi.org/10.1109/ICCVW.2011.6130298 ]

Tan R T. 2008. Visibility in bad weather from a single image//Proceedings of 2008 IEEE Conference on ComputerVision and Pattern Recognition. Anchorage, USA: IEEE: 1-8[ DOI: 10.1109/CVPR.2008.4587643 http://dx.doi.org/10.1109/CVPR.2008.4587643 ]

Tang K T, Yang J C and Wang J. 2014. Investigating haze-relevant features in a learning framework for image dehazing//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE: 2995-3002[ DOI: 10.1109/CVPR.2014.383 http://dx.doi.org/10.1109/CVPR.2014.383 ]

Tarel J P and Hautière N. 2009. Fast visibility restoration from a single color or gray level image//Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto, Japan: IEEE: 2201-2208[ DOI: 10.1109/ICCV.2009.5459251 http://dx.doi.org/10.1109/ICCV.2009.5459251 ]

Tarel J P, Hautière N, Cord A, Gruyer D and Halmaoui H. 2010. Improved visibility of road scene images under heterogeneous fog//Proceedings of 2010 IEEE Intelligent Vehicles Symposium. San Diego, USA: IEEE: 478-485[ DOI: 10.1109/IVS.2010.5548128 http://dx.doi.org/10.1109/IVS.2010.5548128 ]

Tripathi A K and Mukhopadhyay S. 2012. Single image fog removal using bilateral filter//Proceedings of 2012 IEEE International Conference on Signal Processing, Computing and Control. Waknaghat Solan, India: IEEE: 1-6[ DOI: 10.1109/ISPCC.2012.6224342 http://dx.doi.org/10.1109/ISPCC.2012.6224342 ]

Wang J B, Lu K, Xue J, He N and Ling S. 2018. Single image dehazing based on the physical model and MSRCR algorithm. IEEE Transactions on Circuits and Systems for Video Technology, 28(9): 2190-2199[DOI:10.1109/TCSVT.2017.2728822]

Wang P, Zhang Y F, Bao F X, Du H W and Zhang C M. 2018. Multivariate optimized haze removal method based on hazy image degradation model. Journal of Image and Graphics, 23(4): 605-616

王平, 张云峰, 包芳勋, 杜宏伟, 张彩明. 2018. 基于雾天图像降质模型的优化去雾方法. 中国图象图形学报, 23(4): 605-616[DOI:10.11834/jig.170358]

Xiao J S, Shen M Y, Lei J F, Xiong W X and Jiao C K. 2020. Image conversion algorithm for haze scene based on generative adversarial networks. Chinese Journal of Computers, 43(1): 165-176

肖进胜, 申梦瑶, 雷俊锋, 熊闻心, 焦陈坤. 2020. 基于生成对抗网络的雾霾场景图像转换算法. 计算机学报, 43(1): 165-176[DOI:10.11897/SP.J.1016.2020.00165]

Zeng J X and Yu Y L. 2017. Image defogging and edge preserving algorithm based on dark channel prior and bilateral filtering. Journal of Image and Graphics, 22(2): 147-153

曾接贤, 余永龙. 2017. 双边滤波与暗通道结合的图像保边去雾算法. 中国图象图形学报, 22(2): 147-153[DOI:10.11834/jig.20170201]

Zhu Q S, Mai J M and Shao L. 2015. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 24(11): 3522-3533[DOI:10.1109/TIP.2015.2446191

Alert me when the article has been cited

提交

Image haze removal using a bilateral filter

Fuse image enhancement with a regularized correlation filter for target tracking of UAVs

3D Gaussian splatting model for low-light enhancement

Visual saliency detection for 3D object models

Fusion of image enhancement and self-attention mechanism for the generation of active pulmonary tuberculosis CT image data