浏览全部资源
扫码关注微信
纸质出版日期： 2022-05-16 ，
录用日期： 2021-06-02
移动端阅览
石争浩, 刘春月, 任文琦, 都双丽, 赵明华. 沙尘图像色彩恢复及增强卷积神经网络[J]. 中国图象图形学报, 2022,27(5):1493-1508.
Zhenghao Shi, Chunyue Liu, Wenqi Ren, Shuangli Du, Minghua Zhao. Convolutional neural networks for sand dust image color restoration and visibility enhancement[J]. Journal of Image and Graphics, 2022,27(5):1493-1508.
目的
2
在沙尘天气条件下，由于大气中悬浮微粒对入射光线的吸收和散射，户外计算机视觉系统所采集图像通常存在颜色偏黄失真和低对比度等问题，严重影响户外计算机视觉系统的性能。为此，提出一种带色彩恢复的沙尘图像卷积神经网络增强方法，由一个色彩恢复子网和一个去尘增强子网组成。
方法
2
采用提出的色彩恢复子网(sand dust color correction
SDCC)校正沙尘图像的偏色，将颜色校正后的图像作为条件，输入到由自适应实例归一化残差块组成的去尘增强子网中，对沙尘图像进行增强处理。本文还提出一种基于物理光学模型的沙尘图像合成方法，并采用该方法构建了大规模的配对沙尘图像数据集。
结果
2
对大量沙尘图像的实验结果表明，所提出的沙尘图像增强方法能很好地去除图像中的偏色和沙尘，获得正常的视觉颜色和细节清晰的图像。进一步的对比实验表明，该方法能取得优于对比方法的增强图像。
结论
2
本文所提出的沙尘图像增强方法能很好地消除整体的黄色色调和尘霾现象，获得正常的视觉色彩和细节清晰的图像。
Objective
2
The quality of captured images tends to yellowish color distortion
reduced contrast
and detailed information loss in the sand dust atmosphere due to the suspended particles derived incident light absorption and scattering. The issues of outdoor computer vision systems like video surveillance
video navigation and intelligent transportation are severely constrained. Traditional sand dust image enhancement methods are originated from visual perception based sand dust image enhancement and physical model based sand dust image restoration. The visual perception based method is not restricted of the physical imaging model. The visual quality is based on color correction and contrast enhancement. The recovered image still has insufficient color distortion
image brightness and image contrast. Physical models based sand dust image restoration is related to additional assumptions and less prior robustness
complex parameters calculation. Nowadays
existing deep learning-based sand dust image enhancement methods are migrated from the deep learning based haze images methods. Although these methods has achieved good results for haze image processing
the color of the output image still has different degrees of distortion
and the sharpness of the image is also relatively poor in terms of transferred and enhanced sand dust images. An enhanced convolutional neural network (CNN) method of restored color sand dust images can be used to resolve and improve the issues mentioned above.
Method
2
Our proposed network structure consists of a sand dust color correction subnet and a dust removal enhancement subnet. We illustrated a novel sand dust color correction network structure to improve gray world algorithm. First
the proposed sand dust color correction subnet (SDCC) is used to correct the color cast of the sand dust image. The sand dust image is de-composed into 3 channels of R
G
and B. For each channel
a convolutional layer with a convolution kernel size of 3 is used to conducted
and each feature map is processed to obtain color correction image via gray world algorithm. To enhance the sand dust images quality
a benched color-corrected image is transmitted into the dust removal enhancement subnet in the context of adaptive instance normalized-residual blocks (AIN-ResBlock). The dust removal enhancement subnet takes the sand image and the color correction image as input
and uses the adaptive instance normalization module to adaptively restore the color distortion issues in the feature mapping in the dust removal enhancement subnet
and realizes the image sand removal through the residual block. Our AIN-ResBlock is capable to resolve the blurred details and missing image content for the natural color factors of the restored image. Additionally
in view of the difficulty in obtaining pairs of sand dust images and their corresponding clear images as training samples for deep learning
a sand dust image synthesis method is illustrated based on a physical imaging model. Absorb and scatter light to attenuate
and the attenuation degree of light of different colors is different. We optioned 15 color marks close to the color of the sand dust image
and simulate the sand dust image under 15 different conditions
and a large-scale dataset of clear image and sand dust images is finally constructed. Our loss function used is composed of
$$ L_1$$
loss function
perceptual loss function and gradient loss function in training the network. In order to validate the targeted image ground truth
we use
$$ L_1$$
loss in color correction subnet and a dust removal enhancement subnet; A perceptual loss is used to narrow the difference between the perceptual features of the sand dust image enhancement network results and the perceptual features of the real image; In order to better restore the details and structure of the image
we use the horizontal and vertical gradient loss in the network function.
Result
2
The performance of the method is verified by synthetic images and real images. The experimental results illustrated that our sand dust image enhancement method can remove the color cast and dust of the sand dust image
and obtain normal visual colors and details clear image. The performance of our method is based on the peak signal-to-noise ratio (PSNR)
structural similarity (SSIM)
natural image quality evaluator (NIQE)
perception-based image quality evaluator (PIQE)
blind/referenceless image spatial quality evaluator (BRISQUE)
the percentage of newly visible edges
$$ e$$
the contrast restoration quality
$$ \bar{γ}$$
and saturation
$$ σ$$
are estimated each. Compared with the existing methods
our method obtains the highest average PSNR and average SSIM on the composite image
which are 18.705 7 dB and 0.669 5 respectively. Our method can also significantly improve the quality of the enhanced image on real sand dust images
and obtain enhanced images with good visual effects.
Conclusion
2
We propose a CNN-based enhancement method for sand dust images with color restoration. This method can restore the color cast of the sand dust image
improve the contrast of the image
restore the detailed information of the image
and obtain normal visual colors and clear details image. The synthetic sand dust image and the real sand dust image have their priorities in visual effects and facilitate quantitative evaluation indicators structure further.
沙尘图像沙尘图像增强颜色校正自适应实例归一化残差块合成沙尘图像数据集
sand dust imagesand dust image enhancementcolor correctionadaptive instance normalization residual blocksynthetic sand dust image dataset
Cai B L, Xu X M, Jia K, Qing C M and Tao D C. 2016. Dehazenet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11): 5187-5198 [DOI: 10.1109/TIP.2016.2598681]
Chen D D, He M M, Fan Q N, Liao J, Zhang L H, Hou D D, Yuan L and Hua G. 2019. Gated context aggregation network for image dehazing and deraining//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE: 1375-1383 [DOI: 10.1109/WACV.2019.00151http://dx.doi.org/10.1109/WACV.2019.00151]
Dudhane A, Aulakh H S and Murala S. 2019. RI-GAN: an end-to-end network for single image haze removal//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, USA: IEEE: 2014-2023 [DOI: 10.1109/CVPRW.2019.00253http://dx.doi.org/10.1109/CVPRW.2019.00253]
Engin D, Genc A and Ekenel H K. 2018. Cycle-Dehaze: enhanced cyclegan for single image dehazing//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City, USA: IEEE: 938-946 [DOI: 10.1109/CVPRW.2018.00127http://dx.doi.org/10.1109/CVPRW.2018.00127]
Fu H, Gong M M, Wang C H, Batmanghelich K and Tao D C. 2018. Deep ordinal regression network for monocular depth estimation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 2002-2011 [DOI: 10.1109/CVPR.2018.00214http://dx.doi.org/10.1109/CVPR.2018.00214]
Fu X Y, Huang Y, Zeng D L, Zhang X P and Ding X H. 2014. A fusion-based enhancing approach for single sandstorm image//Proceedings of 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP). Jakarta, Indonesia: IEEE: 1-5 [DOI: 10.1109/MMSP.2014.6958791http://dx.doi.org/10.1109/MMSP.2014.6958791]
Gao H, Wei P and Ke J. 2015. Color enhancement and image defogging in HSI based on Retinex model//Proceedings of the SPIE 9622, 2015 International Conference on Optical Instruments and Technology. Beijing, China: SPIE: #962203 [DOI: 10.1117/12.2193264http://dx.doi.org/10.1117/12.2193264]
Guo T T, Li X L, Cherukuri V and Monga V. 2019. Dense scene information estimation network for dehazing//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, USA: IEEE: 2122-2130 [DOI: 10.1109/CVPRW.2019.00265http://dx.doi.org/10.1109/CVPRW.2019.00265]
He K M, Sun J and Tang X O. 2009. Single image haze removal using dark channel prior//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE: 1956-1963 [DOI: 10.1109/CVPR.2009.5206515http://dx.doi.org/10.1109/CVPR.2009.5206515]
Huang S C, Chen B H and Wang W J. 2014. Visibility restoration of single hazy images captured in real-world weather conditions. IEEE Transactions on Circuits and Systems for Video Technology, 24(10): 1814-1824 [DOI: 10.1109/TCSVT.2014.2317854]
Jobson D J, Rahman Z and Woodell G A. 1997. A multiscale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7): 965-976 [DOI: 10.1109/83.597272]
Kwon K J and Kim Y H. 2012. Scene-adaptive RGB-to-RGBW conversion using retinex theory-based color preservation. Journal of Display Technology, 8(12): 684-694 [DOI: 10.1109/JDT.2012.2215954]
Li B Y, Peng X L, Wang Z Y, Xu J Z and Feng D. 2017. AOD-Net: all-in-one dehazing network//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE: 4780-4788 [DOI: 10.1109/ICCV.2017.511http://dx.doi.org/10.1109/ICCV.2017.511]
Liu H, Li C, Wan Y Q and Zhang Y C. 2017. Dust image enhancement algorithm based on color transfer//Proceedings of the 2nd CCF Chinese Conference on Computer Vision. Tianjin, China: Springer: 168-179 [DOI: 10.1007/978-981-10-7299-4_14http://dx.doi.org/10.1007/978-981-10-7299-4_14]
Pizer S M, Johnston R E, Ericksen J P, Yankaskas B C and Muller K E. 1990. Contrast-limited adaptive histogram equalization//Proceedings of the 1st Conference on Visualization in Biomedical Computing. Atlanta, USA: IEEE: 337-345 [DOI: 10.1109/VBC.1990.109340http://dx.doi.org/10.1109/VBC.1990.109340]
Qu Y Y, Chen Y Z, Huang J Y and Xie Y. 2019. Enhanced pix2pix dehazing network//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 8152-8160 [DOI: 10.1109/CVPR.2019.00835http://dx.doi.org/10.1109/CVPR.2019.00835]
Reinhard E, Adhikhmin M, Gooch B and Shirley P. 2001. Color transfer between images. IEEE Computer Graphics and Applications, 21(5): 34-41 [DOI: 10.1109/38.946629]
Ren W Q, Liu S, Zhang H, Pan J S, Cao X C and Yang M H. 2016. Single image dehazing via multi-scale convolutional neural networks//Proceedings of the 14th European Conference on Computer Vision (ECCV). Amsterdam, the Netherlands: Springer: 154-169 [DOI: 10.1007/978-3-319-46475-6_10http://dx.doi.org/10.1007/978-3-319-46475-6_10]
Rizzi A, Gatta C and Marini D. 2002. Color correction between gray world and white patch//Proceedings of the SPIE 4662, Human Vision and Electronic Imaging VII. San Jose, USA: SPIE: 367-375 [DOI: 10.1117/12.469534http://dx.doi.org/10.1117/12.469534]
Shi Z H, Feng Y N, Zhao M H, Zhang E H and He L F. 2019. Let you see in sand dust weather: a method based on halo-reduced dark channel prior dehazing for sand-dust image enhancement. IEEE Access, 7: 116722-116733 [DOI: 10.1109/ACCESS.2019.2936444]
Tan R T. 2008. Visibility in bad weather from a single image//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE: 1-8 [DOI: 10.1109/cvpr.2008.4587643http://dx.doi.org/10.1109/cvpr.2008.4587643]
Wang J, Pang Y W, He Y Q and Liu C S. 2016. Enhancement for dust-sand storm images//Proceedings of the 22nd International Conference on Multimedia Modeling. Miami, USA: Springer: 842-849 [DOI: 10.1007/978-3-319-27671-7_70http://dx.doi.org/10.1007/978-3-319-27671-7_70]
Xu G, Su J, Pan H D, Zhang Z G and Gong H B. 2009. An image enhancement method based on gamma correction//Proceedings of the 2nd International Symposium on Computational Intelligence and Design. Changsha, China: IEEE: 60-63 [DOI: 10.1109/ISCID.2009.22http://dx.doi.org/10.1109/ISCID.2009.22]
Yang X T, Xu Z and Luo J B. 2018. Towards perceptual image dehazing by physics-based disentanglement and adversarial training//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. New Orleans, USA: AAAI: 7485-7492
Yu S Y, Zhu H, Wang J, Fu Z F, Xue S and Shi H. 2016. Single sand-dust image restoration using information loss constraint. Journal of Modern Optics, 63(21): 2121-2130 [DOI: 10.1080/09500340.2016.1184340]
Zhang H and Patel V M. 2018. Densely connected pyramid dehazing network//Proceedings of 2018 IEEE/CVF conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 3194-3203 [DOI: 10.1109/CVPR.2018.00337http://dx.doi.org/10.1109/CVPR.2018.00337]
Zhang J and Tao D C. 2020. FAMED-Net: A fast and accurate multiscale end-to-end dehazing network. IEEE Transactions on Image Processing, 29: 72-84 [DOI: 10.1109/TIP.2019.2922837]
Zhi N, Mao S J and Li M. 2016. Visibility restoration algorithm of dust-degraded images. Journal of Image and Graphics, 21(12): 1585-1592
智宁, 毛善君, 李梅. 2016. 沙尘降质图像清晰化算法. 中国图象图形学报, 21(12): 1585-1592 [DOI: 10.11834/jig.20161203]
Zhou R Z, He J and Hong Z L. 2005. Adaptive algorithm of auto white balance for digital camera. Journal of Computer-Aided Design and Computer Graphics, 17(3): 529-533
周荣政, 何捷, 洪志良. 2005. 自适应的数码相机自动白平衡算法. 计算机辅助设计与图形学学报, 17(3): 529-533
Zhu Q S, Mai J M and Shao L. 2015. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 24(11): 3522-3533 [DOI: 10.1109/TIP.2015.2446191]
相关文章
相关作者
相关机构
京公网安备11010802024621