浏览全部资源
扫码关注微信
收稿日期：2021-02-08，
修回日期：2021-03-25，
录用日期：2021-4-1，
纸质出版日期：2022-05-16
移动端阅览
目的
2
由于海水中悬浮的颗粒会吸收和散射光，并且不同波长的光在海水中的衰减程度也不同，使得水下机器人拍摄的图像呈现出对比度低、颜色失真等问题。为解决上述问题以呈现出自然清晰的水下图像，本文提出了基于神经网络的多残差联合学习的方法来对水下图像进行增强。
方法
2
该方法包括3个模块：预处理、特征提取和特征融合。首先，采用Sigmoid校正方法对原始图像的对比度进行预处理，增强失真图像的对比度，得到校正后的图像；然后，采用双分支网络对特征进行提取，将原始图像送入分支1——残差通道注意分支网络，将校正后的图像与原始图像级联送入分支2——残差卷积增强分支网络。其中，通道注意分支在残差密集块中嵌入了通道注意力模型，通过对不同通道的特征重新进行加权分配，以加强有用特征；卷积增强分支通过密集级联和残差学习，提取校正图像中边缘等高频信息以保持原始结构与边缘。最后，在特征融合部分，将以上双分支网络的特征级联后，通过残差学习进一步增强；增强后的特征与分支1的输出、分支1与分支2的输出分别经过自适应掩膜进行再次融合。选取通用UIEB(underwater image benchmark dataset)数据集中的800幅水下图像作为训练集进行训练，设计了结合图像内容感知、均方误差、梯度和结构相似性的联合损失函数对网络进行端到端训练。
结果
2
在通用UIEB数据集上选取非训练集中有参考图像的90幅图像与当前10种方法进行测试。结果显示，本文方法增强后的水下图像的PSNR(peak signal-to-noise ratio)指标平均达到了20.739 4 dB，SSIM(structural similarity)为0.876 8，UIQM(underwater image quality measure)为3.183 3，均高于对比方法。
结论
2
本文方法不仅在客观质量上超越了对比方法，且在主观质量上也显著提高了对比度，能够产生颜色丰富并且清晰度较高的增强图像。尤其是对于深水场景中偏蓝的水下图像，本文方法获得显著的质量提升。
Objective
2
Suspended particles will absorb and scatter light in seawater. It tends to be low contrast and intensive color distortion due to the attenuated multi-wavelengths for underwater robots based images capturing. Deep learning based underwater image enhancement is facilitated to resolve this issue of unclear acquired images recently. Compared to the traditional methods
convolutional neural network(CNN) based underwater image processing has its priority in quality improvement qualitatively and quantitatively. A basic-based method generates its relevant high-quality images via a low-quality underwater image processing. The single network is challenging to obtain multi-dimensional features. For instance
the color is intensively biased in some CNN-based methods and the enhanced images are sometimes over-smoothed
which will have great impact on the following computer vision tasks like objective detection and segmentation. Some networks utilize the depth maps to enhance the underwater images
but it is a challenge to obtain accurate depth maps.
Method
2
A multi-residual learning network is demonstrated to enhance underwater images. Our framework is composed of 3 modules in the context of pre-processing
feature extraction
and feature fusion. In the pre-processing module
the sigmoid correction is handled to pre-process the contrast of the degraded image
producing a strong contrast modified image. Next
a two-branch network is designated for feature extraction. The degraded image is fed into branch 1
i.e.
the residual channel attention branch network where channel attention modules are incorporated to emphasize the efficient features. Meanwhile
the modified image and the degraded image are concatenated and transferred to branch 2
i.e.
the residual enhancement branch network where structures and edges of the images are preserved and enhanced based on stacked residual blocks and dense connection. Finally
the feature fusion module conducts generated features collection for further fusion. The generated features penetrate into several residual blocks for enhancement based on feature extraction modules. An adaptive mask is then applied to the enhanced features and the outputs of branch 1. Similarly
the outputs of branch 1 and branch 2 are also processed by the adaptive mask. At the end
all results from the masks are added for final image reconstruction. To train our network
we introduce a loss function that includes content perception
mean square error
gradient
and structural similarity (SSIM) for end-to-end training.
Result
2
We use 800 images as the training dataset based on the regular underwater image benchmark dataset (UIEB) and 90 UIEB-based referenced images as the test dataset. Our proposed scheme is tailored to underwater image enhancement methods
including the model-free methods
the physical model-based methods
as well as the deep learning-based methods. Two reference indexes including peak signal-to-noise ratio (PSNR) and SSIM
and a non-reference index underwater image quality measure (UIQM) are employed to evaluate the enhancement results. The demonstrated results illustrate that our method achieves averaged PSNR of 20.739 4 dB
averaged SSIM of 0.876 8
and averaged UIQM of 3.183 3. Specifically
our method plays well in UIQM
which is an essential underwater image quality measurement index and reflects the color
sharpness
and contrast of underwater images straightforward. Moreover
our method achieves the smallest standard deviation of all compared methods
indicating the robustness of our proposed method.
Conclusion
2
We visualize the enhanced underwater images for qualitative evaluation. Our analyzed results improve the color and contrast and preserve rich textures for the enhanced images with no over-exposing and over-smoothing effect. Our method is beneficial to blueish images quality improvement in deep water scenarios.
Ancuti C, Ancuti C O, Haber T and Bekaert P. 2012. Enhancing underwater images and videos by fusion//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE: 81-88 [ DOI: 10.1109/CVPR.2012.6247661 http://dx.doi.org/10.1109/CVPR.2012.6247661 ]
Anwar S, Li C Y and Porikli F. 2018. Deep underwater image enhancement [EB/OL]. [2021-01-23] . https://arxiv.org/pdf/1807.03528.pdf https://arxiv.org/pdf/1807.03528.pdf
Bekerman Y, Avidan S and Treibitz T. 2020. Unveiling optical properties in underwater images//Proceedings of 2020 IEEE International Conference on Computational Photography (ICCP). St. Louis, USA: IEEE: 1-12 [ DOI: 10.1109/iccp48838.2020.9105267 http://dx.doi.org/10.1109/iccp48838.2020.9105267 ]
Berman D, Treibitz T and Avidan S. 2017. Diving into haze-lines: color restoration of underwater images//Proceedings of the British Machine Vision Conference (BMVC). Londaon, UK: BMVA Press, 1(2): 1-12[ DOI: 10.52441C.31.44 http://dx.doi.org/10.52441C.31.44 ]
Berman D, Levy D, Avidan S and Treibitz T. 2021. Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE transactions on pattern analysis and machine intelligence, 43(8): 2822-2837 [DOI: 10.1109/TPAMI.2020.2977624]
Buchsbaum G. 1980. A spatial processor model for object colour perception. Journal of the Franklin Institute, 310(1): 1-26 [DOI: 10.1016/0016-0032(80)90058-7]
Chen X Y, Yu J Z, Kong S H, Wu Z X, Fang X and Wen L. 2020. Towards quality advancement of underwater machine vision with generative adversarial networks [EB/OL]. [2021-01-23] . https://arxiv.org/pdf/1712.00736.pdf https://arxiv.org/pdf/1712.00736.pdf
Drews P L J, Nascimento E R, Botelho S S C and Campos M F M. 2016. Underwater depth estimation and image restoration based on single images. IEEE Computer Graphics and Applications, 36(2): 24-35 [DOI: 10.1109/MCG.2016.26]
Fabbri C, Islam M J and Sattar J. 2018. Enhancing underwater imagery using generative adversarial networks//Proceedings of 2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane, Australia: IEEE: 7159-7165 [ DOI: 10.1109/ICRA.2018.8460552 http://dx.doi.org/10.1109/ICRA.2018.8460552 ]
Galdran A, Pardo D, Picón A and Alvarez-Gila A. 2015. Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 26: 132-145 [DOI: 10.1016/j.jvcir.2014.11.006]
Guo Y C, Li H Y and Zhuang P X. 2020. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 45(3): 862-870 [DOI: 10.1109/JOE.2019.2911447]
He K M, Sun J and Tang X O. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2341-2353 [DOI: 10.1109/TPAMI.2010.168]
Hou M J, Liu R S, Fan X and Luo Z X. 2018. Joint residual learning for underwater image enhancement//Proceedings of the 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece: IEEE: 4043-4047 [ DOI: 10.1109/ICIP.2018.8451209 http://dx.doi.org/10.1109/ICIP.2018.8451209 ]
Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 7132-7141 [ DOI: 10.1109/CVPR.2018.00745 http://dx.doi.org/10.1109/CVPR.2018.00745 ]
Hummel R. 1977. Image enhancement by histogram transformation. Computer Graphics and Image Processing, 6(2): 184-195 [DOI: 10.1016/S0146-664X(77)80011-7]
Iqbal K, Odetayo M, James A, Salam R A and Talib A Z H. 2010. Enhancing the low quality images using unsupervised colour correction method//Proceedings of 2010 IEEE International Conference on Systems, Man and Cybernetics. Istanbul, Turkey: IEEE: 1703-1709 [ DOI: 10.1109/ICSMC.2010.5642311 http://dx.doi.org/10.1109/ICSMC.2010.5642311 ]
Jaffe J S. 1990. Computer modeling and the design of optimal underwater imaging systems. IEEE Journal of Oceanic Engineering, 15(2): 101-111 [DOI: 10.1109/48.50695]
Li C Y, Guo C L, Ren W Q, Cong R M, Hou J H, Kwong S and Tao D C. 2020. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29: 4376-4389 [DOI: 10.1109/TIP.2019.2955241]
Li C Y, Guo J C, Chen S J, Tang Y B, Pang Y W and Wang J. 2016a. Underwater image restoration based on minimum information loss principle and optical properties of underwater imaging//Proceedings of 2016 IEEE International Conference on Image Processing (ICIP). Phoenix, USA: IEEE: 1993-1997 [ DOI: 10.1109/ICIP.2016.7532707 http://dx.doi.org/10.1109/ICIP.2016.7532707 ]
Li C Y, Guo J C, Cong R M, Pang Y W and Wang B. 2016b. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Transactions on Image Processing, 25(12): 5664-5677 [DOI: 10.1109/TIP.2016.2612882]
Li C Y, Quo JC, Pang Y W, Chen S J and Wang J. 2016c. Single underwater image restoration by blue-green channels dehazing and red channel correction//Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Shanghai, China: IEEE: 1731-1735 [ DOI: 10.1109/ICASSP.2016.7471973 http://dx.doi.org/10.1109/ICASSP.2016.7471973 ]
Li J, Skinner K A, Eustice R M and Johnson-Roberson M. 2018. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 3(1): 387-394 [DOI: 10.1109/LRA.2017.2730363]
Liu R S, Fan X, Hou M J, Jiang Z Y, Luo Z X and Zhang L. 2019. Learning aggregated transmission propagation networks for haze removal and beyond. IEEE Transactions on Neural Networks and Learning Systems, 30(10): 2973-2986 [DOI: 10.1109/TNNLS.2018.2862631]
Liu Y C, Chan W H and Chen Y Q. 1995. Automatic white balance for digital still camera. IEEE Transactions on Consumer Electronics, 41(3): 460-466 [DOI:10.1109/30.468045]
Lu H M, Li Y J, Zhang L F and Serikawa S. 2015. Contrast enhancement for images in turbid water. Journal of the Optical Society of America A, 32(5): 886-893 [DOI: 10.1364/JOSAA.32.000886]
McGlamery B L. 1980. A computer model for underwater camera systems//Proceedings Volume 0208, Ocean Optics Ⅵ. Monterey: SPIE: 221-231 [ DOI: 10.1117/12.958279 http://dx.doi.org/10.1117/12.958279 ]
Nicholas C B, Anush M and Ryan M E. 2010. Initial results in underwater single image dehazing//Oceans 2010 Mts/IEEE Seattle. Seattle, USA: IEEE: 1-8 [ DOI: 10.1109/OCEANS.2010.5664428 http://dx.doi.org/10.1109/OCEANS.2010.5664428 ]
Panetta K, Gao C and Agaian S. 2016. Human-visual-system-inspired underwater image quality measures. IEEE Journal of Oceanic Engineering, 41(3): 541-551 [DOI: 10.1109/JOE.2015.2469915]
Peng Y T and Cosman P C. 2017. Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing, 26(4): 1579-1594 [DOI: 10.1109/TIP.2017.2663846]
Pizer S M, Johnston R E, Ericksen J P, Yankaskas B C and Muller K E. 1990. Contrast-limited adaptive histogram equalization: speed and effectiveness//Proceedings of the 1st Conference on Visualization in Biomedical Computing. Atlanta, Georgia: IEEE: 337-345 [ DOI: 10.1109/VBC.1990.109340 http://dx.doi.org/10.1109/VBC.1990.109340 ]
Singh G, Jaggi N, Vasamsetti S, Sardana H K, Kumar S and Mittal N. 2015. Underwater image/video enhancement using wavelet based color correction (WBCC) method//Proceedings of 2015 IEEE Underwater Technology (UT). Chennai, India: IEEE: 1-5 [ DOI: 10.1109/UT.2015.7108303 http://dx.doi.org/10.1109/UT.2015.7108303 ]
Song W, Wang Y, Huang D M and Tjondronegoro D. 2018. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration//Proceedings of the 19th Pacific-Rim Conference on Multimedia. Hefei, China: Springer: 678-688 [ DOI: 10.1007/978-3-030-00776-8_62 http://dx.doi.org/10.1007/978-3-030-00776-8_62 ]
Tang X Y, Li M, Xu L L, Hao Z and Zhang X W. 2020. Underwater image enhancement based on turbulence model corrected by transmittance and dynamically adjusted retinex. Journal of Image and Graphics. 25(7): 1380-1392
汤新雨, 李敏, 徐灵丽, 郝真, 张学武. 2020. 基于透射率修正的湍流模型与动态调整retinex的水下图像增强. 中国图象图形学报, 25(7): 1380-1392 [DOI: 10.11834/jig.190482]
Wang N, Zhou Y B, Han F L, Zhu H T and Yao J Z. 2021. UWGAN: underwater GAN for real-world underwater color restoration and dehazing [EB/OL]. [2021-01-23] . https://arxiv.org/pdf/1912.10269.pdf https://arxiv.org/pdf/1912.10269.pdf
Wen H C, Tian Y H, Huang T J and Gao W. 2013. Single underwater image enhancement with a new optical model//Proceedings of 2013 IEEE International Symposium on Circuits and Systems (ISCAS). Beijing, China: IEEE: 753-756 [ DOI: 10.1109/ISCAS.2013.6571956 http://dx.doi.org/10.1109/ISCAS.2013.6571956 ]
Yang Y Z and Lu H. 2019. Single image deraining using a recurrent multi-scale aggregation and enhancement network//Proceedings of 2019 IEEE International Conference on Multimedia and Expo (ICME). Shanghai, China: IEEE: 1378-1383 [ DOI: 10.1109/ICME.2019.00239 http://dx.doi.org/10.1109/ICME.2019.00239 ]
You Q J, Wan C, Sun J, Shen J X, Ye H and Yu Q L. 2019. Fundus image enhancement method based on CycleGAN//Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Berlin, Germany: IEEE: 4500-4503 [ DOI: 10.1109/EMBC.2019.8856950 http://dx.doi.org/10.1109/EMBC.2019.8856950 ]
Yu D B, Huo G Y, Liu Y, Zhou Y and Xu J X. 2019. Underwater image restoration based on red channel and haze-lines prior//Proceedings of the 12th International Conference on Intelligent Robotics and Applications. Shenyang, China: Springer: 148-158 [ DOI: 10.1007/978-3-030-27532-7_13 http://dx.doi.org/10.1007/978-3-030-27532-7_13 ]
Zhang C, Yan Q S, Zhu Y, Li X J, Sun J Q and Zhang Y N. 2020. Attention-based network for low-light image enhancement//Proceedings of 2020 IEEE International Conference on Multimedia andExpo (ICME). London, UK: IEEE: 1-6 [ DOI: 10.1109/ICME46284.2020.9102774 http://dx.doi.org/10.1109/ICME46284.2020.9102774 ]
Zhang Y L, Li K P, Li K, Wang L C, Zhong B N and Fu Y. 2018. Image super-resolution using very deep residual channel attention networks//Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer: 294-310 [ DOI: 10.1007/978-3-030-01234-2_18 http://dx.doi.org/10.1007/978-3-030-01234-2_18 ]
相关作者
相关机构
京公网安备11010802024621